Enterprise Architect. User Guide Series. Testpoints. Author: Sparx Systems. Date: 30/06/2017. Version: 1.0 CREATED WITH

Transcription

1 Enterprise Architect User Guide Series Testpoints Author: Sparx Systems Date: 30/06/2017 Version: 1.0 CREATED WITH

2 Table of Contents Testpoints 3 Test Domain Diagram 7 Test Cut 9 Test Set 10 Test Suite 11 The Testpoints Window 12 Testpoints Toolbar 14 Testpoint Editor 16 Testpoint Constraints 18

3 Testpoints Testpoints present a scheme by which constraints and rules governing the behavior of objects can be taken from the model and applied to one or more applications. The advantages schemes such as this offer are tolerance to code changes - adding and subtracting lines from a function has no effect on the constraints that govern it. Another advantage is that changes to the behavioral rules do not require a corresponding change to any source code; meaning nothing has to be re-compiled! Also, the ability to verify multiple applications using a single test domain is a simple rather than onerous matter. The Test Domain is a both a logical and relational model; constraints in the Class model can be partitioned with Test Cuts. These can be aggregated simply into Test Sets and Test Suites using connectors. Due to the decoupling of the Test Domain from the codebase, it is a simple choice of buttons to run a program normally, or run it for a specific Test Domain. This system also delivers practical benefits in that no instrumentation is required at all. Test results are displayed in the report window during the run, in real-time, as the program runs. These results can be retained, and reviewed at any time in the Test management window Alt+3 or using EA's documentation features. Features Feature Testpoint Composition Details Testpoint composition is performed using the Testpoint Window. The Testpoint Window is context-sensitive and displays the Test Domain for the selected element in either the Project Browser or diagram. Selecting a single Class will display the Class structure. A 'pencil' icon is displayed against Classes and methods that have existing constraints. When you select a Test Cut, Set or Suite Test, the Testpoint window displays the entire Domain structure including all the Classes that make up the domain. Note: You can navigate the domain hierarchy using the 'Navigation' pane on the right. Testpoints are composed as expressions using the variable names of the Class members. The intelli-sense shortcut Ctrl+Space is available within the editor to help you find these. Expressions that evaluate to True are taken to mean a pass. Returning False is taken to mean a fail. (c) Sparx Systems Page 3 of 21 Created with Enterprise Architect

4 You can add or edit an existing Invariant by double-clicking the Class. You can add or edit an existing pre- or post-condition similarly by double-clicking the method. Double-clicking a Testpoint will automatically display the source code if it is available. Line conditions are best added from within the code editor using its shortcut menus. This image is of a pre-condition in the above Test domain. Testpoint Trace Statements Test Domain Composition Each Testpoint can have its own Trace statement. The Trace statement is a dynamic message that can reference variables in its object or local scope. They are output during the evaluation of a test. They can be configured to be output every time a constraint is evaluated, or more usually when a test has failed. Trace statements can be directed to the 'Testpoints' tab of the System Output Window, or to an external file. You can configure this in any Analyzer Script. The Test Domain diagram is a dynamic medium where Testpoints are assembled to test Use Cases. Use Cases in a Test Domain diagram are provided in three different (c) Sparx Systems Page 4 of 21 Created with Enterprise Architect

5 stereotypes: Test Cut, Test Set and Test Suite. Management of the domain is as easy as modeling on any diagram. The toolbox and shortcut menus provide access to any Test Domain Artifacts. In brief, Testpoints from multiple Classes are aggregated into Test Sets. Test Sets are then linked to form Test Suites. Both Test Cuts and Test Sets are re-usable assets. Linking the same Test Set to the one or more Test Suites is a matter of drawing connectors. «testset» Sell Stock «testcut» Sell Account «testcut» Sell Broker «testcut» Sell Stock Account Broker Stock Test Domain and the Class Model Rarely would a Use Case involve all the methods of a single Class. Most likely it is realized using a variety of methods from collaborating Classes. We call this subset of methods a cut and the Test Cut Artifact is the tool we use to make these cuts. The Testpoint Window will adapt depending on the context, be that a Test Domain or Class element. This image shows the Testpoint window when a Test Cut has been selected. Note the checkboxes. These are only visible for a Test Cut. They denote the methods (Test Cut) which are contributing to a Test Set. In this example the Test domain was generated by the Execution Analyzer, which did the method identification work for us. Testpoint Evaluation The Testpoint window is used to evaluate Test domains. The window has a toolbar for starting or attaching to the target application. The domain to test is always reflected by the element that has context, so if you select a Class the window will show only the Class structure and Testpoints of that Class. If you select a Test Suite, the window will display the entire domain hierarchy and all the Testpoints included in it. Clicking on the Run button will load the Testpoint domain in the Execution Analyzer, which will then evaluate, collect and update the report window as Use Cases pass or fail each test. The exact details of each constraint type and the (c) Sparx Systems Page 5 of 21 Created with Enterprise Architect

6 when and how of that constraint's capture are: A Class Invariant is evaluated by the Analyzer whenever any method called on an object of this Class type is completed; the invariant serves to test that the state of a complying object is both known and permitted Pre-conditions are evaluated immediately before an operation is called Post-conditions are evaluated (at the same time as a Class invariant) when the method is completed Line-conditions are evaluated if and when their specific line of code comes into scope during program execution (c) Sparx Systems Page 6 of 21 Created with Enterprise Architect

7 Test Domain Diagram The Test Domain diagram is the medium where you assemble and group test cases for a particular domain. An example of a Test domain might be 'Customer'. The breadth and depth of the domains you assemble is up to you. You might have separate domains for 'Add Customer' and 'Delete Customer', depending entirely on how you consider best to balance the domain hierarchy. The diagram toolbox and shortcut menu provide a number of Artifacts to help model the domain. Because the medium is dynamic, allowing you to revisit and build on relationships between Test domains, the system is a great model for delivering reusable assets to a organization that are low overhead and integrate with both the UML view of the world, and the Software Engineering nuts and bolts of daily life. Facilities Facility Test Domain Generation Details If you think the process of composing a Test Domain is complex, it can be, but help is at hand! The Execution Analyzer can produce a Test Domain diagram for you. It cannot write the Tests for you, but it can do some of the leg work. It can identify the Classes and pick out only those methods that participated in a Use Case. And this is not guesswork. The Analyzer Test Domain is obtained from a running program. This image shows the Test Domain generated by the Execution Analyzer from recording an Example Model program. «testset» Buy Stock «testcut» Buy Sector «testcut» Buy Broker «testcut» Buy Stock «testcut» Buy Account «testcut» Buy Exchange And this is the recording itself (as a Sequence diagram) from which the Test Domain was generated. Broker: Broker Exchange.Exchange Exchange.Stock: Stock Exchange.Account: Account Exchange.Sector 1.0 Buy(1, 2, 100): bool 1.1 FindAccount(1): Account* 1.2 FindStock(2): Stock* 1.3 GetStockVolume(): UINT 1.4 GetStockCost(): FLOAT 1.7 GetSector(): ESector 1.5 GetBalance(): INT 1.6 Credit(2, 100, - 100) 1.8 UpdateSector(secEquities, 100) 1.9 GetSector(secEquities): Sector* 1.10 Move (100) Sequence diagram generated in Enterprise Architect using recording marker in a Use Case The first task on a Test domain diagram is to create the Use Cases (Test Sets). (c) Sparx Systems Page 7 of 21 Created with Enterprise Architect

8 Test Domain Composition These define this particular domain's responsibility. The Diagram Toolbox and shortcut menu provide Artifacts to help you achieve this. The first of these is the Test Cut and this is used in the next step; identifying those methods (from the class model) that you consider to be participants in the Use Case. The Test Cut Artifact is useful because its allows us to partition a Class, selecting only those methods which are relevant. Test Cuts can be run individually or link to one or more Test Sets. Test Sets in turn can be linked to one or more Test Suites. In any case, any element of the Test domain tree can be run individually or as a whole. «testsuite» Trade Stock «testset» Buy Stock (from Buy) Test Set generated from recording history of Exchange:Buy Use Case «testset» Sell Stock (from Sell) Test Set generated from recording history of the Exchange::Sell Use Case (c) Sparx Systems Page 8 of 21 Created with Enterprise Architect

9 Test Cut Description A Test Cut element is a stereotyped Object element, used internal to Enterprise Architect for defining test sets using the Testpoint code testing facilities. A task, such as 'Print', might involve operations on different Classes. In order to create a 'Print' test, you would want to include only the 'Print' operations of these Classes and exclude any other operations. A Test Cut enables you to capture only the operations that represent the behavior (in this case, 'Print') defined for a single Class. You might then place the Test Cut from each of several Classes into a single task as a Test Set. When you drag a Test Cut element onto a Test Domain diagram, you create a Dependency relationship with the required Class element. As a result, when you select the Test Cut element on the Testpoints Window, the operations of the Class are listed in the window, each with a checkbox. You then select the checkbox against each Class operation to include in the Test Cut. Toolbox icon (c) Sparx Systems Page 9 of 21 Created with Enterprise Architect

10 Test Set Description A Test Set element is a stereotyped Use Case element used to aggregate one or more groups of methods (Test Cuts), which perhaps span multiple Classes, into a single task. Test Sets can also be aggregated into Test Suites. You link the Test Cut elements to the Test Set using Dependency connectors. Toolbox icon (c) Sparx Systems Page 10 of 21 Created with Enterprise Architect

11 Test Suite Description A Test Suite element is a stereotyped Use Case element, used to aggregate one or more groups of tasks (Test Sets). You link the Test Set elements to the Test Suite using Dependency connectors. Toolbox icon (c) Sparx Systems Page 11 of 21 Created with Enterprise Architect

12 The Testpoints Window The Testpoints Window is the hub where Test Domain constraints are composed. It is also the control that lets you verify a particular Test Domain on a program. The program might be already running or it can be launched using the control's Toolbar. Here you will also be able to see the results of your tests, as they happen. This control is context-sensitive, responding to the selection of elements in the Project Browser or on a diagram. Depending on the selection, tests can be carried out on a single class, a Use Case (Test Set) or a collection of Use Cases (A Test Suite). Access Ribbon Execute > Analyze > Testing > Show Testpoints Window Testpoint Window Columns Column Tests Usage Displays the name of the selected Testpoint object and the hierarchy of objects beneath it. The selected object can be a: Class Operation Test Cut Test Set or Test Suite Id For an Operation, this column shows a Testpoint marker icon ( ) when the Analyzer has successfully bound this operation in the target application. If no icon appears in this column during a run, it indicates that the model and code base might not be synchronized; perhaps the signature of the function has changed, or the operation is a new method you are working on, that exists in the source code but not yet in your model. For a Testpoint, this column shows a generated id number. This id number is used in trace output to indicate which constraint is being referenced. Constraints A pencil icon ( ) in this column indicates that one or more constraints are defined for this Class or Operation. Status During a test run, indicates these possible statuses: ( ) Failed - Constraint has evaluated as false one or more times. ( ) Invalid Statement - Constraint failed to parse due to invalid syntax. ( ) Variable not found - A referenced variable name was not found at the location where the constraint was evaluated. No icon is shown if a constraint has Passed. (c) Sparx Systems Page 12 of 21 Created with Enterprise Architect

13 Evals Passes Fails Last Run By Last Run Date Last Run Result Parent Collections Pane During a test run, indicates the number of times the Execution Analyzer has evaluated this constraint. During a test run, indicates the number of times the test passed. During a test run, indicates the number of times the test failed. Displays the username of the last person to run this test. Displays the date and time this test was last evaluated. Displays the result of the last test run. Lists any parent collections that include the selected object as part of their design. Double-click this collection to make it the selected object in the left pane. The Parent Collections pane can be hidden by clicking the Show / Hide Parent Collections pane button on the Testpoints Window Toolbar. (c) Sparx Systems Page 13 of 21 Created with Enterprise Architect

14 Testpoints Toolbar The Testpoints Window Toolbar provides options to execute configured tests on the currently selected Testpoint object, stop a test run currently in progress, filter the displayed items, and save the results of a completed test run. Access Ribbon Execute > Analyze > Testing > Show Testpoints Window Testpoints toolbar options Toolbar Button Action Field showing the name of the currently selected Testpoint object. Execute the test run. Stop the test run currently in progress. Toggle between showing all items and showing only those items that have constraints defined. Toggle between showing all items and showing only operations that have been marked for inclusion in this Test Cut; this button is only enabled when a Test Cut object is selected. When a Test Cut is selected, each of the operations of its associated Class are displayed with a checkbox; you use this checkbox to mark the operations that apply to this Test Cut. Click on the drop arrow next to this icon to display the 'Test Run Options' menu, providing these options: 'Prefix Trace output With Function Call' - Prefix all trace output lines with the executing function name 'Enable Standard Breakpoints during Testing' - When not checked, the test run ignores any breakpoints in the current breakpoint set, and any attempts to set breakpoints during the run are ignored 'View Trace output' - Display the 'Testpoints' tab of the System Output window Click on this icon after completion of a test run to save the results to Test item on the current object. Saved tests can be viewed using the Testing Workspace. A prompt displays to select the Test Class - Unit, Integration, System, Inspection, Acceptance or Scenario. Select the appropriate Test Class and click on the OK button. (c) Sparx Systems Page 14 of 21 Created with Enterprise Architect

15 Display the Testpoint Management Help topic. Show or hide the Parent Collections pane. (c) Sparx Systems Page 15 of 21 Created with Enterprise Architect

16 Testpoint Editor The Testpoint Editor is used to compose constraints for Classes and Operations. The types of constraints permitted are dependent on the selected object. For Classes, the type will always be Invariant. For operations, the type can be either Pre-Condition, Post-Condition or Line-Condition. Invariants are evaluated by the Analyzer when any method called on an object of the selected Class type completes. Pre-conditions are evaluated at the beginning of each call to the specified operation. Post-conditions are evaluated upon completion of each call to the specified operation. Line-conditions are evaluated each time the specified line of code is executed. Access Ribbon Execute > Analyze > Testing > Show Testpoints Window : Double-click on a Class or Operation in the Testpoints window Constraint Group fields Field Type Offset Condition Action on Fail Evaluate When Usage The type of constraint for the selected Class or Operation: Invariant - Evaluated after any method called on the specified Class has completed Pre-Condition - Evaluated at the beginning of each call to a specific Operation Post-Condition - Evaluated after completion of each call to a specific Operation Line-Condition - Evaluated upon execution of a specific line of code within an Operation For Line-Conditions only, the Line number within the specified operation upon which to evaluate the constraint. An offset value is automatically set if the Testpoint was created using the Code Editor context menu. The constraint to be evaluated when this Testpoint is triggered. A status of pass or fail will be recorded depending upon whether this constraint condition evaluates as true or false. Click on the drop-down arrow and select from the three options: 'Continue' - ignore failure of this constraint and continue execution 'Break execution' - halt execution and display the Stack trace 'Disable on fail' - do not execute the constraint again after failing once (Optional) An additional constraint which must be met before the main Testpoint Condition is evaluated, providing greater control over test coverage. (c) Sparx Systems Page 16 of 21 Created with Enterprise Architect

17 Trace Group fields Option Level Action Specifies when the trace statement (if defined) will be output. Available options are: 'Fail Only' - Output trace statement only when this Testpoint condition fails 'Always' - Output trace statement every time this Testpoint is evaluated Statement (Optional) A message to be output when this Testpoint is evaluated. Variables currently in scope can be included in a trace statement output by prefixing the variable name with a $ token for string variables, or token for primitive types such as int or long. Output from a Trace Statement can be directed either to the 'Testpoints' tab of the System Output Window, or to an external file, as configured by the Analyzer Script for the parent Package. (c) Sparx Systems Page 17 of 21 Created with Enterprise Architect

18 Testpoint Constraints A Constraint is typically composed using local and member variables in expressions, separated by operators to define one or more specific criteria that must be met. A constraint must evaluate as true to be considered as Passed. If a constraint evaluates as false, it is considered as Failed. Any variables referenced within the constraint must be in scope at the position where the Testpoint or Breakpoint is evaluated. General/Arithmetic Operators Operator Description + Add Example: a + b > 0 - Subtract Example: a - b > 0 / Divide Example: a / b == 2 * Multiply Example: a * b == c % Modulus Example: a % 2 == 1 () Parentheses - Used to define precedence in complex expressions. Example: ((a / b) * c) <= 100 [ ] Square Brackets - Used for accessing Arrays. Example: Names[0].Surname == "Smith". Dot operator - Used to access member variables of a Class. Example: Station.Name == "Flinders" -> Alternative notation for the Dot operator. Example: Station->Name == "Flinders" Comparison Operators Operator Description = Equal To (c) Sparx Systems Page 18 of 21 Created with Enterprise Architect

19 Example: a = b == Equal To Example: a == b!= Not Equal To Example: a!= b <> Not Equal To Example: a <> b > Greater Than Example: a > b >= Greater Than or Equal To Example: a >= b < Less Than Example: a < b <= Less Than or Equal To Example: a <= b Logical Operators Operator AND OR Description Logical AND Example: (a >= 1) AND (a <= 10) Logical OR Example: (a == 1) OR (b == 1) Bitwise Operators Operator & Description Bitwise AND Example: (1 & 1) = 1 (1 & 0) = 0 Bitwise OR Example: (1 1) = 1 (c) Sparx Systems Page 19 of 21 Created with Enterprise Architect

20 (1 0) = 1 ^ Bitwise XOR (exclusive OR) Example: (1 ^ 1) = 0 (1 ^ 0) = 1 Additional Examples Example ((m_nvalue & 0xFFFF0000) == 0) ((m_nvalue & 0x0000FFFF) == 0) m_value[0][1] = 2 a AND (b OR c) Description Use a Bitwise AND operator (&) with a hexadecimal value as the right operand to test that no bits are set in high order bytes of the variable. Use a Bitwise AND operator (&) with a hexadecimal value as the right operand to test that no bits are set in low order bytes of the variable. Accessing a multi-dimensional array Combining AND and OR operators, using parentheses to ensure precedence. In this example, variable 'a' must be true, and either 'b' or 'c' must be true. Notes String comparisons are case-sensitive (c) Sparx Systems Page 20 of 21 Created with Enterprise Architect

21 (c) Sparx Systems Page 21 of 21 Created with Enterprise Architect