Performing J Compliance Test with CANoe Version Application Note AN-ION

Transcription

1 Version Author(s) Restrictions Abstract Michalski, Jens Public This Application Note describes the method for performing a J Compliance Test with CANoe. It covers test preparation, test execution and evaluation of test results. Table of Contents 1.0 Motivation Test philosophy Test preparation Database Test environment Test generation Test execution Test results Individual extensions J1939 XML Test Module Manager Test Automation Editor TAE Additional Resources Contacts Motivation The quality of a network in heavy-duty vehicles is determined above all else by the interoperability of its individual nodes. The nodes must be properly linked to one another over the bus. The stated goal of the SAE J1939 protocol is to guarantee stable network communication and offer every individual ECU producer full flexibility for its application. To test J1939-compatible devices, SAE has provided a Conformance Test in the form of a single SAE J1939 document. The SAE J document merges all requirements for a conformant device, presenting them thematically in eight tables with over 150 test cases, and it provides the tester with a checklist-like form. Vector s CANoe.J1939 product offers automatic test generation and the means for interpreting these test cases. Generation of a test goes quickly with the help of a wizard, and it is easy to repeat the generated test at any time. Since a test run just takes a few minutes, the generated test is also excellent for integration in the development process. Any problems that occur can be detected and corrected long before network integration. Later network quality is sustainably improved. The purpose of this document is to offer an overview of the fundamental interpretation of the SAE J document by Vector and offers the user valuable practical tips on how to create the test. Fundamental information on tests in CANoe is contained in the Application Note AN-IND Testing with CANoe. 2.0 Test philosophy The tests generated by CANoe.J1939 are what are known as Black Box tests, i.e. the generator has no knowledge of the implementation details or the application behavior of the device under test (DUT). The tests are based on the information from the database and the SAE J1939 standard. In keeping with the Black Box concept, the signals of the J1939 messages are in principle not evaluated. One exception is the communication-relevant Copyright Vector Informatik GmbH Contact Information: or

2 data fields; they occur typically in Messages from the Data Link Layer (-21) and Network Management Layer (-81), e.g. the Request message (PGN 59904) or the Acknowledge message (PGN 59392). Since the standardized communication behavior is exclusively checked, the quality of the database is crucial for the quality of the generated tests. The database can be optimized in the test preparation phase with simple steps for test generation. 3.0 Test preparation Automatic test generation requires detailed information on the communication behavior of the DUT. It must be known which messages are sent or received by the ECU, and the cycle times of periodic messages must also be known. The advantage of this approach is that not all theoretically possible messages need to be checked, in order to check the behavior for the supported messages. This definition is made, as already mentioned, in the related database. 3.1 Database In CANoe, the database (.dbc file) serves to describe the communication behavior of an ECU. The generation of test cases is based to a great extent on this database and utilizes it as a reference during the test run. The information provided on the behavior of the DUT is crucial for optimal test generation and verification of the results. In preparing the database for a DUT, the following questions are helpful: Are all messages contained in the Tx and Rx lists? Are all system messages also contained in the Tx and Rx lists? (examples: TPCM, TPDT, ACL, CA, ACKM, RQST) Are the message attributes GenMsgSendType and GenMsgCycleTime correctly set? Is the full 64-bit NAME of the ECU stored in the database? These questions can be answered simply with the help of CANoe. Appendix A Preparation of the database gives tips on how to easily prepare the database. 3.2 Test environment In its specification, SAE describes a defined test environment consisting of tester and DUT. For most nodes this is an unfamiliar situation, because they rely on information from other network nodes to function. However, since the Vector J Compliance Test exclusively tests on the communication level and not the application level, the lack of stimulation should not present any problems. In addition, most test cases are conceived such that they cannot be influenced by other network nodes. If an ECU absolutely requires other nodes, they can also be operated in the network. Nonetheless, this should be considered in the analysis of test results. Side effects may occur which cause the test to fail, although the DUT behaved correctly. The tester usually communicates with source address 250. If this leads to conflicts, the source address can be replaced by any other desired address in the J1939 XML Test Module Manager. The tester does not participate in the network management process. It does not send any Address Claim message for its address and also does not respond to related Request messages. 2

3 4.0 Test generation Once all preparations have been completed, test generation can be started. The J1939 XML Test Module Manager is started in the CANdb++ Editor, and it appears in the popup menu of the DUT. The first time it is run, an assistant opens that guides the user in test generation. A static XML description of the test cases is generated, and so any changes to the database always require regeneration of the test cases. Figure 1: Display of the generated test cases in the J1939 XML Test Module Manager The test cases are structured in tables in the SAE J document, which is similar to sequential execution. However, they are self-contained units and can therefore also be executed individually. The test cases are set up in a modular toolkit approach, where individual elements can be rearranged into different combinations as desired to fulfill different purposes and tasks. All generated test cases are assembled from basic elements, and a description of the individual elements can be found in CANoe s online help under the key word Test Feature Set. The assistant generates a test group for each test table of the SAE J document, in which the individual lines of the tables are listed as test cases. The individual test steps and the pre- and post-processing of the test can also be done here (see section on Individual extensions). For an overview of the J test cases that can be generated and a detailed description of the individual test steps, see online help under the key word J Compliance Test. To learn more about the execution of the individual test steps of a test case, the title of the test case can be input as a search word in online help. 3

4 Figure 2: Online Help Description of individual test steps for a test case Some parameters, such as the source address of the tester and the time that should elapse between any two Request messages are saved as so-called Entities in the XML description. These values are only referenced in the individual test steps, while the assignment of values is made globally and can be changed for all tests simply and quickly. The values of entities can be changed conveniently in the settings of the J1939 XML Test Module Manager. 5.0 Test execution Basic elements of the generated test module are one or more test cases consisting of test conditions and test functions. Test conditions continually monitor the test flow, but they do not influence the communication. Test functions, on the other hand, actively intervene in communication. For example, they might send messages or wait for very specific events before other actions are executed. Test conditions are active over the entire life of the test case, while test functions can be executed sequentially. Just a few of the test cases require very significant testing depth, and this typically leads to very long execution times of the test cases. Especially if testing has been conducted regularly during development, the maximum test depth is not desired for every test run. In the test module, the user can choose between the variants FULL and FAST, where the latter variant uses ten parameter groups as a sampling of the overall population of parameter groups to be tested. The test case is then executed as a sample for these ten parameter groups, and this significantly shortens test execution time. To attain the testing depth required by SAE in the J document, the FULL variant must be selected. Here, the behavior of the DUT is checked for all theoretically possible parameter groups in some test cases, which lengthens the duration of the test. The test cases generated by Vector s CANoe.J1939 follow various strategies, but there are essentially three variants: Many test cases actively send a message to the DUT, e.g. Request, to test its reaction to it. This might be done to trigger transmission of a specific message by the DUT, for example. Some test cases monitor bus communication for a specific time period without actively intervening in the communication. If no faulty behavior is found over this time period, the test is passed. This also applies if no communication occurred at all. 4

5 Some test cases require user inputs or ask the user to actively influence the DUT. Under some circumstances, the user input can be removed from the test and could, for example, be substituted by a stimulation in preparation for the test case (see section on Individual extensions). 6.0 Test results The test results are provided as an XML file, from which an HTML report is automatically generated. The form of the report must be selected before test execution. Several XSLT stylesheets are available for selection in the tester configuration dialog. The J _ComplianceReport.xslt stylesheet generates a report, which is visually similar to the SAE J document and is therefore easy to interpret by all persons familiar with the standard. If detailed test results are needed, e.g. for analysis during product development, report variants are offered that provide significantly more information. Here, the extendednavigation.xslt stylesheet has proven to be very helpful in practice. 7.0 Individual extensions As already mentioned, the generated tests are what are referred to as Black Box tests. The test tool only needs a little information about the DUT. For some test cases, it might be necessary to put the DUT in a specific state. Here, it is possible to utilize CANoe s Test Feature Set for the necessary simulations; it is used to create all generated tests. For this case, the XML sections <preparation/> and <completion/> are intended for pre- and postprocessing. They are available in every individual test case and can be individually modified. The generated test module can also be supplemented with additional test cases, e.g. to test the application. The elements for this are also contained in the modular Test Feature Set and are easy to use. For a description of the available functions for stimulation or individual tests, refer to online help under the key word Test Feature Set. All test steps are described in a XML file. This file can be edited with any XML editor. However, this is more convenient with the Vector tools described below. 5

6 7.1 J1939 XML Test Module Manager The J1939 XML Test Module Manager, which is part of Option J1939, offers support for simple modifications and extensions to the test module. Automatic test cases are also generated in this editor. Figure 3: J1939 XML Test Module Manager Editing individual test steps 6

7 7.2 Test Automation Editor TAE The Test Automation Editor offers many different capabilities; it is especially easy to generate test cases by drag & drop operation and parameterize them with data from the database. The Test Automation Editor is not part of Option J1939. Figure 4: Test Automation Editor 2.0 Editing individual test steps 8.0 Additional Resources VECTOR APPLICATION NOTE AN-ION Introduction to J1939 AN-IND Testing with CANoe AN-AND CANoe Test Feature Set Tutorial 7

8 9.0 Contacts Germany and all countries not named below: Vector Informatik GmbH Ingersheimer Str Stuttgart GERMANY Phone: Fax: France, Belgium, Luxemburg: Vector France SAS 168 Boulevard Camélinat Malakoff FRANCE Phone: Fax: information@fr.vector.com Sweden, Denmark, Norway, Finland, Iceland: VecScan AB Theres Svenssons Gata Göteborg SWEDEN Phone: Fax: info@se.vector.com United Kingdom, Ireland: Vector GB Ltd. Rhodium, Central Boulevard Blythe Valley Park Solihull, Birmingham West Midlands B90 8AS UNITED KINGDOM Phone: Fax: info@uk.vector.com China: Vector Automotive Technology (Shanghai) Co., Ltd. Sunyoung Center Room 1701, No.398 Jiangsu Road Changning District Shanghai P.R. CHINA Phone: Fax: info@cn.vector.com India: Vector Informatik India Pvt. Ltd. 4/1/1/1, Sutar Icon, Sus Road, Pashan, Pune INDIA Phone: Fax: info@in.vector.com USA, Canada, Mexico: Vector CANtech, Inc Orchard Hill Place, Suite 550 Novi, MI USA Phone: Fax: info@us.vector.com Japan: Vector Japan Co. Ltd. Tennozu Yusen Bldg. 16F Higashi-shinagawa, Shinagawa-ku, Tokyo JAPAN Phone: Fax: info@jp.vector.com Korea: Vector Korea IT Inc. #1406, Mario Tower, Guro-dong, Guro-gu Seoul, REPUBLIC OF KOREA Phone: Fax: info@kr.vector.com 8

9 Appendix A Preparation of the database Are all messages contained in the Tx and Rx lists? In particular, the messages from the Data Link Layer (-21) and Network Management Layer (-81) as well as TPCM, TPDT, ACL, CA, ACKM and RQST. A filter in the Trace Window can filter all messages that are sent or received by a node. They can then be compared to the entries in the database. The fixed mode of the Trace Window offers a better overview than the continuous mode. all PDU1 messages with global destination address all PDU2 messages all PDU1 messages with destination address 0xB all messages with source address 0xB switches between fixed and continuous mode Figure 5: J1939 filter block and Trace Window Analyze communication of the DUT Are the message attributes GenMsgSendType and GenMsgCycleTime correctly set? When the node is selected in the CANdb++ Editor, all associated messages are shown in table format, and here it is possible to quickly get an overview of their attributes. Messages with alternating cycle times should be shown with cycle time 0. 9

10 Attribute GenMsgCycleTime Attribute GenMsgSendType Figure 6: CANdb++ Editor Showing the send type and cycle time of Tx messages Is the full 64-bit NAME of the ECU stored in the database? The 64-bit NAME of the ECU can be saved in the database. After selecting the associated ACL message, the signal values can be set. Figure 7: CANdb++ Editor Definition of individual 64-bit NAME of the DUT 10