A Flexible Integration Strategy for In-Car Telematics Systems

Transcription

1 A Flexible Interation Stratey for In-Car Telematics Systems Thomas Bauer Jens Herrmann Peter Liesmeyer Christopher Robinson-Mallett University of Potsdam Hasso-Plattner-Institute DaimlerChrysler AG Berlin University of Kaiserslautern Fraunhofer IESE University of Potsdam Hasso-Plattner-Institute ABSTRACT This paper presents an approach for the plannin of interation tests of automotive telematics systems. To our knowlede no method for the determination of an interation order exists that takes the project and the system environment into account, which in our opinion reatly influence the interation order. Furthermore, most known test eneration methods and structural quality measures demand syntactically sound specifications to be applied efficiently. In our projects Messae Sequence Charts are often created manually from the scratch with many different tools, and therefore they are of rather low syntactical quality. This paper addresses the determination of an interation stratey, which can easily be adapted to chanes in the project or in the system environment, and which can be manually applied to any iven specification. Cateories and Subject Descriptors D..5 Testin and Debuin, D.8 Metrics, D.9 Manaement. General Terms Manaement, Measurement, Reliability, Human Factors, Standardization, Verification.. INTRODUCTION The development of in-car telematics systems (fiure ) is usually a common task of a sinle system developer, i.e. the car manufacturer, and many external component suppliers. The development of the telematics components is based on requirements, which are defined in contracts between system developer and suppliers. These documents miht contain various document types, e.. functional and requirements specifications, statecharts, or messae sequence charts (MSCs). The interation and the implied interation testin of the externally developed components is achieved by the system developer. The interation test is based on specifications which describe the architecture of the telematics system, includin the interaction of the components. Typically, the internal structure and the behavior of the components are hidden to the system interator and the required component behavior is unit tested riht after delivery. Therefore, an interation test of a telematics system is usually based on requirements documents that mainly describe the component interaction and communication and which are in our case MSCs of varyin quality. Keywords Interation Testin, Automotive, Telematics, Test Plannin, Components, Messae Sequence Charts PERMISSION TO MAKE DIGITAL OR HARD COPIES OF ALL OR PART OF THIS WORK FOR PERSONAL OR CLASSROOM USE IS GRANTED WITHOUT FEE PROVIDED THAT COPIES ARE NOT MADE OR DISTRIBUTED FOR PROFIT OR COMMERCIAL ADVANTAGE AND THAT COPIES BEAR THIS NOTICE AND THE FULL CITATION ON THE FIRST PAGE. TO COPY OTHERWISE, OR REPUBLISH, TO POST ON SERVERS OR TO REDISTRIBUTE TO LISTS, REQUIRES PRIOR SPECIFIC PERMISSION AND/OR A FEE. ICSE SEAS 05, MAY, 005, ST. LOUIS, MISSOURI, USA. 005 ACM /05/ $5.00 Fiure : In-car telematics MOST network To our knowlede no sufficient solutions nor experiences on interation testin of in-car telematics systems, e.. info- and entertainment systems, on the basis of MSCs exist, which take the project and the system environment into account. This paper presents a flexible plannin method for interation tests of in-car telematics systems in an industrial environment that is based on environmental aspects, risk analysis and communication structure.

2 As the most important task durin interation of telematics systems the development of an interation stratey is closely examined. A sufficient interation stratey must be intuitively, easy to adapt to chanin requirements, and furthermore it has to represent the project situation and the telematics characteristics. Diverse testin methods must be chosen and applied in conjunction with the interation stratey. In each interation step and on each specification document a different testin method may be appropriately used for the construction of test cases. Therefore, this paper also aims at the measurement of the characteristics of specification documents used in the development of telematics systems. This paper is oranized in seven sections. In section related work is presented and the motivation of our work is explained in the presence of existin interation testin approaches. The basics of in-car telematics systems are iven in section 3. In section 4 a schema of an interation test plan is presented, which is structured into packaes to map the stepwise nature of interation testin. The development of the interation stratey is presented in section 5. This section contains three subsections for environment, risk, and structure analyses that are used to determine the interation order. The application of the interation order to the MSCs of a project and a concept for the construction of test-cases are presented in section 6. Finally, this work is concluded and future research topics are presented.. RELATED WORK The eneral use of messae sequence charts in softwaredevelopment is defined in two most important standards, the ITU Z.0 [] and the UML.0 Specification [3]. The Z.0 standard aims the area of communication networks where MSCs are used for the description of the interaction between distributed components. The UML specification describes MSCs with a more eneral purpose, but which are often associated with the description of object interaction. A detailed comparison of these standards can be found in [4]. The Testin and Test Control Notation, formerly referred to as Tree and Tabular Control Notation, in its latest version TTCN-3 was developed to allow the specification of detailed tests for communicatin systems [8]. The papers of Baker et. al. [5] and Ebner [6] are describin in detail the test-case eneration on the basis of MSCs for the use with TTCN [7,8]. The method of Baker et. al. was realized in the ptk-tool to enerate TTCN- test-cases from MSC-000 specifications. The method of Ebner is eneratin TTCN-3 testcases from UML-MSCs. Relatively few work exists on the determination of an interation order. Briand et. al. presented an approach for the determination of an optimal interation order on the basis of couplin measures and enetic alorithms [9,0]. Wu et. al. proposed a method for the interation of component software on the basis of UML statecharts and collaboration diarams, usin structural analysis techniques for the examination of the component-dependencies []. Other work was presented on the eneration of test-cases for interation testin on the basis of UML diarams [,3,4,5]. The work of Basanieri and Bertolino [4] also aims for the development of an practical approach. Beside the application of coverae criteria to MSCs, the test-case eneration is also based on use-case-maps, and class diarams. Althouh this work is aimin for similar practical problems, their work is restricted to structural analyses. Driver C C Stub C 3 Stub Fiure : Testbed Confiuration of an interation step Thouh, the mentioned approaches and methods solve many of the practical problems that we are facin in the interation of automotive telematics systems, there was none that could be applied to our projects. First of all, none of these methods take the project and the system environment into account, which in our opinion reatly influence the interation order. Furthermore, most of the presented test eneration methods and structural quality measures demand syntactically sound MSCs to be applied efficiently. In our projects MSCs are often created manually from the scratch with many different tools, and therefore MSCs are of rather low syntactical quality. Therefore, this paper addresses the determination of an interation stratey which can easily be adapted to chanes in the project or in the system environment, and which can be manually applied to the iven MSCspecifications. 3. IN-CAR TELEMATICS SYSTEMS The telematics systems under test are interated in-car entertainment and infotainment components such as a central user interface, naviation system, radio, audio amplifier, cd-chaner etc. Most of the telematics features are realized by the simultaneous interplay of these components, e.. on phone call the cd-player is paused and the audio amplifier allocates audio channels for the hands-free speakin system. A cost effective way to continue innovatin in all these areas is to allow the devices to be developed independently by different suppliers and to be networked usin standard hardware and software interfaces. Challenes arise due to the limited component observability durin interation testin, because the behavior can only be observed throuh the component s interfaces. The various components of the system are connected usin several networkin standards. One prominent standard is currently Media Oriented System Transport (MOST) []. In fiure a characteristic example of the application of MOST in the automotive sector is presented. The development and interation of automotive telematics system components is accompanied by a number of testin phases. Initially each component is separately unit tested aainst its specification. Thouh, much of the functionality of a system can only be tested when a number of devices are interated in the network. Durin interation testin functionalities are tested aainst system specifications that are distributed across a number of devices., e.. telephony, naviation. Finally, durin system testin the interoperability of features and the interaction between system and environment are checked. The suppliers are mainly responsible for unit testin while the car manufacturer typically focuses on interation and system testin of the telematics network. Due to our experience, the interation testin phase is the most critical testin phase for the car manufacturer.

3 Interation Stratey Testbed Minimal Confiuration Test Plan Test Protocol Requirements Environment Risk Structure Component Metrics Standards Guidelines Contracts Messae Sequence Charts Test References Input Interation Stratey Time- and Cost Plan Testin Methods Test Cases Test Run Activities Test Evaluation Activities Plannin Goals Plannin Testin Methods Step i Test Cases Step i Testbed Step i Relevant for Step i Standards Guidelines Contract Pararaphs Relevant for Step i MSCs References i Interation-Step i Output Fiure 3: Interation Test Plannin 4. INTEGRATION TEST PLAN The interation test of telematics systems aims for the detection of faults in the interaction and communication of the concurrently workin components. The system is interated and tested incrementally, i.e. in each step exactly one component is interated and the interaction and communication concernin this component is covered with test-cases. As a prerequisite, each component has to be sufficiently unit-tested before interation. The order of the components durin interation follows a chosen stratey, which has to take environment, criticality, and structure into account. The missin components in an interation step have to be substituted and simulated by drivers and stubs. Therefore each interation step demands a specific hardware and software infrastructure which embeds the components under test (CUTs). An example of a hardware confiuration of such a test bed is presented in fiure. A test bed consists of a minimal confiuration, which is not chaned durin the whole interation process, and a flexible confiuration, which is adapted to the CUTs in each interation step. The minimal confiuration is constructed and applied before the interation is started. It has to provide the tester with the ability to observe and control the communication and the interaction of the CUTs. The several step-specific confiurations are constructed and applied on the basis of information derived from the interation stratey. A step specific confiuration may consist of additional components to simulate component behaviour, referred to as stubs, or to stimulate behaviour, referred to as drivers. After construction of the step-specific confiuration the test-cases are automatically executed. The resultin test-protocol of a step is evaluated aainst a test reference which contains all relevant documents related to the CUTs. These test references may include functional and requirements specifications, contractual areements, or standards and uidelines. In this paper it is assumed, that the component communication and interaction is specified in the form of MSCs, aainst which the test-protocols are evaluated. In fiure 3 a schema of plannin the interation testin of telematics systems is presented. In the center the oals of the plannin process are placed. The input of the plannin process consists of the minimal test bed confiuration, environment, risk and structure measures of each component, and the test references. Durin interation test plannin the stratey is determined, i.e. the order in which components are interated. Based on the interation stratey and the component metrics for each interation step, respectively component, testin methods are chosen and optionally test cases can be determined. Furthermore, all requirements for runnin tests are identified, which means first of all complementin the test bed for each interation step. Based on the interation stratey the relevant testin references for each step are identified. At last, a time and cost plan is arraned which also takes the relevant activities and resources of each interation step into account. 5. INTEGRATION STRATEGY An sufficient interation stratey must be flexible to react on chanes in the requirements or in the environment of the software development project, e.. component delivery delay or sudden customer presentations. Fiure 3 presents the development of an interation stratey from environment, risk and structure analyses. Furthermore standards, rules and uidelines have to be taken into account. In this section the development of an interation stratey throuh analytical means is presented in detail.

4 5. Environment Analysis The environment influences the interation of components throuh economical, technical and oranizational constraints. A chane in one of these constraints miht enforce a chane in the interation order of the CUTs. Table : Interation by Delivery Date ORDER CUT DELIVERY DATE b approved d Oct 0 th a Oct th e Oct. 3 th c Oct. 3 th f Oct. 3 th Oct. 4 th 004 An important oranizational and economical aspect of the environmental constraints is the delivery date of a CUT. The delivery dates of the CUTs constitute an economical interation order. CUTs with early delivery dates have a hih interation priority, while CUTs with late delivery dates are of low priority. The interation stratey has to be chaned, when a CUTs delivery is delayed after its scheduled interation date. A delayed component is scheduled for interation as soon as possible after the delayed delivery date. The interation by delivery date of an exemplary system of seven components is presented in table. The interation order of the components e, c, f and b is arbitrary, because e, c, and f are delivered on the same day and b is approved. Table : Interation by Approval ORDER CUT LAST CHANGE a Oct. 0 th. 004 d Oct. st e Au. th f Mar. 3 rd Jan. 0 th b Dec. th c Jan. st 003 In many development projects a number of CUTs are approved components or have the same delivery date. In these cases the determination of an interation order demands other criteria. Another environmental aspect for the interation of components is the practical approval of CUTs. A CUT which has been approved in the field for a lon period demands less testin than a lately developed component. Therefore CUTs miht be ordered accordin to their approval in the field. The example in table presents the interation by approval of the system with seven CUTs. These results can be combined with table, which allows a determined interation each CUT. For the purpose of interation testin the orderin by approval is only a very weak criteria, because the approval of a component in a different environment is only a weak indicator that it will work properly in the iven environment. To prove this is actually the purpose of the interation test. Another aspect for the determination of an interation order is the technical environment of the CUTs. In some cases it miht be necessary to interate multiple CUTs at once or in a fixed order, when the technical dependence between these components is very stron, e.. a CD-chaner that stronly depends on a stereo controller component. There are probably many more technical and oranizational aspects than we are able to consider in this paper, therefore we only propose for the iven example the consideration of technical and oranizational aspects of the system and the project environment. An additional table for each considered aspect can achieve this. 5. Risk Analysis The oal of a development project is put at risk by problems concernin the product and the project itself. Therefore the risk analysis and risk reducin tasks have important influences to the success of a project. A risk should be discovered and reduced as early as possible. Obviously, this also influences the interationtestin phase. A risk analysis is based on functional and requirements specifications and experiences from foreoin projects in the form of fault and risk statistics. For the determination of an interation stratey the risk analysis of the CUTs and of the system are of major importance. A risk analysis can be carried out with checklists, questionnaires, and reviews. Table 3: Interation by Risk ORDER CUT RISK LEVEL e 7 c 7 3 d b 5 6 f 3 7 a 3 The output of a risk analysis can be used for the determination of an interation stratey. A CUT with a hiher risk is interated earlier, than a CUT of lower priority. Therefore, CUTs of hih priority are loner tested in favour to lower priority CUTs. In table 3 the example of the system with seven CUTs is ordered on the basis of a risk analysis. The risk is measured in terms of low risk (3), medium risk (5), and hih risk (7) - the risk values 3,5, and 7 are freely chosen. Obviously, this risk analysis is too coarse to provide a complete interation stratey, thouh it orders a, b, and c. In combination with the environmental analysis only e can be interated arbitrarily. A detailed risk analysis could help to determine the interation of e, but usually a complete risk analysis just for the purpose of interation cannot be achieved economically. It is more realistic to suppose, that a risk analysis from earlier development phases is used.

5 a md5 ma md3 ma md6 ma b d f Derive Communication Graph from MSCs. Map Interaction Components on Nodes. Map Communication between Components on Edes 3. Label Edes with Number of Messaes sent between Components b c 5 a d e 3 c e 3 f 3 b d e d md3 me m md4 Fiure 4: Communication Structure of Seven CUTs Example 5.3 Structure Analysis The CUTs are communicatin via the MOST rin. Despite the topoloy of the underlyin communication system the communication structure of the system is dependin only on the implementation of the CUTs. Based on a structural analysis an interation stratey can be developed, which takes the intensity of the interaction and communication of a CUT with other CUTs into account. Therefore, CUTs with a hih interaction and communication intensity are earlier interated than components with a low intensity. Table 4: Interation by Rane a b c d e f R a b c d e f In fiure 4 the communication structure of the example of the seven CUTs is presented in the form of a denoted, directed raph. The nodes represent the CUTs. An ede between a and b represents a communication from a to b, which in our case means a sends a messae to b. Each ede is denoted with an inteer value, which represents the number of sent messaes. For the analysis of the communication structures the activity and the rane of a CUT are measured. The rane R is the number of f me3 mf m m3 partners, a CUT is sendin messaes to. The activity A is the number of messaes a CUT is sendin overall. In table 4 the calculation of the ranes for the example with seven CUTs is demonstrated. The rows of the relation matrix represent the communication of the row components with the column components. A rane R of a component is the sum of the values of its row in the rane matrix. Table 5: Interation by Activity a b c d e f A a b c d e f In table 5 the calculation of the activities for our example with a relation table is demonstrated. The rows of the table contain the numbers messaes bein sent by a row CUT to a column CUT. An activity measure A of a component is the sum of the values of its row in the activity matrix. Table 6: Interation by Intensity ORDER CUT INTENSITY d 3 a 3 f 7 4 c 6 5 e 4 6 b 3 7 The rane and the activity measures are used to determine the interation order. Hihly active components with a wide rane are interated first. The sum of the activity and rane values results in the communication intensity. For further application other combination rules may be applied. 5.4 Interation Order The environment, risk and structure analyses are finally combined in an interation stratey. First of all, the environment measures are applied in order to achieve flexibility on chanes in the project environment. The risk measures are taken into account secondly, if still unordered CUTs exist. Finally, the structure measures complete the interation order. This order is mainly motivated by our experience, that in most projects the oranizational and the technical restrictions are of hihest importance. Structural measures are of lowest importance, because they lack intuition and usually demand a certain deree of formality to be applied efficiently. In our projects we usually find specifications of varyin quality. Therefore, we cannot apply interation methods which depend on formal specifications and tool-support e.. time and cost optimal interation methods, which minimize the effort of the test environment implementation.

6 ORDER CUT DELIVERY DATE b approved d Oct 0 th a Oct th e Oct. 3 th c Oct. 3 th f Oct. 3 th Oct. 4 th 004 ORDER CUT RISK LEVEL e 7 c 7 3 d b 5 6 f 3 7 a 3 3 ORDER CUT LAST CHANGE a Oct. 0 th. 004 d Oct. st e Au. th f Mar. 3 rd Jan. 0 th b Dec. th c Jan. st 003 ORDER CUT INTENSITY d 3 a 3 f 7 4 c 6 5 e 4 6 b 3 7 ORDER CUT b d 3 a 4 c 5 e 6 f 7 In fiure 5 the determination of the interation order for the example with seven CUTs is presented as a whole. The dotted lines in the example represent alternative or additional criteria, which are not used here. The lines demonstrate the orderin of the CUTs. The delivery date leaves the components b, e, c, and f unordered. Therefore, the risk level is used, althouh the CUTs e and c still remain unordered. At last the intensity ives order to e and c and a final interation stratey is determined. 6. STEPWISE TEST-CASE DEVELOPMENT After determination of an interation order the test-cases for each interation step can be developed. The MSCs of the example in fiure 4 are sorted out and derived to stepwise MSCs, which are the basis for the test-case eneration. In fiure 6 the MSCs of the example are used to create test-cases for the seven interation-steps. Based on the CUTs which are interated in one step, the relevant MSCs are chosen. If a MSC contains CUTs that are not interated until this interation step, those CUTs, and all the communication amon them, are deleted from the MSC. Instead of the deleted CUTs stubs are added to the MSC. The remainin communication of the MSC is defined only between interated CUTs, stubs and drivers. The stepwise test-case eneration is dependin on the remainin structure of the MSCs. Testin methods, e.. messae, condition, or data flow coverae criteria, are chosen dependin on the existence of messae parameters, conditions and uards, loops, and lobal data. The details of test-case eneration on the basis of MSCs was presented in a number of papers and is not in the focus of this work. For further information on this topic we refer to literature mentioned in the related work section. Fiure 5: Combined Interation Stratey 7. CONCLUSION In this work, we presented an approach for the plannin of interation tests of automotive telematics systems. The interation testin is based on informal MSCs, which are used for the specification of the interaction of telematics components. In contrast to other interation strateies in the presence of raphical communication specifications, our approach prefers environmental and risk analyses over structure analyses. Because of the simplicity of the measures and techniques used in our approach, we think it can be efficiently applied to many industrial projects. Even in the case that only informal MSCs are used and therefore tool assistance is not possible, the tester can easily develop a test plan and test cases manually. In further research and publication we will focus on the test case eneration on the basis of step-wise MSCs. An important requirement of such a method is the quick reaction on chanes in the project environment. Therefore an efficient testin method for this approach must be hihly adaptable. To ease the practical application of out approach we aim for the development of a set of easy-to-use spread-sheet implementations, checklists, and questionnaires. The aim of this development will be to formalise the process of interation test plannin and to increase the acceptance of this approach amon system developers and testers. In the near future we will apply this approach to a number of industrial projects for the development of automotive info- and entertainment systems. Furthermore, we are workin on the adaption of this approach to other automotive software systems e.. enine manaement or driver assistance systems.

7 Literature [] MOST-Cooperation, Media Oriented System Transport, Step a b c b d [] ITU, ITU Z.0 Messae Sequence Charts (MSC-000), / [3] Object Manaement Group, OMG UML.0, technoloy/documents/modelin_spec_catalo.htm [4] O.Hauen, Comparin UML.0 Interactions and MSC-000, SAM 004, 4 th International SDL and MSC Workshop, Ottawa, Canada, 004 [5] P. Baker, P. Bristow, C. Jervis, D. J. Kin, B. Mitchel, Automatic Generation of Conformance Tests from Messae Sequence Charts, SAM 00 3 rd International SDL and MSC Workshop, Aberystwyth, 00 [6] M. Ebner, TTCN-3 Test Case Generation from Messae Sequence Charts, ISSRE 04, Workshop on Interated Reliability with Telecommunications and UML, Rennes, 004 [7] ITU-T Recommendation X.9, OSI conformance testin methodoloy and framework for protocol Recommendations for ITU-T applications The Tree and Tabular Combined Notation (TTCN), 998 [8] ITU-T Recommendation Z.40, The Tree and Tabular Combined Notation version 3 (TTCN-3): Core Lanuae, 00 [9] L. Briand, Y. Labiche, Y. Wand, An Investiation of Graph- Based Class Interation Test Order Strateies, IEEE Transactions on Software Enineerin, 9 (6), 003 [0] L. Briand, J. Fen, Y. Labiche, Experimentin with Genetic Alorithms and Couplin Measures to Devise Optimal Interation Test Orders, Software Enineerin with Computational Intellience, Spriner Verla, Berlin, 003 [] Y. Wu, M. Chen, J. Offutt, UML-based Interation Testin for Component-Based Software, nd International Conference on COTS-Based Software Systems (ICCBSS), Ottawa, 003 [] W. K. Chan, T. Y. Chen, T. H. Tse, An Overview of Interation Testin Techniques for Object-Oriented Prorams, Proceedins of the nd Annual International Conference on Computer and Information Science (ICIS 00), Mt. Pleasant, 00 [3] F. Basanieri, A. Bertolino, E. Marchetti, A. Ribolino, G. Lombardi, G. Nucera, An Automated Test Stratey Based on UML Diarams, Proc. Ericsson Rational User Conference, Upplands Vasby, Sweden, October 00 [4] F. Basanieri, A. Bertolino, A practical approach to UMLbased derivation of interation tests, Proc. Software Quality Week Europe QWE000, 000 [5] R. Heckel, L. Mariani, Component Interation Testin by Graph Transformations, International Conference on Computer Science, Software Enineerin, Information Technoloy, e- Business, and Applications, Cairo, 004 Step Step 3 Step 4 Step 5 Step 6 Step 7 a b d d f d f d f me e d f m me e d f m me e d f m me Fiure 6: Stepwise Test-Case Development mf m m3 mf m m3