Software Traceability Establishment Revolution Based on Complexity Science

Transcription

1 Software Traceability Establishment Revolution Based on Complexity Science Po-Kang Chen 1, Jay Xiong 2 1 Y&D Information system, Inc. USA 2 International Software Automation, Inc. (ISA, currently being reorganized), USA Abstract - software becomes system of complexity in the modern time. 40 years passing, modern software design tends to complexity and precision. Until 2012, software design tool doesn t support excellent model s and function s traceability during design program. On the other hand, software design won t adapt on modern software design, which needs a methodology based on complex science for its maintenance and design. Traceability is a important factor influencing quality of software. This paper presents automated, dynamic, accurate, precise, and self-maintainable traceability among related software documents and test cases and source code, established through test case execution and some keywords used within the test case descriptions to indicate the format of the documents as well as the file paths and the bookmarks for automatically opening the documents from the corresponding positions when the related test case is selected for forward tracing or traced from the corresponding source code backwardly. When a test case is executed a Time Tag will be automatically inserted into both the test case description and the database of the test coverage measurement results for mapping them together. No matter if the contents of a document is modified, or the parameters of a test case are changed, or the corresponding source code is modified, after rerunning the test case the traceability will be updated automatically without any manual rework. Here a document means a regular file for requirement specification, design description, test requirement specification, user manual, project development plan, cost report, or a web page as well as a batch file for dynamically running a related program such as a tool for selectively playing back the GUI operations captured with the test case execution, and displaying the test coverage measurement result shown in a new type control flow diagram which is interactive and traceable with untested source modules and branches highlighted at the same time for automated software acceptance testing. Above all, it will bring drastic, complete, and fundamental change of paradigm, resolving some outstanding and generally recognized problems. No other way can efficiently resolve those outstanding and generally recognized problems. Keywords: : software traceability, requirement traceability, validation, verification, testing, quality assurance, maintenance 1 1. Introduction Software is a nonlinear complex system where a small change can ripple through the entire system to cause major unintended impacts Butterfly-Effects, so that prior to performing the actual change, maintainers need facilities in order to understand and estimate how a change will affect the rest of the system. Traceability offers benefits to organizations in the areas of project management, process visibility, requirement validation and verification, and software maintenance. Traceability needs to be hardcoded into a process to be replicated iteratively on each and every project[1]. Without bidirectional traceabnilities software maintenance can not be performed globally and holistically to prevent side-effects. Local and blind software changes will make the software product unstable and unlierable. 1.1 The problems addressed The lack of traceability among software documents, test cases, test results, and source code is caused by several factors, including: (1) the fact that these artifacts are written in different languages (natural language vs. programming language); (2) they describe a software system at various abstraction levels (requirements vs. implementation); (3) processes applied within an organization do not enforce maintenance of existing traceability links; (4) a lack of adequate tool support to create and maintain traceability[2],(5)there are many different types of documents, some of which are created manually, some of which are generated automatically by internal tools, some of which are generated automatically by third parties tools, some of which are designed using graphic editors; (6) some documents are stored locally, some documents are stored in other places through a network; (7) some related documents are web pages, which can be read through the internet only; (8) some documents are related to the software development, while some documents are related to the project management which should also be traceable; and (9) some documents are not static materials, must be viewed dynamically through a program execution. Unfortunately, neither manual traceability methods nor existing COTS traceability tools available on the market are adequate for the current needs of the software engineering industry. Poor methods and tool support are perhaps the biggest challenge to the implementation of traceability - when those tools are used, the traceability information is not always maintained, nor can it always be trusted to be up-to-date and accurate. [1]. Studies have shown that existing commercial traceability tools provide only simplistic support for traceability [3]. Why does software maintenance take 75%

2 or more of the total effort and total budget[4] in most software project development? One of the critical issues is the lack of bidirectional traceabilities among the requirement specification, the design documents, the test cases, the test results, and the source code of a software product. 1.2 The solution The new requirement traceability approach proposed and implemented by the authors is graphically shown in Fig. 1. Figure 1. The facility for automated, bi-directional, and self-maintainable traceability among the documents and the test cases and the source code of a software product The objectives of this traceability facility are: Helping software developers to prevent side-effects in the implementation of software changes; Solving the inconsistency issue to make the documents and the source code traceable with each other to keep consistency; Removing the problems existing with a man-made Requirement-Traceability-Matrix which is inaccurate, time consuming to do, and almost un-maintainable; Making the software development process visible; Making the requirement validation and verification much easy to perform; Making the software product much easy to understand, test, and maintain. As shown in Figure 1, this facility for bidirectional traceability consists of two parts: (1) Part 1 Part 1 of the facility is related to the traceability between test cases and the corresponding source code executed by running the test cases. It is done with the use of Time Tags which are automatically inserted into both the test case descriptions and the corresponding test coverage database. For instance, if test case 1 is executed at 09:00 AM on September 2, 2009, and test case 2 is executed at 10:00 AM on the same day, and test case 3 is executed at 11:00 Am on the same day, then the three different Time Tags will be inserted into the three test cases and the corresponding test coverage database separately. So, when test case 2 is selected for forward tracing, the Time tag of 10:00 AM on September 2, 2009 will be taken from the test case description to search the test coverage data with the same time tag, so the corresponding test coverage data will be read and the corresponding modules and branches will be highlighted on a control flow diagram. On the other hand, when a module or code segment shown on a control flow diagram is selected, the related time tags (which can be more than one) used to indicate what time the module or segment was executed will be taken to search the test case descriptions to see how many test cases with the mapping time tags through backward tracing, then highlights all test cases mapped on the window showing the test case script. (2) Part 2 Part 2 of the facility is to extend the bi-directional traceability from test cases and the source code to include all related documents, the test cases, and the source code. It is done using some key words (written into the comment part of the description of a test case) followed with the corresponding file path and a bookmark to indicate the format of the document, the full path name of the file, and the corresponding location in the document, so that when a test case is selected for forward tracing, or traced from a module or segment backwardly, the corresponding document will be opened and shown from the location indicated by the bookmark. It is recommended to organize the requirement specification and the related documents hierarchically (even if some documents have not been really designed) with inherited (or meaningful) bookmarks as shown in table 1. Table 1 Document Hierarchy. It is important to make the document hierarchy include the test case scripts (test cases numbers) so that when a requirement needs to be changed or selected for validation, it is easy to find what test cases to be used. The major steps for establishing and applying the bidirectional traceability are as follows: Step 1: Organize the requirement specification and the related documents hierarchically with the bookmarks, clearly indicate each requirement and the corresponding test scripts and the test case numbers; Step 2: Design the test case scripts with the corresponding keywords to indicate the formats and the file paths and the bookmarks for the related documents; Step 3: Perform code instrumentation for test coverage analysis to the entire program;

3 Step 4: Compile the program instrumented; Step 5: Execute the test case scripts with the corresponding tool. Step 6: Show the modified test case script files with time tags inserted in a window; Step 7: Show the program test coverage measurement result using a control flow diagram in another window; Step 8: Perform forward tracing from a test case with a tool to map and highlight the corresponding modules and code branches tested by the test case through the inserted time tag at the same time, open the related documents according to the document formats, file paths, as well as the bookmarks (or run the corresponding batch file if keyword is used); Step 9: Perform backward tracing from a program module or code branch with a tool to map and highlight the related test cases though the inserted time tags - at the same time, open the related documents according to the document formats, file paths, as well as the bookmarks (or run the corresponding batch tile if keyword is used); Step 10:After the implementation of code modifications, go to step 3. Step 11: If a related document is modified in the contents only without changing the bookmarks, there is nothing to do; but if the bookmarks are modified (such as the name of a bookmark is changed), modify the corresponding test case scripts according to the new bookmarks, then go to step 5; Step 12: if only the test cases are modified, go to step 5; Step 13: if the source code is modified, go to step 3; Step 14: If it is the time to perform requirement validation and verification (V&V), use the document hierarchy information organized in step 1 to get each requirement and the corresponding test cases to perform forward tracing one by one to see whether the requirement is completely implemented; Step 15: if a requirement is needed to modify: (1) get the test cases related to this requirement to perform forward tracing to locate the documents needed to update, and the source modules or branches needed to modify; (2) perform backward tracing from those modules or braches to see whether more requirements are related if it is related to more requirements, the implementation of the code modification must satisfy all of the related requirements to avoid requirement conflict. Step 16: if it is the time to perform regression testing after modification, get the modules or branches modified to perform backward tracing to collect the corresponding test cases which can be used to re-test the modified program efficiently. Sometimes, there may be a need to add new test cases. 2.1 Automated This facility works automatically with the capability to insert the Time Tags into both the test cases description part (see Fig. 2) and the database of the program test coverage measurement result, and highlight the test cases selected on the corresponding test script window, and the source code modules/branches shown in a control flow diagram on the corresponding source code window, or vice versa, as well as open the related documents traced from the locations pointed by the bookmarks. and headers (final page numbers and running heads will be inserted by the publisher). Select a standard size paper such as A4 (210 X 297 mm) or letter (8.5 X 11 in) when preparing your manuscript. 2.2 Self-maintainable This facility is self-maintainable no matter if the contents of a document is modified, the parameters of a test case is modified, or the source code is modified - after rerunning the test case scripts, the traceability will be automatically updated without manual rework. 2.3 Methodology-independent This facility is methodology-independent no matter which methodology or process models are used to develop the product. 2.4 Nonlinear, bidirectional, and parallel This facility works in a nonlinear, bidirectional, and parallel style as shown in Figure 3 and Figure 4. For example, when a design defect is found after the product delivery, the developers can perform forward tracing to check the related requirement, and backward tracing to find and fix the related source code, etc. as shown in Figure 5. 2 The major features

4 Figure 5 Fixing a design defect through forward and backward traceability 2.5 Accurate This facility is based on the dynamic execution of the test cases and test coverage measurement and the time tags to map the test cases and the source code tested, so that it is accurate. After code modification or parameter changes of the test cases, we can re-run the test cases to automatically update the facility. (A) (B) Figure 3 Supporting parallel work: (A) application in requirement validation through forward traceability; (B) application for defect prevention in code modification Figure 4 Safe implementation of a requirement change with side-effect prevention 2.6 Precise This facility is precise to the highest level up to the code statement/segment (a set of statements to be executed with the same conditions) level bi-directionally. It is particularly useful for side-effect prevention in software maintenance. 2.7 Extended to include software project management documents This facility is extended to include not only the software development documents, but also include the project management documents such as the product development schedule charts, the cost estimation reports, and so on to combine the software development process and the software management process together. If a project management document (such as a gantt chart) is designed using a third party s tool, a corresponding batch file should be designed and used with keyword to indicate the location of the batch file in the test case description part such as the following C:\isa_examples\ganttpro\ganttpro.bat 2.8 Extended to include web pages For supporting web-based software development and applications, this facility is extended to include web pages to be traced and automatically opened through the use keyword to indicate the URL address and the bookmark (#NAME) such as the following example:

5 When the corresponding test case is selected for forward tracing or traced backwardly form a source code module or a source code branch mapped to the test case, the corresponding CrossTalk web page will be opened automatically if the internet is connected. 2.9 Extended for multi-project support This facility is extended to support multi-project development by making the related project progress, special event reports, schedules, budget control documents, and cost reports traceable between two related projects (or among more related projects) as shown in Figure 6. Figure 6 Multi-projects development support 3 Dynamic This facility is extended to have the capability to trace a batch file and dynamically execute the batch file for many kind applications such as playing back the captured GUI operations selectively through the time tags in automated acceptance testing, or running a third party s tool to handle the corresponding documents generated by that tool (see Fig. 7), or dynamically execute a related program for other purposes. Fig. 7 An application example of the dynamic traceability to run a batch file to open a gantt chart showing a project development schedule generated by a third party s tool 3.1 Easy to add on at any time, in any status This facility can be added on at any time and in any status of a software product development project, even if in the requirement development phase where the product design and coding have not started yet in this case we can design a dummy main program without a real output but can be executed for checking the consistency between requirement specifications, prototype design documents, test requirements, and test scripts it is recommended to design the test scripts with the requirement specifications at the same time before the product design. In the case this facility is used for a product developed or being developed using other methodologies, the users only need to set bookmarks to the related documents and modify the test case description with simple rules listed as follows: a # character at the beginning position of a line means a comment. an empty line means a separator between different test cases. Within comments, users can use some keywords to indicate the format of any document, followed by the full path name of the document, and a bookmark. After the comment part, there is a line to indicate the directory for running the corresponding program. The final line in a test case description is the command line (which may start a program with the GUI ) with the options. Other work can be done automatically by the corresponding tools. 4 Conclusions This automated and self-maintainable traceability technique has been successfully applied in requirement validation and verification, side-effect prevention for the implementation of requirement changes and code modifications, inconsistency checking among documents and test cases and source code, efficient regression testing through backward tracing from a modified module or branch to select the corresponding test cases, and quality assurance in the entire software development lifecycle through defect prevention and defect propagation prevention. That is Why software traceability is important. Current tools doesn t support efficient way to traceability, but our tools can be done it ; That is difference to other solution. 5 References

6 [1] Andrew Kannenberg, et al. Why Software Requirements Traceability Remains a Challenge, CrossTalk, Jul/Aug 2009 Issue. [2] Juergen Rilling, et al. CASCON 2007 Workshop Report,Traceability in Software Engineering - Past, Present and Future,IBM Technical Report: TR October 25, 2007 [3] Ramesh, Balasubramaniam, and Matthias Jarke. Toward Reference Models for Requirements Traceability. IEEE Transactions on Software Engineering 1 (2001): [4] Ambler S W. A Manager s Introduction to The Rational Unified Process (RUP), Ambysoft [5] Jay Xiong, Tutorial, A Complete Revolution in Software Engineering Based on Complexity Science, WORLDCOMP'09 -, Las Vegas, July 13-17, [6] Jay Xiong, Jonathan Xiong, A Complete Revolution in Software Engineering Based on Complexity Science, WORLDCOMP'09 SERP (Software Engineering Research and Practice 2009),