NSE Dynamic Software Documentation - Software Documentation Revolution Based on Complexity Science

Transcription

1 NSE Dynamic Software Documentation - Software Documentation Revolution Based on Complexity Science Liana Ye 1, Lin Li 2 1 PeaceNames.com, USA 2 NSESoftware, LLC, USA Abstract This paper introduces Nonlinear Software Engineering (NSE) Documentation Paradigm based on complexity science. By making software documents automatic, graphical, colorful, traceable, virtual, maintainable, and always consistent with source code dynamically, NSE brings revolutionary changes to software documentation. Keywords: software documentation, software document, visualization, diagram, testing, maintenance 1. Introduction ehow contributor O Paul pointed out that software documentation pervades the software life cycle. It is the visible part of the software process. Without it, software cannot be maintained, users are hard to train and the software is virtually unusable. Without it, new developers would have to re-invent the wheel in software development. Software documentation is the most important manifestation of software. It is the guide through the software maze [1]. Experience has shown that technical software documentation is unsatisfactorily maintained. The implications of outdated documentation are costly and can damage businesses ( Software documentation has been created manually with "cut and paste", or generated by a semi-automated tool for a large software product. Often the documents obtained with current software documentation techniques and tools are * not traceable, * not accurate, * not precise, and * not consistent with the source code, and * cannot be holistic. This paper discusses a source code based software documentation approach based on complexity science solving all of the problems listed above efficiently, and bringing revolutionary changes to software documentations. 2. What Does a Revolution in Software Documentation Mean? According to The Structure of Scientific Revolutions [2], a revolution in software documentation means: (1) Bringing drastic, complete, and fundamental change to the software documentation paradigm (2) Resolving some outstanding and generally recognized problems in software documentation; (3) There is no other way to efficiently resolve those outstanding and generally recognized problems in software documentation. 3. Documentation Paradigms Documentation is a software production process, reflecting multiple levels of decision-making by people, especially software engineers. Fig.1 shows the mapping from requirements to source code production cycles in waterfall model on a reductionism base. Fig.1. Three-dimensional software production cycles need documentation Documentation is required for software development in the large arrow direction, where engineers are focused on coding and maintenance. Unconnected words indicate standardization efforts and numerous internal debates about software development Software documentation revolution condition (1) NSE software documentation paradigm brings drastic, complete, and fundamental change to the software documentation paradigm by paradigm shift of the foundation of software documentation from the reductionism base of Fig 1, which is manually and partially traced, and its marketing, design and coding documents and specifications are strongly top-down typed and hard to

2 change. NSE is a complexity science based, innovative framework FDS shown in Fig. 1. Fig. 1. FDS (Five-Dimensional Structure Synthesis method) framework 3.2. Software documentation revolution condition (2): The outstanding and generally recognized problems in software documentation are characterized by: (a) Reductionism and superposition principle: the whole of a complex system is the sum of its components, and almost all tasks and activities in software documentation are performed linearly, partially and locally. (b) Linear process models: where workflow goes linearly in one direction with only one track without counting upstream movement. It requires software developers to always document their software with no errors and no wrong decisions. (c) Constructive holism: the components of a complex system are completed first, then the system as a whole is assembled from its components. (d) Impossible to be holistic: many small pieces of documents are generated with no automated summation of the entire system, missing the big picture of the software product. Even if some tools can be used to document an entire software product, short of automated and selfmaintainable traceability, its system level graphical documentation contain too many connection lines as shown in Fig. 2, making the documents hard to understand and almost useless. (f) Documents are generally not traceable. (g) Documents obtained are stored statically as images in Postscript, XML, or other formats, requiring huge amounts of space and long loading times. (h) Manually or graphic editor created graphic documents are not automatically generated, time-consuming to draw, hard to change, and hard to maintain. (i) Documentation is error prone and may not be accurate. (j) The obtained documents are not precise. They can not directly and graphically show whether a code branch or condition combination has been tested or not. 3.3 Solution from NSE software documentation (a) Based on complexity science, particularly the nonlinearity principle and the holism principle, all software documentation tasks and activities are performed holistically and globally, as shown in Fig. 3. Fig. 3 An example in NSE documentation on a complex software holistically (b) Nonlinear process model: Workflow goes nonlinearly through two-way interaction with multiple tracks, supporting both upstream and downstream movement as shown in Fig. 1. (c) Generative holism: The whole of a complex system comes first in embryonic form. It grows up with its components. (d) Holistic: Documents for an entire software product are automatically generated to make it easy to view the documents and understand internal connections, as shown in Fig. 4. Fig. 2 A traditional call graph without traceability. Source: (e) Graphic documents and source code are separated making it hard to keep them consistent, especially after product modification. Fig. 4 An application example for NSE to trace a module and all its related modules upstream and downstream.

3 (e) Source code (either a stub program or a functional program) is the source for most graphical document generation. The graphic document is the visual face of the corresponding source code. They are always consistent with each other. (f) Documents are traceable to and from the source code. Fig. 5 Static visibility example Forward tracing a test case to view which modules can be tested (g) Documents are consistent with their source code after product modification with updates to database: 3.4. Software revolution condition (3): No other method can efficiently resolve those outstanding and generally recognized problems in software documentation except generating documents automatically from source code. Although there are many software documentation tools available on the market, they are disjoint tool sets. Alistair Cockburn, characterizing people as non-linear, first-order components in software, development: We methodologists and process designers have been designing complex systems without characterizing the active components of our systems, known to be highly non-linear and variable (people). From source code to generated documents, NSE produces automatically generated documentation from source code, displays coded decisions made by people keeping bidirectional traces for every symbolic connection in the system, updates function call statements or testing conditions dynamically with graphical representation for highly non-linear and variable people, with precise quantitative information about how many requirements may be affected and how many function call statements may need to be modified. We see no other way to efficiently resolve previously mentioned outstanding and generally recognized problems but to take automatically generated documentation from source code. 4. Documentation in NSE is Dynamic Fig. 6 Requirement Changes: Defect Prevention using forward tracing through the related test cases to determine which modules should be modified for a requirement change. Source code is the ultimate software system documentation. Complete graphical relationship representation of each component is determined when the source code is compiled. NSE collects all information on the software system, then generates holistic, graphical, interactive, and traceable documents automatically, including a stub program meeting development and product specifications and a functioning program for forward engineering or reverse engineering. (h) Documents are dynamically generated directly from the source code. Corresponding databases are virtual without images in memory or on hard disk (with user option to print) to greatly save space and make the speed of display about 1000 times faster than established practice. (i) Generated graphical documents are accurate to the code. (j) Generated graphical documents are precise. They can directly and graphically show whether a code branch or condition combination has been tested or not. NSE software documentation supports the entire software development process, from requirement development through software maintenance. Fig. 7 Traceability among all related documents, test cases, and source code

4 As shown in Fig. 7, source code is the source for automatically generating graphic design documents, while those graphic design documents become visual faces of source code. The design becomes pre-coding, and coding becomes further design. The graphical documents are traceable for static review and executable for dynamic defect removal, as shown in Fig. 4. ActionPlus Diagram, J-flow and J-Diagram as shown in Fig. 9. Self-maintainable documentation facility is established through dynamic traceability among design documents, test cases and the source code, as shown in Fig. 7. After the source code is altered following a requirement change, the design documents and modified source code are automatically incrementally updated, facilitating defect prevention, as shown in Fig Workflow in NSE Documentation Paradigm Workflow for NSE software documentation paradigm consists of source code of regular programs and stub code of design documents: Fig. 9 NSE Graphical representation for a class Time-Event table documents testing events Time-event tables are written in the comment part of a stub program or regular program. An example is listed below: /* Time-Event table: Fig. 8 The workflow of NSE documentation paradigm The objectives of NSE documentation paradigm are: (a) combining software programming and graphical software documentation seamlessly; (b) making one source for both human understanding and computer understanding - through static review by people of the graphical documents and dynamic program execution to ensure the upstream quality of a software product. (c) realizing all kinds of documents (both manually drawn and generated by third party tools) traceable to source code to keep them consistent with each other through the established transparent-box method, combining functional testing and structural testing together seamlessly with the capability of establishing bidirectional traceability. (d) generating most software documents automatically, as much as possible. (e) making software documents visible, as much as possible. (f) making a software product truly maintainable and adaptive to the changed or changing environment NSE software visualization notations A majority of NSE documentation paradigm is visualization in 3J (J-Chart, J-Diagram, and J-Flow diagram) graphics. For example, Classes, using HAETVE model, are represented in several graphical notations: J-chart,... */ 4.3. J-Diagram documents message sending & receiving Graphically representing massage sending and receiving in the automatically established click-to-jump facility as shown in Fig. 10. Fig. 10 Click-to-jump facility automatically established for showing message sending and receiving Software product is visible in multiple views (a) Static view of the cyclomatic complexity of a program.

5 Fig. 11 An example of Cyclomatic complexity analysis (b) Dynamic view of a program Fig. 15 an example of function cross-reference analysis (b) Data analysis, Fig. 16. Fig. 12 An example of performance analysis 4.5. From macro to micro with MC/DC test coverage (a) Holistic analysis for an entire product, Fig 13: Fig. 16 An application example of variable analysis 4.7. From System level to file level to statement level (a) System-level version comparison, Fig. 17. Fig. 13 Holistic MC/DC (Modified Condition/Decision Coverage) test coverage analysis (b) Detailed MC/DC test coverage analysis for an individual class or function, Fig. 14. Fig. 17 An example of holistic version comparison at system level. (b) File-level version comparison, Fig. 18. Fig. 14 An application example of detailed test coverage analysis of a module 4.6. From procedure to data (a) Function cross-reference analysis, Fig. 15. Fig. 18 An example of file-level version comparison (c) Statement version comparison, Fig. 19.

6 With NSE, different graphical documents can be linked together - see Fig. 23. Generated logic diagrams can be converted to control flow diagrams see Fig. 24. Fig. 19 An example of statement version comparison 4.8. Dynamic visibility: Tracing a test case not only to find what modules can be tested, but also to directly play the captured test operations back through the batch file (.bat) specified in keyword within the test case description part see Fig. 20. Fig. 23 An application example - linking a call graph to the logic diagram Fig. 24 An example of diagram conversion from a logic diagram to control flow diagram Fig. 20 Dynamic visibility tracing a test case to play the captured operations back Local documentation to Internet With NSE, many static and dynamic analysis reports can be automatically generated as in Fig Interactively traceable With NSE the generated documents are interactive for instance, the user can click on a module-box to use that module as a root to generate a sub-call-graph, Fig. 21. Fig. 25 An application example of static and dynamic program analysis and reporting Fig. 21. Interaction example: click on a module-box to generate an isolated sub-call-graph Generated reports for static and dynamic program analysis can be saved in HTML format to be used as web pages, Fig. 26. Most of the generated documents are traceable, Fig. 22. Fig. 22 Tracing a module to see all the related modules Linkable and convertible Fig. 26 Code analysis reports saved in HTML format to be used as web pages

7 4.12. Preventing requirement conflicts: Perform backward tracing to the modified modules (in this example, two requirements are related), Fig Project management: Through keywords used in testing traced back to project management view, Fig. 29 dynamically generated from source code, so there is no need to save their hard copy images in memory or disk, so that a huge amount of space can be saved. The display speed is about 1000 times faster compared with established practice under comparable and similar operational environment. The generated holistic charts and diagrams are shown within one window, no more and no less. When scrolling occurs, the new portion of the chart will be generated dynamically. From a users point of view, there is no difference between virtual charts and the regular charts occupying a huge amount of space in computer memory. Massive the graphical documents are generally about 100 times the size in disk space of their source code. It can be automatically generated at system-level, file-level, and module-level. For each class or function, NSE documentation paradigm automatically generates the logic diagram shown in J-Diagram notation with untested branches and untested conditions highlighted, control flow diagram shown in J-Flow diagram notations, quality measurement result shown in Kiviat diagram, etc. The graphic display capability is massive. 6. Application NSE documentation paradigm has been commercially available and supported by Panorama++. All screenshots shown in this paper come from real application examples. 5. Major Features of NSE Documentation Paradigm The graphical documents generated by NSE documentation paradigm are: Holistic NSE documentation paradigm generates holistic charts and diagrams to document an entire software product. Interactive the generated graphical documents are interactive, the generated charts/diagrams themselves are also the interfaces to accept user s commands. Traceable with NSE most of the generated documents are traceable, useful for validation, verification, and semiautomated inspection and walk through, Accurate Source code to NSE, including a stub program, is also the source to automatically generate most graphical documents, so that the generated documents are accurate and consistent with the source code. Precise the generated graphical documents are precise, and the corresponding documents can show how many times a branch is executed, and what code branches and conditions have not been executed. Virtual with NSE, most graphical documents are 7. Conclusions The established software documentation paradigm based on reductionism and superposition is outdated. NSE software documentation paradigm resolves the outstanding and generally recognized problems in software documentation, and causes a paradigm shift to complexity science based principles, making the whole of a complex system greater than the sum of its components. The characteristics and behaviors of the whole emerge from interaction of its components. Source code is not the best documentation of a software product, but source code is the best source to directly and automatically generate holistic, interactive, traceable, consistent, accurate, precise, massive, and graphical documentations for the software concerned. There is no way to solve the problem of inconsistency in documentation except to generate documentation directly from source code. Panaroma++ demonstrates the feasibility of such an approach, turning software reverse engineering into a software documentation revolution. 8. References [1] ( [2] Kuhn T (1962) The structure of scientific revolutions. The University of Chicago Press, Chicago