level requirement for hardware reliability could be satisfied by a policy of shadowed pairs or one of majority voting. Different engineering roles use

Transcription

1 TRACEABILITY FOR COMPLEX SYSTEMS ENGINEERING Stephanie White Grumman Aerospace & Electronics Mail Stop B38-35 Bethpage, NY Abstract. Improved requirements traceability would result in significant cost savings since more requirements problems are due to failures in requirements management than failures in technical functions. Without adequate requirements traceability, errors are found late in system development and are expensive to repair. Our research identifies problems with industry's capability to trace product and process information, and 1 offers some solutions. We have defined a unique method, Concurrent Stimulus-Response Threads (CSRT), that traces complex system behavior. Based on CSRT and Entity-Relationship modeling, we make recommendations that will improve traceability in two requirements methods, Requirements Driven Design (RDD), and Real-Time Structured Analysis (RT-SA). ISSUES IN TRACING SYSTEM INFORMATION The complexity of DOD systems is enormous and engineers developing these systems require traceability. System engineers need the capability to control change, control the process, and control risk. Maintenance engineers need traceability to better understand the product, and the process used to develop the system. All engineers need quick access to information, and since the information is vast and complex, engineers need information abstraction and visualization techniques that help promote understanding. Unfortunately tracing is difficult with current technology, and minimally supported. It is expensive to capture information. Tools cross disciplines and do not have the same lexicon. We cannot measure the benefits This research was performed under contract to Naval Surface Warfare Center, Dahlgren Division, White Oak Detachment, and was funded under the Engineering of Complex Systems Technology Program sponsored by the Office of Naval Research, Contract Number N C of traceability, and the cost is high. We do not know the level of granularity at which to trace, and how to represent much of the information that should be traced (e.g., behavior, non-functional requirements). In most cases, DOD customers are not asking for software traceability except for that covered in DOD-STD 2167A. This standard addresses traceability of documentation (requirements, design, implementation, test cases), and traceability of unit Software Development Folders to component Software Development Folders. Engineers need greater capability for traceability. They must be notified of changes, and should rigorously trace their impact. Change can affect cost, schedule and feasibility, system design and implementation, system tests, and support software and hardware. Engineers need traceability to assess system artifacts during development. They should assess whether the system is under or over designed, specifications contradict each other, lower level decisions are consistent with higher level decisions, test cases cover requirements, detailed behavior and high level behavior are consistent, and non-functional requirements are met. Current traceability aids are reasonably good for product functions, but are limited for tracing system behavior and product attributes (Director of Defense, Research and Engineering '91). Behavior is difficult to trace using current methods, as behavior is specified by stimulus-response statements that cross functional lines. Current trace mechanisms link stimulus-response paths to a significantly large system subset or the entire system. Current methods manage product attributes such as timing and reliability as they would manage weight, by allocating a portion to a component. However, timing and reliability are emergent properties and cross functional boundaries. These properties cannot be analyzed by examining each component in isolation. Timing and reliability vary by scenario, and should be traced to these scenarios. Current methods trace high level requirements for reliability, availability, security, and safety directly to lower level requirements. In many cases, high level requirements for these attributes should be traced to a high level design policy, and that policy should be traced to the lower level requirements. For example, a high

2 level requirement for hardware reliability could be satisfied by a policy of shadowed pairs or one of majority voting. Different engineering roles use information for different purposes. There may even be a conflict between roles. For example, reliability engineers are interested in availability and safety engineers do not want any failures. For this reason the stakeholders themselves must decide what should be traced. Because there is a vast amount of information, these engineers need automated support for capturing and linking information while performing their normal engineering functions. All information may not have to be collected and stored. Some information can be derived automatically by smart tools. RESEARCH OVERVIEW Our research solves problems in tracing system behavior. We have defined a formal method, Concurrent Stimulus-Response Threads (CSRT), for representing and tracing threads (White '93). CSRT represents system behavior in a manner that can be traced throughout the system. These threads of behavior relate system stimulus to system response, and relate environment and system events with system functions. Threads of requirements can be traced to threads in the system design and implementation to determine if behavioral requirements are implemented correctly. Concurrent Stimulus-Response Threads define ordered histories of interaction among system aspects and the environment, providing an excellent mechanism for defining threads for testing. While formal, CSRT is practical for use on large projects. Because CSRT is formal, it is reasonable to assume that analysis techniques such as temporal logic could be used to reason about system threads. We also investigated two requirements methods for real-time systems, RDD and RT-SA, to determine if their languages adequately support traceability and to identify the strengths and weaknesses of each. A method's language is important for traceability, as information that is not specified cannot be traced. We have identified weaknesses in both languages that can be repaired. We achieved this by examining the entities and relationships each method uses to express requirements. This work was based on previous research documented in (White '87). In order to compare languages, we have formally defined the semantics of each language in Entity-Relationship-Attribute (ERA) format. RDD entities and relationships were abstracted from RDD language reference manuals and training materials (Ascent Logic '92). A formal definition of the RT-SA language as expressed in both (Mellor and Ward '85) and (Hatley and Pirbhai '87) was not previously available and a definition of RT-SA in ERA format is a contribution of our research. CSRT Existing methods that organize system behavior by function and object provide poor visibility of interfunctional threads through the system. Because behavior is divided among a number of functions, system engineers find it difficult to verify that the entire thread from environment stimulus to system response is complete, that inter-component aspects are consistent, and that no unnecessary functions are included. This inability to verify the completeness and consistency of inter-functional threads is inherent in existing methods, and is a problem even when one engineer is creating the design. The problem is compounded on a large system with several contractors and numerous designers. Engineers need a method for identifying critical stimulus-response threads that cut horizontally across objects and functions in the requirements model. They need a method that can trace these system threads to more detailed threads, and to threads in the design and implementation. Our approach, Concurrent Stimulus- Response Threads (CSRT) formally defines threads of system behavior. This creates a firm foundation for tracing and analyzing critical threads. The highest level threads are the system requirements. These threads specify the environment stimulus and the system response. More detailed requirement and design threads define paths of system operation from system stimulus to system response that implement the high level requirement. Once the system stimulus occurs, other environment or system events can occur (e.g., a failure). Each thread is uniquely determined by a specific sequence of these conditions and events. The threads specify the behavior of the system in response to the conditions and events. We define a single thread of behavior T as an alternating sequence of events and processes, T = E, P, E, P, E, P,...,E, P n n where an event E is a compound expression of events i and conditions or a sequence of events. Examples: Ei = E 1 when C 1 OR C2 E i = E 1 when C 1 AND C2 E i = E 1, E 2, E 3... En

3 Thread 1 AND Thread 2 Event 1: Coin Accept & Count Coins Event 1: Customer Selection Respond to Customer Selection Thread 2.1 OR Thread 2.2 Event 1: Customer Selection is Coin Return Respond to Return Coin Request Event 1: Customer Selection is Product Selection Provide Product Thread OR Thread Event 1: Customer Selection is Product Selection Validate Payment Event 2: Payment Invalid Process 2: Return Coins Event 1: Customer Selection is Product Selection Validate Payment Event 2: Payment Valid Process 2: Dispense Product & Change Figure 2 Soda Vending Machine Threads A process P is a sequence of processes or a set of j concurrent processes, but cannot consist of alternate processes as disjuncture indicates separate threads. Examples: P = P, P, P... P j j1 j2 j3 jn P = P & P & P j j1 j2 j3 In a single thread, Pj cannot be represented as P j1 OR P j2. Assume a thread T: T = E, P, E, P, E, P,...,E, P n n A more detailed version of T, called T' will include the events and processes of T. In T', the events may be expressed in terms of more detailed data items, the processes may be expanded into more detailed threads, and new events and processes may be added (e.g., to handle interfaces due to allocation), see Fig. 1. Each thread T should be traced to the more detailed thread T'. Our concept is that threads should be generated from existing models, for example from RDD, RT-SA, or Object Oriented methods. We do not recommend developing a new modeling method based on threads. In Fig. 2, we show several threads for a soda vending machine that were manually generated from an RT-SA model by examining data flow diagrams and unraveling state machines. In (White '93) we also generate threads from an RDD model of the soda vending machine. The methods used for generating threads from RT-SA and RDD models could be automated. Figure 2 shows threads of soda vending machine behavior, at several levels of abstraction. At the top level, we have two concurrent threads, Thread 1: React to Coin, and Thread 2: React to Customer Selection. Thread 2 traces to two threads, Thread 2.1: React to Coin Return Request, and

4 Thread 2.2: React to Product Selection. Threads 2.1 and 2.2 are alternate threads for the entire Thread 2. They are not an expansion of a particular function in Thread 2. Thread 2.2 traces to two alternate threads, Thread 2.2.1: React to Product Selection and Payment Invalid, and Thread 2.2.2: React to Product Selection and Payment Valid. Note that each thread can be uniquely identified by a specific sequence of conditions and events. In this paper we trace threads within the same model, but related threads should also be traced between requirements, design, and implementation. Developers need these thread views of system behavior to isolate and understand different scenarios of system operation, and to be sure that threads are consistent and requirements are met. SUPPORT FOR TRACING IN CURRENT METHODS Requirements and design methods capture information about a system. Frequently, this information is formally defined using an Entity Relationship Attribute (ERA) notation. The ERA notation is especially useful for supporting information capture in a database and traceability. The schema defines entities and entity attributes that the modeler can use to describe a system, and relationships that the modeler can use to link entities. Engineers should trace entities, attributes, and relationships to the design and implementation. For example, in Fig. 3, relationship 210 indicates that Functions are activated by an Event when a Condition is true. Engineers should trace this relationship for specific functions, events, and conditions, to ensure that functions in the design and implementation are activated under the proper circumstances. Methods and tools that define models semiformally (e.g. via graphics and tables), should capture information in ERA or other formal notation. Many structured analysis tools do not capture all relationships in a form that can be traced. In order to encourage traceability when using RT-SA, we have formalized the language as represented in the two most popular books on Structured Analysis, "Structured Development for Real- Time Systems" (Mellor and Ward '85), and "Strategies for Real-Time System Specification" (Hatley and Pirbhai '87). We do not address entities and relationships captured by a specific tool, as there are numerous tools that support the structured analysis method. We have captured the RDD language in the same ERA format to compare information that the two methods can express. We found slightly different descriptions of the schema in the RDD User Manual, RDD Dynamic Verification Facility Manual, and RDD Training Material, perhaps because they were written at different times. Our description of RDD entities, relationships, and attributes is taken primarily from the training material. The following recommendations are based on our comparison of the ERA models for RT-SA and RDD. IMPROVING RT-SA FOR TRACEABILITY RDD was designed as a systems engineering method, and RT-SA was first designed as a software engineering method. This discipline focus accounts for use of words such as Function, Component, and Item in RDD, and use of words such as Process, Module, Data Flow, and DataStore in RT-SA. RDD has certain capabilities because of its system engineering focus that can be used to improve RT-SA. RT-SA does not trace derived requirements such as functions, data flows, and states to System Requirements or the Source of these requirements, nor does it capture Critical Issues, Decisions, Constraints, System Parameters (such as reliability), and Performance Indices (such as mean time to failure), all of which are expressible in RDD. This information is as important for software traceability as it is for system traceability. Also RDD can identify and trace sequences of data that enter and leave the system. RT-SA cannot specify the time history of stimuli or the time history of responses as single items. This history is only visible through state machine behavior and is not traceable. The concept of Interfaces is more fully developed and complete in RDD. By examining RDD interface relationships, we determine that an interface is defined by communication devices, functions that service the interface, components and systems that are connected by the interface, and data or things that cross the interface. Interface requirements that are specified using these entities and relationships can be traced, to verify that designs and implementation are consistent and complete. Function replication and control (e.g., for multiple elevators) is well representedin RDD. RT-SA does not have any mechanisms for representing this structure. Replicated functions should be traceable to an attribute that specifies limits on the number of functions, and should be traceable to their implementation. IMPROVING RDD FOR TRACEABILITY States and Events are not recognized as entities in RDD, as they are in RT-SA. RDD is based on the premise that sequences of Items (things or data) arrive at See Appendix A for a definition of terms. 2

5 the System; Functions transform those Items; and the system state is represented by the state of Items in the system after transformations are performed. State and Event are implicit rather than explicit, and cannot be traced. In addition, RDD does not relate conditions that determine function flow to Items (things, data) on which conditions are based. Also, RDD does not have a specific item to represent a Data Store, so the modeler cannot easily represent the updating of persistent data by various Functions. Although this is a drawback graphically, it is not too harmful from a traceability point of view. RDD can relate Items to their subparts and to Functions that generate and use them, and the attribute State Item allows the modeler to indicate that data is persistent, as opposed to transient. Ascent Logic should upgrade RDD to include the concepts of State, Event, and Data Store, and should relate conditions to Items. RDD does not support information modeling, while RT-SA does. This capability would be helpful for understanding the relationship among "things" and data in the system, and ensuring that these relationships are preserved in the implementation. IMPROVING BOTH RT-SA AND RDD Neither RDD nor RT-SA can identify a unique system Thread and trace this thread to less abstract threads at a more detailed level of system specification. This capability is needed to verify requirements with customers, and to support test and integration. A thread view is also needed for ensuring that required order is preserved between requirements, design and implementation. We recommend that CSRT be used with RT-SA and RDD. Neither method traces Positions, Arguments, and Assumptions as in (Ramesh and Luqi '93). These entities are needed to understand and trace the design and decision making process during system development and re- engineering. The capture and tracing of these entities supports understanding, tradeoff analysis, and change impact analysis. Many tools that implement RT-SA and RDD, fail to formally capture some relationships that are expressed graphically. In RT-SA, we can express graphically in a State Transition Diagram, that an Event occurring in a State causes an Action. However, many tools trace the state transition diagram (STD), but not relationships embodied in the STD. If we expressed state transition relationships in ERA format as in Fig. 4, we could trace these relationships to system design and implementation. FUTURE WORK We intend to integrate formal methods for specifying thread interdependencies with CSRT, and develop methods for analyzing thread consistency. In Fig. 1, T represents a single thread, but complex systems are composed of multiple concurrent threads with interdependencies. These threads must either include synchronization points, may be restricted by "constrained expressions" that specify the required ordering of events (Wiledon '86), or may be restricted by using a combination of conditions and events as in the Naval Research Laboratory Software Cost Reduction (SCR) Method (Heninger '80). We will investigate these techniques for use with CSRT. We will also investigate the use of temporal logic to analyze whether the sequence of events and functions is consistent in related threads. CONCLUSIONS Good process and product traceability would reduce costs significantly. Contractors should place more emphasis on the need to improve information capture and traceability. Tracing critical stimulus-response requirements to threads in requirements models, design and implementation would locate inter-functional and inter- component errors, inconsistencies, and missing elements. We plan continued experimentation with CSRT on larger problems. ACKNOWLEDGMENTS The author would like to thank Mike Edwards, Naval Surface Warfare Center, for his critique of this research and helpful suggestions. REFERENCES Ascent Logic Corporation, RDD Manual, Release 3.0, Jun Director of Defense Research and Engineering, DOD Software Technology Strategy, Dec Department of Defense System Management College(DSMC), DSMC Glossary, Defense Acquisition Acronyms and Terms, DSMC Acquisition Policy Department, Fort Belvoir, Va., Sept Hatley, D.J. and Pirbhai, I.A., Strategies for Real-Time System Specification, Dorset House, NY, Heninger, K., "Specifying Software Requirements for Complex Systems: New Techniques and their

6 Application", IEEE Trans. on Software Engineering, Vol. SE-6, Jan. 1980, pp Mellor, S.J. and Ward, P.T., Structured Development for Real-Time Systems, Vol. 1-3, Prentice Hall, Englewood Cliffs, NJ, Ramesh, B. and Luqi, "An Intelligent Assistant for Requirements Validation for Embedded Systems, submitted to IEEE Expert, White, S.M., A Pragmatic Method for Computer System Definition, Ph.D. Dissertation, Polytechnic University, University Microfilms, Ann Arbor, MI, Jun. 1987; also Tech. Report RE-791, Grumman Corporate Research Center. White, S.M., "Tracing Entities, Relationships, and System Behavior", Final Report, NSWC Contract No. N C-0051, Grumman Corporate Research Center, October Wiledon, J.C., "Applying Event Based Analysis to Specification and Designs", IFIP 1986, Elsevier Science Publishers, pp APPENDIX A TABLE OF DEFINITIONS Action: An event which is caused by the system or the environment. Condition: A condition is an expression in which the operands are conditions and the operators are Boolean. A primitive condition is a relation between a data item or "thing" and a value. At a particular instant in time, a condition is either true or false. Data Flow Diagram: A graphic representation of a system or system part, including all or some of the following nodes: data sources, data sinks, storage and processes. The logical flow of data is shown as links between the nodes. Data Flow: Data passed between system processes or between processes and the environment. Data Store: Persistent system data. relationship. Event: A condition becoming true or false. Item (in RDD): An observable thing (energy, matter, or data) that arrives at or leaves a system, component, process, or function (Ascent Logic '92). Performance Index: A quantifiable measure of a function's performance, for example, accuracy, reliability, speed, efficiency, etc. (Ascent Logic '92). State: A set of conditions, or a system variable which is based on an initial set of conditions and the history of events that affect the system. State Transition Diagram: A directed graph used for describing a system in terms of state changes. Nodes correspond to system states, and edges correspond to transitions. System Parameter: A measure of performance in which the units of measurement are directly related to the performance index. (DSMC '91). Thread: A path of system behavior that is uniquely determined by specific environmental stimuli, system conditions, and system events. A thread cuts across multiple functions, as for example, the thread in a surveillance system that produces a report on detecting a target. Author's Biography. Stephanie White, principal engineer of software process at Grumman Aerospace and Electronics, improves software and system engineering methods for use on Grumman and Navy programs. Her primary research interests are system and software requirements engineering, traceability and analysis of system behavior, and reengineering. She serves as vicechair of the IEEE Computer Society Task Force on the Engineering of Computer-Based Systems (ECBS), and previously chaired its State of Practice Working Group. She has an M.S. in mathematics from New York University and a Ph.D. in computer science from the Polytechnic University of New York. Entity-Relationship-Attribute Model (ERA Model): A model in which objects (entities) are related to their attributes and to other objects. The verb or verb phrase which describes how the objects are related is the relationship. For example, in "Process receives Input". "Process" and "Input" are entities and "receives" is the