Session 1b: Overview of systems analysis methodology The system specification (ESD) Ways of documenting an ESD Criteria to be satisfied Historical survey Comp 320 / 420, spring, 2018 Alternative names for the main product of systems analysis Functional specifications (Detailed) User requirements External system design (ESD) Whatever we call it, it must describe: everything about what the proposed new application system will do nothing (or as little as possible) about how. Why not? Criteria for the ESD This is (or should be) non-controversial! 1. Must be understandable in complete detail by How do we know? 3. Must provide a reasonable basis for estimating the rest of the project. Experts may disagree about how well a particular technique supports the criteria, but let's agree on the criteria themselves. Why? A non-criterion for the ESD "The ESD is final. The specifications are frozen until the new system is installed." We hear this one not as a serious proposal, but more often as a canard accusation from those who favor doing away with rigorous specifications altogether! No matter what technique we choose for documenting the ESD, we must be able to respond at any time during the project to: new insights changing environment discovering errors or omissions COMP 320 / 420 1-4 copyright Conrad Weisert
Rough chronology of ESD documentation methodology ~1959: Iterative development ~1966: "Victorian novel" ~1980: Structured analysis ~1990: Object-oriented analysis ~1996: Unified Modeling Language ~1999: Discrete requirements list ~2004: Use-cases stories ~2008: Iterative development! Which ones satisfy our criteria for the ESD? Iterative software development Programmer and sponsoring end user confer; programmer takes notes. Programmer develops partial "solution" based on understanding of the problem Programmer shows output to sponsor. What needs to be changed or added? Programmer revises program accordingly Which ESD criterion does this violate? nothing Done Criteria for the ESD 1. Must be understandable in full detail by 3. Must provide a reasonable basis for estimating cost and time for development. (NOTE: "Iterative development" violates this criterion) "Victorian novel" approach Analyst prepares a collection of documents consisting of: narrative description of everything the proposed system is to do, usually in processing sequence. flow charts showing processing sequence and decision logic record layouts for master files, work files, input transations, and output reports. Derogatory term coined by Tom DeMarco reflects massive size, especially of the narrative sections. Which ESD criteria does this violate? COMP 320 / 420 5-8 copyright Conrad Weisert
Criteria for the ESD 1. Must be understandable in full detail by 3. Must provide a reasonable basis for estimating. The structured revolution ~1972: Structured programming publicized by Harlan Mills et al. as a systematic approach to software organization It consisted of Structured coding (go-to-less flow control) Structured design (highly modular with minimal coupling, etc.) Top-down stepwise refinement deveopment sequence It put a floor under how bad programs could be. It was considered a huge success. But systems analysis was still a major impediment to smooth, predictable software development. More structured revolution ~1975: We ought to be able to do for analysis something similar to SP ~1979: Structured analysis Published by: Gane & Sarson Tom DeMarco (others) Not directly related to structured programming Not a way of doing analysis, but just of documenting the results, i.e. the System specification Characteristics of SA Centered on a system model Emphasis on graphical rather than narrative documentation. Hierarchical organization A definite place to start Every component is tied to its context We know when the specification is complete The dataflow diagram ties everything together provides an understandable picture of the system COMP 320 / 420 9-12 copyright Conrad Weisert
Structured specification components 1. Dataflow diagrams showing a. processes b. data stores c. interfaces to users and to other systems d. dataflows between pairs of the above 2. Data dictionary showing a. composition of composite data items b. rigorous definition of elementary data items 3. Process specifications a. algorithms b. decision rules Which ESD criteria does this violate? Criteria for the ESD 1. Must be understandable in full detail by 3. Must provide a reasonable basis for estimating. Object-oriented analysis (OOA) ~1990: Object-oriented programming was becoming well established. By analogy with structured programming inspiring structured analysis, experts tried to apply object oriented concepts to systems analysis What object-oriented concepts? Early OOA books Coad & Yourdon (1991) Booch (1994) Rumbaugh (1991) Object-oriented characteristics Emphasis on the data model Class structure (Abstract Data Type) Always Hidden internal representation Encapsulated operations (methods, functions) Potential for Inheritance Polymorphism What do all of those mean? Why are they desirable? COMP 320 / 420 13-16 copyright Conrad Weisert
Shortcomings of early OOA Incomplete set of tools for modeling a system specification. Emphasis was on the data model (class hierarchy). Weak on processes Confusing competing approaches (especially graphical symbols). OOA was initially a flop. What could be done to rescue it? UML OOA was considered a good idea, but the confusion of different versions slowed acceptance. Unified Modeling Language eliminated differences among competing OOA versions Booch & Rumbaugh both hired by Rational; told to cooperate on a single set of graphical conventions Blessed as a standard by OMG Who's that? UML has been immune from most criticism "It's not a methodology, only a language." Therefore UML promoters reject criticisms on methodology grounds. "It's not for system specification, but for system modeling." Therefore UML promoters reject criticisms on understandability grounds. Which ESD criteria does UML fail to satisfy? Criteria for the ESD 1. Must be understandable in full detail by 3. Must provide a reasonable basis for estimating. (more to come about UML) COMP 320 / 420 17-20 copyright Conrad Weisert
Repairing UML It was clear that UML's diagrams were incomprehensible to non-technical people without special training. In response Ivar Jacobson suggested adding use-case scenarios to UML. He had also joined Rational and contributed with Booch & Rumbaugh to UML BUT there is nothing object-oriented about use cases! You can do OOA without use cases You can use use cases without OOA What are use cases? A return to sequential narratives that were popular in the 1960s More modern notations Emphasis on mainline process; worry about exceptions later cf. agile "stories" Unstructured; hard for a reader to know where to start, when to stop, how individual use cases are related Let's look at UML with use cases Is the UML a methodology component Of course. When UML gurus claim it's not, they mean only that it's not a standard process or system development life cycle. Actually it is closely tied to its own standard life cycle! It's called the UP (but it has nothing to do with a part of Michigan). The UML's Life Cycle Since UML is process-independent, UML gurus advise that it's OK to follow any SDLC you like, as long as that life cycle is: use-case driven architecture centric iterative and incremental By the way, here's one ("Objectory" -- Jacobson): 1. inception phase 2. elaboration phase 3. construction phase 4. transition phase Rational has renamed it. RUP 5 COMP 320 / 420 21-24 copyright Conrad Weisert
Objectory (UP) project phases 1. Inception phase: Like traditional "project initiation" + "business requirements" Defines: project scope initial justification 2. Elaboration phase: Is this where we do most of the systems analysis? Like traditional "functional specifications" or "external design" "During the elaboration phase you need to get a good handle on the requirements and their relative priorities." - Fowler p. 17 Objectory phases (continued) Elaboration phase (another view): "... usually uses one iteration cycle and involves the following activities: Elaborate the specification of the effort from the previous phase. Baseline and completely delimit scope, objectives, and requirements. Baseline the business case by solidifying the vision, solidifying success criteria, and mitigating the highest risks. Baseline the plan with a schedule. Distribute the requirements among multiple iteration cycles within the construction phase. more... Objectory phases (continued) Elaboration phase (continued): Focus on understanding or forming a notion of the problem to determine the requirements that the problem imposes on its solution..., establishing and verifying the foundation for the overall solution (architectural design) and distributing the requirements among the iteration cycles of the construction development phase. - Alhir, p. 23 Clearing it all up "During the elaboration phase, as we have already noted, we build the architecture. We identify the use cases that have a significant impact on the architecture. We realize these use cases as collaborations. It is in this way that we identify most of the subsystems and interfaces -- at least the ones that are architecturally interesting. Once most of the subsystems and interfaces are identified, we flesh them out, that is, write the code that implements them. Some of this work is done before we release the architectural baseline and it continues throughout all of the workflows." - Ivar Jacobson COMP 320 / 420 25-28 copyright Conrad Weisert
Objectory phases (continued) 3. Construction phase (Fowler p. 15): "... consists of many iterations, in which each iteration builds production quality software... that satisfies a subset of the requirements....each iteration contains all the usual life-cycle phases of analysis, design, implementation, and testing. In principle you can start at the beginning: Pick some functionality and build it, pick some other functionality, and so forth. However it is worthwhile to spend some time planning." Wait! There's more. Objectory phases (continued) 3. Construction phase (UML UG p. 34): "... the third phase of the process, when the software is brought from an executable architectural baseline to being ready to be transitioned to the user community. Here also, the system's requirements and especially its evaluation criteria are constantly reexamined against the business needs of the project, and resources are allocated... to actively attack risks to the project." Which version do we like better? Objectory phases (continued) 4. Transition phase: Like the traditional "installation phase" + ongoing maintenance + more "iterations". When does the project end? Objectory phases (concluded) Summary: A little of this and a little of that in each phase. Emphasis on system architecture from the start poses serious problems: obscures the business problem or opportunity blurs distinction between analysis and design What else? Descriptions by gurus marred by vagueness and lack of rigor ("get a handle on", "form a notion of", etc.) COMP 320 / 420 29-32 copyright Conrad Weisert
Hold everything! We forgot something important A widespread policy for application system development in organizations "We will develop custom application software only when it is shown that no suitable packaged software product exists." Not a variant, but the mainstream in many, probably most, user organizations since ~1980 This policy is foreclosed when requirements "emerge" by iteration. Is that obvious? How does it affect systems analysis? Five views of a system Design view Process view use-case view Implementation view Deployment view from UML Users' Guide, p. 31 What ties them all together? Who makes sure they're consistent? Modeling different views "When you model a system from different views, you are in effect constructing your system simultaneously from multiple dimensions.... If you do a poor job of choosing these views or if you focus on one view and the expense of all others, you run the risk of hiding issues and deferring problems that will eventually destroy any chance of success." - UML UG p. 98 How do you decide? So, which ESD criteria are satisfied by UML with use-cases? COMP 320 / 420 33-36 copyright Conrad Weisert
Criteria for the ESD 1. Must be understandable in full detail by 3. Must provide a reasonable basis for estimating. Lots more kinds of UML diagram Class diagrams Object diagrams Package diagrams Activity diagrams Interaction diagrams sequence diagrams collaboration diagrams State-transition diagrams Deployment diagrams What's the problem with that? Class diagrams "Static design view" of a system One point of similarity between UML and other versions of OOA (e.g. Coad & Yourdon) For each type of data item, specifies: Its attributes (components / "has-a" hierarchy) Its behavior (methods / interface) For related classes: Inheritance ("is-a" hierarchy) Entity-relationships Logical database design Some UML issues (?) Assumes custom architecture and in-house software development; little provision for evaluating, selecting, and integrating application program packages There's no clear starting point: for creating the specification for reading the specification-+ There's no sure way of knowing when the model is complete. Relies on the system modeler to keep multiple independent diagrams and other documents consistent with one another COMP 320 / 420 37-40 copyright Conrad Weisert
A giant step backward After it became clear that UML, even with use-cases, was incomprehensible to most non-technical audiences, UML promoters: did not suggest any ways of improving UML, but instead recommended separating requirements (before we start UML) from modeling & design This has led to yet another approach to documenting requirements: The discrete requirements list. Another phase before UML's 4 phases! Discrete requirements list Unstructured list, usually numbered, of prescriptive English statements in a form like: "The system shall... Some are functional; others "non-functional" What's that? Which ESD criteria does that satisfy? Criteria for the ESD 1. Must be understandable in full detail by It all sounds plausible, so they think they've understood. 3. Must provide a reasonable basis for estimating. A recommended 2018 methodology for documenting the results of systems analysis Continue using a modernized Structured Systems Analysis (e.g. per De Marco, 1978) as the framework. Use O.O. class diagrams, in the data dictionary encapsulated behavior where appropriate, inheritance. What else? Continue following a disciplined SDLC. What SDLC? COMP 320 / 420 41-44 copyright Conrad Weisert
A sample Life Cycle for Application System Development 1. Project definition 2. Business requirements specification 3. External design What are other names for this phase? 4. System architecture 5. Construction 6. Installation 7. Review In this course, we don't care what happens in these later phases. The Waterfall Approach There's no such thing among responsible and knowledgeable project managers! The term was coined as a pejorative by methodologists trying to discredit the phased SDLC. It implies extreme inflexibility: Once the deliverables from phase N are approved, you must move on to phase N+1; you can't return to phase N or an earlier phase! Until the deliverables from phase N are approved, you may not start on phase N+1! Examine phase deliverables Re-do part of phase, or.. End-of-phase decisions Satisfactory? Y Review budget & schedule for next phase N Content Funding Review Review Y More funds available? N Abort project N Assess costs, benefits, & risks Justified? Y Next phase Alternatives to a formal ESD A. "Emergent specifications" A return to iterative development Test-driven development can use the test cases as specifications. What's wrong with that? B. Users' manual If you have to prepare a manual anyway, why not use it as the specification? What kind of system would that work for? Do all modern applications have a detailed users' manual? COMP 320 / 420 45-48 copyright Conrad Weisert
Methodology choices for this course We've seen that the practice of systems analysis in 2018 is characterized by vigorous disgreements among practitioners and even "experts". In this course we shall: Follow mainstream approaches that have been shown to work well. Describe some of the popular alternatives to those approaches, citing pros and cons. Other alternatives for course assignments & examinations What if you strongly prefer some alternative approach, because: you work for a company that has adopted it? you took some other course that persuasively recommended it? you've formed your own well-thought-out independent judgment? That's all right, as long as you: Understand the mainstream methods we present in this course. Can explain why your approach is more appropriate than the methods presented in the course. Raise questions if you're unsure. COMP 320 / 420 49-52 copyright Conrad Weisert