Software Engineering Fall 2015 (CSC 4350/6350) TR. 5:30 pm 7:15 pm Rao Casturi 11/10/2015 http://cs.gsu.edu/~ncasturi1
Class announcements Final Exam date - Dec 1 st. Final Presentations Dec 3 rd. And Dec 8 th. Ability to run the program in the class Prepare for Q&A Each member should talk about their role in the project Final deliverables & Date Dec 3 rd. One Printed copy of the whole project (All phases & Code) 2
Testing 3
Testing What is testing? Process of finding the divergence between the expected behavior of the designed system model vs implemented final system. Why testing is important? Cost to fix is high Credibility What is the goal? Who Should Do Testing? 4
Stages in Testing Unit Testing Differences between a specification of an object and its realization as a component Structural Testing Differences between the system design model and a subset of integrated subsystems Functional Testing Differences between the use case model and the system Performance Testing Differences between nonfunctional requirements and actual system performance 5
Testing Concepts Software reliability is the probability that a software system will not cause system failure for a specified time under specified conditions. Failure is any deviation of the observed behavior from the specified behavior. An erroneous state (also called an error) means the system is in a state such that further processing by the system will lead to a failure, which then causes the system to deviate from its intended behavior. A fault, also called defect or bug, is the mechanical or algorithmic cause of an erroneous state. The goal of testing is to maximize the number of discovered faults, which then allows developers to correct them and increase the reliability of the system. 6
How to increase the Software Reliability? Fault avoidance Technique Fault Detection Technique Fault Tolerance Technique 7
Fault avoidance Technique Detect faults before they can happen This includes Development methodologies: Unambiguous relationship Data Encapsulation Coupling and Coherence Configuration Management: Better communications and upgrades/edits in other parts of the system Verification Check pre and post conditions Trust but verify Review Methods. (Walkthrough and inspection) Frequent Code Review 8
Fault detection & Tolerance Techniques Detection: During the development or after the release. Debugging. Uncontrolled and controlled experiments used during the development to find faults in the system. Testing (Planned way) Tolerance: Know issues but ready to release. Fault resistance systems with redundancy built in 9
Test Planning Usability Testing Overall Testing Activities User Interface Testing Unit Testing Integration Testing & Structural Testing System Testing Functional Testing (Requirements from RAD and User Manual) Performance Testing (Non Functional and SDD) Acceptance Testing (Agreed upon project goals) 10
Testing Concepts 1. A component: a part of the system that can be isolated for testing could be an object, a group of objects or one or more subsystems. 2. A fault: bug or defect a design or coding mistake that may cause abnormal component behavior. 3. An error: manifestation of a fault during the execution of the system. 4. A failure: deviation between the specification of a component and its behavior. This is triggered by one or more errors. 11
Testing Concepts 5. A test case: a set of inputs and expected results that exercises a component with the purpose of causing failures and detecting faults. 6. A test stub: a partial implementation of components on which the test component depends. 7. A test driver: a partial implementation of a component that depends on the tested component. 8. A correction: a change to a component repair a fault. 12
Why do Faults, errors and failures occur? Communication. Subsystem Integration Issues. Resource rotation. Lack of proper design and model documentation. Fault and Erroneous state Fault by Algorithmic or Mechanical Cause Source: OOSE-Bernd Bruegge & Allen H. Dutoit 13
Example Use Case Test Case Source: OOSE-Bernd Bruegge & Allen H. Dutoit Source: OOSE-Bernd Bruegge & Allen H. Dutoit 14
Test Cases: A test case is a set of input data and expected results that exercises a component with the purpose of causing failures and detecting faults. A test case has five attributes: name, location, input, oracle, and log Source: OOSE-Bernd Bruegge & Allen H. Dutoit 15
Black Box and White Box Testing Blackbox tests : focus on the input/output behavior of the component. Blackbox tests do not deal with the internal aspects of the component, nor with the behavior or the structure of the components. Functionality of the System Whitebox tests focus on the internal structure of the component. A whitebox test makes sure that, independently from the particular input/output behavior, every state in the dynamic model of the object and every interaction among the objects is tested. As a result, whitebox testing goes beyond blackbox testing. Structural and Dynamic aspects of the components Unit testing includes both methods. 16
Black-Box Testing Black-box testing is deferred to be done later in the software development process, unlike white-box testing which is done earlier. Some questions that are used to guide a black-box test are: 1. How is functional validity tested? 2. What classes of input will make good test cases? 3. Is the system particularly sensitive to certain input values? 4. How are the boundaries of a data class isolated? 5. What data rates data volumes can the system tolerate? 6. What effect operation? will specific combinations of data have on system 17
White-Box Testing This test uses the control structure to design the test cases. The tests derived to conduct white-box testing: 1. Guarantee that all independent paths within a module have been exercised at least once. 2. Exercise all logical decisions on their true and false sides. 3. Execute all loops at their boundaries and within their operational bounds. 4. Exercise internal data structures to assure their validity. 18
Test Stubs and Drivers Test drivers and test stubs are used to substitute for missing parts of the system. A test driver simulates the part of the system that calls the component under test. A test driver passes the test inputs identified in the test case analysis to the component and displays the results. A test stub simulates a component that is called by the tested component. TEST DRIVER CALLS COMPONENT TESTED CALLS TEST STUB 19
Correction: Once a problem has been found, then corrections are made. Corrections could be simple and applied to only the component that is under test or they could be more involved requiring changes to an entire class or subsystem. There may be cases whereby entire subsystems need to be redesigned, which may eventually introduce new errors. The authors suggested several techniques that can be used to track and handle any new faults that may evolve: Problem tracking: once documentation of the entire process of finding errors and correction is kept, then it is easy to revise those portions of the system with the intent of finding faults. Regression testing: re-execution of all prior tests after a change. Rationale maintenance: justifying the changes that are made with the requirements of the subsystem. 20
Component Inspection Inspections find faults in a component by reviewing its source code. It could be done before or after the unit test. Fagan suggested a fivestep method for inspection: 1. Overview: The author of the component briefly presents the purpose and scope of the component and the goal of the inspection. 2. Preparation: The reviewers become familiar with the implementation of the component. 3. Inspection meeting: A reader paraphrases the source code (reads each source code statement and explains what the statement should do) of the component and the inspection team raise issues with the component. A moderator keeps the meeting on track. 4. Rework: The author revises the component. 5. Follow-up: The moderator checks the quality of the rework and determines the component needs to be re-inspected. 21
Unit Testing Unit Testing focuses on the building blocks of the software system. 3 major points whey we Unit Testing is done. 1. Reduce Complexity. 2. To pinpoint the issues and correct. 3. Introduces parallelism in testing. 22
Equivalence testing Equivalence testing: a black-box testing technique that minimize the number of test cases. The possible inputs are partitioned into equivalence classes and a test case is selected for each class. Only one member of an equivalence class needs to be tested. The test consists of two steps: identification of the equivalence class and the selection of the test inputs. To identify equivalence class we use: Coverage: Every possible input belongs to one of the equivalence classes. Disjointedness: No input belongs to more than one equivalence class. Representation: If the execution demonstrates an error when a particular member of an equivalence class is used, then the same error should appear if any other member of the class is used. For each equivalence class, two pieces of data is used a typical input and an invalid input. 23
Boundary Testing Focuses on the conditions at the boundary of the equivalence class. The testing requires that elements be selected from the edges of the equivalence class. Example to find the Number of days in a given month: Generally years that are multiple of 4 are leap years, but note that years that are multiple of 100 are not leap years unless they are multiple of 400, even though they may be multiple of 4. Example 2000 is a leap year but 1900 was not a leap year, even though it was a multiple of 100 and 4. Hence, both 1900 and 2000 are good boundary cases for year 0 and 13 would be good boundary cases for month. 24
Path Testing A white-box testing technique that identifies faults in the implementation of the component. The idea behind path testing is that each line of code would be tested at least once, and thus if any fault (error) exist in any of the paths tested, it would be found. Flow Graph is constructed A flow graph consists of nodes representing executable blocks and edges representing flow of control. A flow graph is constructed from the code of a component by mapping decision statements (e.g., if statements, while loops) to nodes. Statements between each decision (e.g., then block, else block) are mapped to other nodes. Associations between each node represent the precedence relationships. 25
Code for Days in a Month Class Source: OOSE-Bernd Bruegge & Allen H. Dutoit 26
Flow Graph Source: OOSE-Bernd Bruegge & Allen H. Dutoit 27
Integration Testing Big bang testing: Assumes that all components are tested individually and then are put together and tested. This test could be expensive, because if a fault is found when doing the big test, it would be difficult to locate and fix, especially for huge programs. Also interface failure may be difficult to distinguish from component failures. Bottom-up testing: all components of the bottom layer are tested individually and then integrated with the layer up. This is continued until the entire system is tested. Note: when two components exist at the same level and are tested together, it is known as a double test (if three components are tested together, it is known as triple test, and four together as quadruple test). Test drivers are used to simulate the components that are not tested. Top-down testing: The reverse of the bottom-up test; i.e. components from the top layers are tested first and progressively integrate the lower layers. 28
System Testing Functional Testing: test of functional requirements from use cases. Performance Testing: test of nonfunctional requirements. Pilot Testing: tests of common functionality among a selected group of end users. Acceptance Testing: usability, functional and performance tests done at the developers environment by the customer against the acceptance agreement. Installation Testing: usability, functional and performance tests done at the customer environment by the customer against the acceptance agreement. 29
Performance Testing Stress testing: checks to see if the system can respond to many simultaneous requests. Volume testing: attempts to find faults associated with large amounts of data such as static inputs imposed by the data structure, etc. Security testing: attempts to find security faults in the system. Few systematic methods exist to carry out such test. Usually this is carried out by teams of individuals who try to break into the system using their experience and knowledge. Timing tests: attempts to find behavior that violates timing constraints described by the nonfunctional requirements. Recovery tests: evaluate the ability of the system to recover from errors such as hardware failure, etc. After all of these tests are completed without finding any errors, then the system is said to be validated. 30
Pilot & Acceptance Testing Alpha Users and Beta Users Three ways in which the client evaluates a system during acceptance testing: Benchmark test: a set of test cases are prepared that represents typical conditions under which the system will operate. Competitor test: used when a new system is replacing an old system. They are tested against each other. Shadow test: new and old system run in parallel and their outputs compared. If all is well, then the customer accepts the system. If not, the developers are notified pertaining as to what is wrong. The developers will modify, delete or add conditions as specified by the client. 31
Questions? 32
References Use Cases Combined with BOOCH, OMT UML Process and Products - Putnam P Texel, Charles B Williams Object-Oriented Software Engineering Using UML Patterns, and JAVA -Bernd Bruegge & Allen H. Dutoit Software Engineering 9th Edition - Ian Sommerville 33