Anders Fröberg TDDD80 STORAGE AND TESTING

Transcription

1 Anders Fröberg TDDD80 STORAGE AND TESTING 1

2 Agenda: Test Unit testing vs Traditional Testing Debugging and Refactoring Deployment (Test Driven Development (TDD)) (Acceptance Test Driven Development) 2

3 Software Faults, Errors & Failures Software Fault : A static defect in the software Software Failure : External, incorrect behavior with respect to the requirements or other description of the expected behavior Software Error : An incorrect internal state that is the manifestation of some fault 3

4 Fault and Failure Example A patient gives a doctor a list of symptoms Failures The doctor tries to diagnose the root cause, the ailment Fault The doctor may look for anomalous internal conditions (high blood pressure, irregular heartbeat, bacteria in the blood stream) Errors 4

5 The Term Bug BUG Bug is used informally Sometimes speakers mean fault, sometimes error, sometimes failure often the speaker doesn t know what it means! This class will try to use words that have precise, defined, and unambiguous meanings an analyzing process must equally have been performed in order to furnish the Analytical Engine with the It has been just so in all of my inventions. The first step is an intuition, and comes with a burst, then difficulties arise this thing gives out and [it is] then that 'Bugs' as such little faults and difficulties are called show themselves and months of intense watching, study and labor are requisite... Thomas Edison necessary operative data; and that herein may also lie a possible source of error. Granted that the actual mechanism is unerring in its processes, the cards may give it wrong orders. Ada, Countess Lovelace (notes on Babbage s Analytical Engine) 5

6 Mars Polar Lander crash site? Spectacular Software Failures THERAC-25 design Intel s Pentium FDIV fault : Public relations nightmare NASA s Mars lander: September 1999, crashed due to a units integration fault THERAC-25 radiation machine : Poor testing of safety-critical software can cost lives : 3 patients were killed Ariane 5 explosion : Very expensive Ariane 5: exception-handling bug : forced self destruct on maiden We need our software to be dependable Testing is one way to assess dependability flight (64-bit to 16-bit conversion: about 370 million $ lost) 6

7 Northeast Blackout of generating units and 256 power plants shut down Affected 10 million people in Ontario, Canada Affected 40 million people in 8 US states Financial losses of $6 Billion USD The alarm system in the energy management system failed due to a software error and operators were not informed of the power overload in the system 7

8 Costly Software Failures NIST report, The Economic Impacts of Inadequate Infrastructure for Software Testing (2002) Inadequate software testing costs the US alone between $22 and $59 billion annually Better approaches could cut this amount in half Huge losses due to web application failures Financial services : $6.5 million per hour (just in USA!) Credit card sales applications : $2.4 million per hour (in USA) In Dec 2006, amazon.com s BOGO offer turned into a double discount 2007 : Symantec says that most security vulnerabilities are due to faulty software 8

9 9

10 Enhetstest (Unit Testing) Unit Testing vs Traditional Testing Benefits of Unit Testing 10

11 Traditional Testing Test the system as a whole Individual components rarely tested Errors go undetected Isolation of errors difficult to track down 11

12 Traditional Testing Strategies Print Statements Use of Debugger Debugger Expressions Test Scripts 12

13 Unit Testing Each part tested individually All components tested at least once Errors picked up earlier Scope is smaller, easier to fix errors 13

14 Unit Testing Ideals Isolatable Repeatable Automatable Easy to Write 14

15 Why Unit Test? Faster Debugging Faster Development Better Design Excellent Regression Tool Reduce Future Cost 15

16 Concept of Unit Testing Static Unit Testing Code is examined over all possible behaviors that might arise during run time Code of each unit is validated against requirements of the unit by reviewing the code Dynamic Unit Testing A program unit is actually executed and its outcomes are observed One observe some representative program behavior, and reach conclusion about the quality of the system Static unit testing is not an alternative to dynamic unit testing Static and Dynamic analysis are complementary in nature 16

17 Static Unit Testing (Code Review) Step 1: Readiness Criteria Completeness Minimal functionality Readability Complexity Requirements and design documents Roles Moderator Author Presenter Record keeper Reviewers Observer Step 2: Preparation List of questions Potential Change Request (CR) Suggested improvement opportunities 17

18 Dynamic Unit Testing The environment of a unit is emulated and tested in isolation The caller unit is known as test driver A test driver is a program that invokes the unit under test (UUT) It provides input data to unit under test and report the test result The emulation of the units called by the UUT are called stubs It is a dummy program The test driver and the stubs are together called scaffolding Dynamic unit test environment The low-level design document provides guidance for selection of input test data 18

19 19 Unit Testing in Programming

20 20 Unit Testing in Programming

21 JUnit ( Java-based unit testing framework Elegantly simple Easy to write unit tests Mature Framework De facto java standard Ant integration Generic testing framework Easy to manage unit tests Open source = Free! 21

22 Key Concepts in (J)Unit Test interface Assert TestCase asserttrue assertequals fail TestSuite TestDecorator/TestSetup Failures vs Errors Assert TestCase setup() teardown() <Test> run(testresult) TestSuite run(testresult) 22

23 (J)Unit is Easy import unittest class TestStringMethods(unittest.TestCase): def test_upper(self): self.assertequal('foo'.upper(), 'FOO') def test_isupper(self): self.asserttrue('foo'.isupper()) self.assertfalse('foo'.isupper()) def test_split(self): s = 'hello world' self.assertequal(s.split(), ['hello', 'world']) # check that s.split fails when the separator is not a string with self.assertraises(typeerror): s.split(2) if name == ' main ': unittest.main() 23

24 Writing a Unit Test DEMO 1. Create a class to hold the unit tests 2. Initialise objects (setup() method) 3. (State assertions preconditions)* 4. Call operations on the objects that are being unit tested 5. State assertions/failures expected 6. Clean up (teardown() method) 7. Execute the unit test 24

25 Code Coverage DEMO 25