CS18000: Programming I Testing Basics 19 April 2010 Prof. Chris Clifton Testing Programs Your programs are getting large and more complex How do you make sure they work? 1. Reason about the program Think carefully about what it does Not what you think it should do 2. Test But what to test for? 4/21/2010 CS18000 2 2010 Chris Clifton 1
Goals of Testing 1. Ensure program does what it should Does expected functions correctly. 2. Identify when program behaves improperly Does not behave badly when asked to do the unexpected. In practice, these are two different things! 1. Determine how to fix errors 2. Assist in program maintenance 4/21/2010 CS18000 3 Types of Testing Black-Box Testing Based purely on program specifications White-Box Testing Exercise the code (as written) Regression Testing Re-run tests when changes made 4/21/2010 CS18000 4 2010 Chris Clifton 2
Black-Box Tests Boundary Value Testing If known ranges of inputs, test at ends of ranges Both inside and outside of bounds Equivalence Partitioning Choose one (or more) from a group of functionally similar values All-Pairs Testing Test different pairs of input values Fuzz Testing Random choices of (potentially unlikely) values 4/21/2010 CS18000 5 Black-Box Testing Example Foo f[] = new Foo[maxF]; Foo checkfoo ( int top; boolean newfoo ) { /* If newfoo, return a new Foo. Otherwise check all Foo in f before top, return top. */ } Boundary Value Testing top=-1, 0, maxf, maxf+1 Equivalence Partitioning Negative top top where f not null top where f null top >maxf All-Pairs Testing Values for top with newfoo true values for top with newfoo false Fuzz Testing 4/21/2010 CS18000 6 2010 Chris Clifton 3
White-Box Tests Branch Testing Make sure every statement is exercised Ensure each side of a decision (if-then or loop) is taken Loop Testing Like boundary case testing for loops All-Paths testing Make sure every path through the program is tested Gets to be very expensive Exponential growth in number of cases 4/21/2010 CS18000 7 White-Box Testing Example Foo f[] = new Foo[maxF]; Foo checkfoo ( int top; boolean newfoo ) { /* If newfoo, return a new Foo. Otherwise check all Foo in f before top, return top. */ int i = 0; if ( newfoo ) return new Foo; while ( i < top ) f[i++].check(); return f[i]; } Branch Testing newfoo true newfoo false and top>0 Loop Testing newfoo false and: top <= 0 top = 1 top = maxf All-Paths Testing newfoo true newfoo false and top <= 0 newfoo false and top >= 0 4/21/2010 CS18000 8 2010 Chris Clifton 4
System Testing Test the entire program Testing Scope And the environment the system runs in Unit Testing Test individual modules Use skeletons / stub procedures, Driver code Integration Testing Test interfaces between units 4/21/2010 CS18000 9 Testing Tools Lots of automated support to Run tests Generate tests Take Software Engineering (CS 30700) to learn more JUnit: Unit Testing Tool for Java Generates tests But requires code be annotated 4/21/2010 CS18000 10 2010 Chris Clifton 5
JUnit Use 1. Download.jar file from http://www.junit.org 2. Add import org.junit.*; to class to be tested 3. Annotate methods to be tested @Test public void methodtotest() { } 4. Compile with JUnit javac classpath.;junit-4.5.jar ClassToTest.java 5. Run the tester on your code java classpath.;junit-4.5.jar org.junit.runner.junitcore ClassToTest 4/21/2010 CS18000 11 More on JUnit Can only test some methods public void return no arguments Additional annotations @Before: Run this method before any test @After: Run this method after any test Assertion extentions import static org.junit.assert.*; asserttrue( expression ); assertequals ( value1, value2 ); 4/21/2010 CS18000 12 2010 Chris Clifton 6
CS18000: Programming I Code Management 21 April 2010 Prof. Chris Clifton Managing Large Projects: Issues Keeping track of all the code Many classes / modules Multiple languages/systems Dealing with multiple developers Managing changes to an entire system Code Specifications / Requirements Documentation Development tools 4/21/2010 CS18000 14 2010 Chris Clifton 7
Tools to Help Build Management When we compile (install, etc.), ensure we don t forget anything Not just compilation Source code versioning repositories Track changes Keep old versions Manage concurrent changes 4/21/2010 CS18000 15 Build Management Tools javac compiles all needed files generates.class files from.java files recompiles any that have changed does it in the proper order (dependencies) Not built in to all compilers Need to specify Forgetting can lead to bad results 4/21/2010 CS18000 16 2010 Chris Clifton 8
What if? void caller() { String int } x = callee(3); I Recompile caller int String callee (int y) { } return y+1; Integer.toString(y+1); I forget to recompile callee 4/21/2010 CS18000 17 Solutions Compilation tools that check dependencies Works for things written in that language But what about other things? (e.g.,.java file generated from main.xml) Software Development Environments Eclipse Android SDK Make (primarily for Unix systems) 4/21/2010 CS18000 18 2010 Chris Clifton 9
Make General-purpose build management Independent of language Specify rules Dependencies Transformation Run make to perform transformations needed to meet dependencies make Lab13activity make TestLab13 make installonemulator make installonphone Lab13activity : Lab13activity.class LinkedList.class R.java : main.xml %.class : %.java javac %.java Really quite complex / powerful almost a programming language 4/21/2010 CS18000 19 Code Management Tools Safe storage Prevent accidental deletion Versioning Undo changes made in error Get back to a previous version Concurrency Multiple programmers don t overwrite each other s work Centralized Repository Developers work on their own machines 4/21/2010 CS18000 20 2010 Chris Clifton 10
Code Management Examples Language-specific software development environments Language independent tools RCS (Revision Control System) cvs (Concurrent Versions System) svn (Subversion) Each has advantages and disadvantages 4/21/2010 CS18000 21 Example: cvs Repository Directory in a file system Can be on a remote machine Never edit files in the repository Project Create initially with a set of files Stores these in the repository 4/21/2010 CS18000 22 2010 Chris Clifton 11
Example: cvs Checkout Get complete copy of a project from current files in the repository Update Bring checked out files up to date with repository Flag or merge files that you have changed and have been changed in the repository Commit Put your changes in the repository Creates a new version Requires that you add a comment (placed in each file updated) 4/21/2010 CS18000 23 Summary Large projects need added support Automation to prevent little mistakes Managing multiple developers Two key areas Build management Source code management Don t worry about the details now Go with what is in use wherever you are Just make sure you use something 4/21/2010 CS18000 24 2010 Chris Clifton 12