Shmuel Ur Shady Copty Software and Verification Technologies

Transcription

1 Multi Threaded Testing with AOP is Easy And it Finds Bugs! Shmuel Ur Shady Copty Software and Verification Technologies IBM Labs in Haifa 2004 IBM Corporation

2 Outline Noise making as a testing technique The ConTest testing tool AspectJ AOP based noise maker Results Other applications Conclusions 2 IBM Labs in Haifa 2004 IBM Corporation

3 A Simple Java Example Function increase() public void increase() { counter++; Although written as a single increment operation, the ++ operator is actually mapped into three JVM instructions [load operand, increment, write-back] 3 IBM Labs in Haifa 2004 IBM Corporation

4 A Simple Java Example - Continued Thread A Thread B Load operand Increment Write-back Context Switch Load operand Increment Write-back counter = 34 4 IBM Labs in Haifa 2004 IBM Corporation

5 A Simple Java Example Noise Makers Thread A Thread B Load operand Increment Context Switch Write-back Context Switch counter = 34 Load operand Increment Write-back Context Switch 5 IBM Labs in Haifa 2004 IBM Corporation

6 ConTest Overview ConTest is composed of the following components An instrumentation engine that Creates hooks for the irritator and for coverage printing Generates coverage models The instrumentation is done at the byte code level An irritator that randomly, or using heuristics, generates new interleaving on-the-fly Seed replay component Coverage component Debugging aids 6 IBM Labs in Haifa 2004 IBM Corporation

7 AspectJ by Example class IDManager{ public int NewID() { return ++_counter; private int _counter; public aspect Example { pointcut IssueID(): call (int IDManager.NewID()); after(): IssueID() { System.out.println( Just gave a new ID ); 7 IBM Labs in Haifa 2004 IBM Corporation

8 A more specific Aspect class IDManager{ public int NewID() { return ++_counter; private int _counter; public aspect Example2 { pointcut ReadCounter(): get(int IDManager._counter); after(): ReadCounter() { System.out.println( read the counter ); 8 IBM Labs in Haifa 2004 IBM Corporation

9 A more general Aspect class IDManager{ public int NewID() { return ++_counter; Your Application private int _counter; public aspect Example4 extends Thread{ pointcut Noise(): public aspect Example3 { (get(* *) set (* *)); pointcut AccessVariable(): (get(* *) set(* *)); after(): Noise() { try { after(): AccessVariable() { yield(); System.out.println( accessed something ); catch (Exception e) {; 9 IBM Labs in Haifa 2004 IBM Corporation

10 Woven bytecode Load operand Increment Write-back Load operand Context Switch Increment Write-back Context Switch 10 IBM Labs in Haifa 2004 IBM Corporation

11 An AOP Testing Tool public aspect SleepNoise extends Thread{ private static Random rand = new Random(); pointcut noisevictem(): ((get(* *) set (* *))&& within(!sleepnoise)); after(): noisevictem() { try{// noise if (rand.nextint(100) == 1){ // activation sleep(rand.nextint(50)); // type catch (Exception e) {; 11 IBM Labs in Haifa 2004 IBM Corporation

12 Experiment Several programs with documented bugs BubbleSort Assumed threads will finish in time and didn t synchronize BubbleSort2 Programmer sends new threads to sleep to initialize his data structure IDManager non-atomic update of an ID counter DoubleLock Deadlock caused by different order of locking 12 IBM Labs in Haifa 2004 IBM Corporation

13 Experiment - results 12 lines of AspectJ code 13 IBM Labs in Haifa 2004 IBM Corporation

14 Contest Instrumentation, AspectJ Falling Short? For Testing gets/sets for variables Concurrent related functions (sleep, yield, wait, ) Synchronization blocks In addition, for coverage Basic blocks Method calls 14 IBM Labs in Haifa 2004 IBM Corporation

15 Tampering with Timeouts public aspect SleepMutator extends Thread{ pointcut noisevictem(long i): call(void sleep(long)) && args(i) && within(!sleepmutator); private static Random rand = new Random(); void around(long i): noisevictem(i) { try{ long newsleeptime = rand.nextint((int)i*3); // [0,3*sleep] if (rand.nextint(5) == 1) proceed(newsleeptime); ; catch (Exception e) { 15 IBM Labs in Haifa 2004 IBM Corporation

16 Tampering with UDP public aspect UDP extends DatagramSocket pointcut UDPNoise(DatagramPacket i): call(void send(datagrampacket)) && args(i) && within(!udp); ; private static Random rand = new Random(); void around(datagrampacket i): UDPNoise (i) { if (rand.nextint(5) == 1) proceed(i); 16 IBM Labs in Haifa 2004 IBM Corporation

17 Coverage ConTest supports Method coverage Branch coverage Concurrent point coverage Interfered location pairs coverage Synchronization coverage 100% coverage? A new feature to AspectJ was added during our work - ShowWeaveInfo 17 IBM Labs in Haifa 2004 IBM Corporation

18 Concluding Remarks AOP is suitable for multi threaded testing Finds bugs, extremely easy AspectJ as an open source instrumentor If something is missing it is easy to add Detailed study of implementing ConTest features in the paper Source code is available on our website 18 IBM Labs in Haifa 2004 IBM Corporation

19 19 IBM Labs in Haifa 2004 IBM Corporation

20 20 IBM Labs in Haifa 2004 IBM Corporation