Replaying and Isolating Failing Multi-Object Interactions. Martin Burger Andreas Zeller Saarland University

Size: px

Start display at page:

Download "Replaying and Isolating Failing Multi-Object Interactions. Martin Burger Andreas Zeller Saarland University"

Matthew Harrell
6 years ago
Views:

1 Replaying and Isolating Failing Multi-Object Interactions Martin Burger Andreas Zeller Saarland University

3 client written in Java 100,200 LOC ~ 1,600 Java classes 17 developers Actively developed from February 2001 to August 2007

6 Debugging reproduce the original failure - manually by using GUI - test driver that reproduce faulty behavior

7 Debugging reproduce the original failure - manually by using GUI - test driver that reproduce faulty behavior 2. fix the actual defect - focus on relevant behavior - simplify faulty behavior

8 Faulty Component Behavior component application set of classes

9 Faulty Component Behavior crosses border interaction - method calls - field accesses - exception flow -... component application set of classes

10 Faulty Component Behavior crosses border interaction - method calls - field accesses - exception flow -... failure component application set of classes

11 Reproducing component

12 Reproducing 1. interaction component

13 Reproducing 1. interaction 2. component computes in the same way component

14 Reproducing 1. interaction 3. failure reproduced 2. component computes in the same way component

15 Simplifying

16 Simplifying many interactions

17 Simplifying many interactions replaying causes original failure

18 Simplifying minimal set of interactions replaying causes original failure

19 Simplifying delta debugging binary search minimal set of interactions replaying causes original failure

20 Implementation: JINSI debugging tool to reproduce and simplify failing components in Java programs reproduce capture and replay of interactions at component level simplify delta debugging minimizes interactions Isolating Relevant Component Interactions with JINSI. Orso + Joshi + Burger + Zeller WODA 2006

21 Original Import Run address book importer (GUI) Columba

22 Capture address book importer (GUI) Columba JINSI instrumentation

23 Capture address book importer (GUI) incoming call - method name Columba - arguments JINSI instrumentation written to event log

24 Replay address book importer (GUI) Columba

25 Replay address book importer (GUI) JINSI replaces Columba

26 Replay address book importer (GUI) JINSI replaces Columba read from event log

27 Replay original failure address book reproduced importer (GUI) JINSI replaces Columba read from event log

28 Delta Debugging address book importer (GUI) JINSI uses delta debugging event log

29 Delta Debugging AddressbookImporter() address book importer (GUI) JINSI uses delta debugging event log

30 Delta Debugging AddressbookImporter() init(file) address book importer (GUI) JINSI uses delta debugging event log

31 Delta Debugging AddressbookImporter() init(file) wizardfinished() address book importer (GUI) JINSI uses delta debugging event log

32 Delta Debugging AddressbookImporter() Problem init(file) - already minimal - too high level Solution wizardfinished() address book - much finer grained importer (GUI) JINSI uses delta debugging event log

33 Inside the Component coarse grained view black box (JINSI 2006)

34 Object Interactions object finer grained view

35 Object Interactions object finer grained view

36 Object Interactions object finer grained view

37 Object Interactions object finer grained view

38 Object Interactions captured by JINSI border crossing interactions are captured by JINSI 2006 captured by JINSI event log

39 Object Interactions all object interactions are captured by JINSI 2008

40 Object Trace all object interactions are captured by JINSI 2008 event log object trace for all objects

41 Importing the Addresses Object Trace ContactModel() setsortstring() setformattedname() setnickname() setfamilyname() addressbook. model (19 classes) 49,849 more JINSI

42 Importing the Addresses Object Trace ContactModel() setsortstring() setformattedname() setnickname() calling addressbook. model delta debugging: setfamilyname() 49,849 more > 2 hours JINSI (19 classes)

43 Object Slice dynamic backward slice: subset that may have influenced a specific object focuses on objects, not on statements objects are a natural abstraction

44 Object Interactions

45 Object Interactions suspicious object

46 Object Slice at first, include suspicious object suspicious object

47 Object Slice include interacting objects

48 Object Slice include objects that interact with those already in slice

49 Object Slice stop when reached largest connected graph

50 Object Slice simplified set of failure-inducing interactions within the component

51 Object Slice simplified set of failure-inducing interactions within the component

52 Importing Addresses Object Slice ContactModel() setgivenname() setfamilyname() setsortstring() getsortstring() getgivenname() addressbook. model (19 classes) getpreferred () JINSI

53 Delta Debugging apply delta debugging to object slice

54 Delta Debugging minimized set of failure-inducing interactions within the component original failure

55 Importing Addresses Delta Debugging ContactModel() addressbook. model (19 classes) getpreferred () JINSI

56 Slicing + Delta Debugging Incoming Interactions > 2 hours delta debugging 2 minimized 49,855 original run

57 Slicing + Delta Debugging Incoming Interactions > 2 hours delta debugging 2 minimized 49,855 object slice 7 original run simplified

58 Slicing + Delta Debugging Incoming Interactions > 2 hours delta debugging 2 minimized 49,855 original run object slice 7 delta debugging 2 minimized simplified

59 Slicing + Delta Debugging Incoming Interactions > 2 hours delta debugging 2 minimized 49,855 original run < 1 min object slice 7 delta debugging 2 minimized simplified

60 Slicing + Delta Debugging Incoming Interactions > 2 hours delta debugging 2 minimized 49,855 original run < 1 min object slice < 1 min 7 delta debugging 2 minimized simplified

61 Method from Minimal Trace public String getpreferred () { Iterator it = get iterator(); // get first item I Model model = (I Model) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without if (model == null) return null; return model.getaddress(); }

62 Method from Minimal Trace public String getpreferred () { Iterator it = get iterator(); // get first item I Model model = (I Model) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without if (model == null) return null; return model.getaddress(); }

63 Method from Minimal Trace public String getpreferred () { Iterator it = get iterator(); // get first item I Model model = null; (I Model) it.next(); if (it.hasnext()) model = (I Model) it.next(); // backwards compatiblity -> its not possible // anymore to create a model without if (model == null) return null; } return model.getaddress();

64 Related Work Test Factoring David Saff et al. Selective Capture-Replay Shrinivas Joshi et al. JINSI is based on these methods Efficient Test Case Minimization Andreas Leitner et al. method calls in random tests; static slicing ReCrash Shay Artzi et al. ReCrash focuses on stack trace JINSI focuses on object interactions

65 Future Work application failing but not defective component

66 Future Work application failing but not defective component

67 Future Work when observing application: too many application interactions failing but not defective component

68 Future Work application failing but not defective component

69 Future Work application failing but not defective component

70 Future Work Cause-Effect-Chain final state e.g. contact model

71 Future Work Cause-Effect-Chain intermediate states final state e.g. contact model

72 Future Work Cause-Effect-Chain input e.g. keystrokes in GUI intermediate states final state e.g. contact model

73 Future Work Capture Component Level application component JINSI captures component interaction event log

74 Future Work Replay Component Level component JINSI replays component interaction... event log

75 Future Work Replay Component Level component JINSI replays component interaction... event log

76 Future Work Replay Component + Capture Objects... and captures object interactions for all objects object trace component JINSI replays component interaction... event log

77 Challenges capture constructors - super call - constructor call as argument who is calling - caller registry to get caller of method

78 Debugging reproduce the original failure - manually by using GUI - test driver that reproduce faulty behavior 2. fix the actual defect - focus on relevant behavior - simplify faulty behavior

79 Debugging reproduce the original failure! - manually by using GUI - test driver that reproduce faulty behavior 2. fix the actual defect - focus on relevant behavior - simplify faulty behavior

80 Debugging reproduce the original failure! - manually by using GUI - test driver that reproduce faulty behavior 2. fix the actual defect - focus on relevant behavior - simplify faulty behavior!

81 Contributions

82 Contributions

83 Contributions

84 Contributions

85 Contributions

Reproducing Crashes. Shifting Role of Testing. Shifting Role of Testing. Waterfall Model, V-Model

Reproducing Crashes. Shifting Role of Testing. Shifting Role of Testing. Waterfall Model, V-Model Reproducing Crashes Andreas Leitner Universität des Saarlandes 11.12.2008 1 Shifting Role of ing ing often ne towards release-time Modern development processes move testing to the center of development