Speed: Precise and Efficient Static Estimation of Program Computational Complexity

Transcription

1 Speed: Precise and Efficient Static Estimation of Program Computational Complexity Sumit Gulwani Krishna K. Mehra Trishul Chilimbi POPL 2009 Presented by Stefan Blumer

2 Motivation

3 Performance Bugs Motivation

4 Motivation Performance Bugs Testing requires writing or generating tests

5 Motivation Performance Bugs Testing requires writing or generating tests Testing doesn't proof absence of performance bugs

6 Motivation Performance Bugs Testing requires writing or generating tests Testing doesn't proof absence of performance bugs Termination Proofers: no information about time complexity of programs

7 Solution

8 Solution Static analysis that returns symbolic complexity bounds for procedures in terms of their input

9 Solution Static analysis that returns symbolic complexity bounds for procedures in terms of their input SPEED: Input: c/c++ programs

10 Solution Static analysis that returns symbolic complexity bounds for procedures in terms of their input SPEED: Input: c/c++ programs Output: symbolic upper bounds on the runtime complexity.

11 How does it work? 1: void square(x, y) { 2: for (i:=0; i<x; i++) { 3: for (j:=0; j<y; j++) { 4: } 5: } 6: }

12 How does it work? Uses program traces 1: void square(x, y) { 2: for (i:=0; i<x; i++) { 3: for (j:=0; j<y; j++) { 4: } 5: } 6: }

13 How does it work? Uses program traces Instrumentation of counter variables to count loop iterations Single counter... 1: void square(x, y) { 2: c0:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c0++; 6: } 7: c0++; 8: } 9: }

14 How does it work? Uses program traces Instrumentation of counter variables to count loop iterations Statically compute an upper bound using an invariant generation tool Single counter... 1: void square(x, y) { 2: c0:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c0++; 6: inv: c0 <= x+x*y; 7: } 8: c0++; 9: } 10: }

15 How does it work? Uses program traces Instrumentation of counter variables to count loop iterations Statically compute an upper bound using an invariant generation tool Single counter... Invariant generation tools can't handle non-linear invariants! 1: void square(x, y) { 2: c0:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c0++; 6: inv: c0 <= x+x*y; 7: } 8: c0++; 9: } 10: }

16 How does it work? Uses program traces Instrumentation of counter variables to count loop iterations Statically compute an upper bound using an invariant generation tool Multiple Counters: 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: } 7: c0++; c1:=0; 8: } 9: } Dependencies between counters Only linear bounds on c0 and c1

17 How does it work? Uses program traces Instrumentation of counter variables to count loop iterations Statically compute an upper bound using an invariant generation tool Multiple Counters: Dependencies between counters Only linear bounds on c0 and c1 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

18 Invariant generation tool

19 Input: Procedure Invariant generation tool

20 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location

21 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location Black Box

22 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location Black Box How efficient?

23 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location Black Box How efficient? Restrictions?

24 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location Black Box How efficient? Restrictions? Implemented using abstract interpretation

25 Input: Procedure Invariant generation tool Output: linear invariants in terms of the procedure input for any program location Black Box How efficient? Restrictions? Implemented using abstract interpretation Non-linear invariant generation isn't feasible

26 Multiple counters 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

27 Multiple counters Each back-edge in the control flow graph must increment one counter 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

28 Multiple counters Each back-edge in the control flow graph must increment one counter Dependencies between counters root 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: } c0 c1

29 Multiple counters Each back-edge in the control flow graph must increment one counter Dependencies between counters No dependency cycles root 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: } c0 c1

30 Multiple counters Each back-edge in the control flow graph must increment one counter Dependencies between counters No dependency cycles Dependent counters are initialized with 0 at back-edges root 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: } c0 c1

31 Proof structure 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

32 Proof structure Let S be a set of counters {c0, c1} 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

33 Proof structure Let S be a set of counters {c0, c1} Let M be a mapping from backedges to counters {(7,c1),(10,c0)} 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

34 Proof structure Let S be a set of counters {c0, c1} Let M be a mapping from backedges to counters {(7,c1),(10,c0)} Let G be a DAG over {(root,c0),(c0,c1)} S {root} 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

35 Proof structure Let S be a set of counters {c0, c1} Let M be a mapping from backedges to counters {(7,c1),(10,c0)} Let G be a DAG over {(root,c0),(c0,c1)} S {root} Let B be a mapping from backedge to some symbolic bound {(7,c1<=y), (10,c0<=x) 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

36 Proof structure Let S be a set of counters {c0, c1} Let M be a mapping from backedges to counters {(7,c1),(10,c0)} Let G be a DAG over {(root,c0),(c0,c1)} S {root} Let B be a mapping from backedge to some symbolic bound {(7,c1<=y), (10,c0<=x)} Tuple (S,M,G,B) is a proofstructure if the invariant generation tool can find a bound for every counter 1: void square(x, y) { 2: c0,c1:=0; 3: for (i:=0; i<x; i++) { 4: for (j:=0; j<y; j++) { 5: c1++; 6: inv: c1<=y 7: } 8: c0++; c1:=0; 9: inv: c0<=x 10: } 11: }

37 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) )

38 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) Make term nonnegative ( 1+ (c', c) G TotalBound (c ' ) )

39 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) The bound of counter c

40 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) Iterate over all predecessors

41 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) TotalBound (c0)= Max(0, x) (1+TotalBound (root))= Max(0, x)

42 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) TotalBound (c0)= Max(0, x) (1+TotalBound (root))= Max(0, x) TotalBound (c1)=max(0, y) (1+TotalBound (c0)) =Max(0, y) (1+Max(0, x))

43 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) TotalBound (c0)= Max(0, x) (1+TotalBound (root))= Max(0, x) TotalBound (c1)=max(0, y) (1+TotalBound (c0)) =Max(0, y) (1+Max(0, x)) U =Max(0, x)+max(0, y) (1+Max(0, x))

44 Computing total bound from proof structure U = c S TotalBound (root)=0 TotalBound (c) TotalBound (c)= Max({0} {B(q) M (q)=c }) ( 1+ (c', c) G TotalBound (c ' ) ) TotalBound (c0)= Max(0, x) (1+TotalBound (root))= Max(0, x) TotalBound (c1)=max(0, y) (1+TotalBound (c0)) =Max(0, y) (1+Max(0, x)) U =Max(0, x)+max(0, y) (1+Max(0, x)) U =x+ y+x y If x, y >= 0

45 How to create a proof structure?

46 How to create a proof structure? How many counters? Which counter where to increment? What dependencies between counters?

47 How to create a proof structure? How many counters? Which counter where to increment? What dependencies between counters? For a procedure, multiple Proof Structures may exist! Different proof-structures may lead to different bounds! Counter dependency doesn't correspond to the loop nesting structure!

48 Optimal proof structure

49 Optimal proof structure Bound is not larger than the bound of any other proof structure

50 Optimal proof structure Bound is not larger than the bound of any other proof structure Check if proof structure is optimal Compare two proof structures Number of Triplets (S, M, G) is exponential in the number of back edges

51 Counter-optimal proof structure

52 Counter-optimal proof structure Heuristic: Minimal number of counters Minimal number of dependencies

53 Counter-optimal proof structure Heuristic: Minimal number of counters Minimal number of dependencies Greedy algorithm: 1) Chose a back edge that has no counter assigned yet 2) Try to assign an existing counter to that back-edge. If it fails, go to 3 otherwise got to 1 3) Add new counter, and make it depend on all other counters. If it fails, go to 1 otherwise go to 4 4) Remove dependencies as long as possible. Go to 1.

54 How to handle data structures 1: void worklist(list<t> l) { 2: c0 := 0; 3: while (!l.empty()) { 4: l.pop_back(); 5: c0++; 6: inv: c0<=old(size(l)); 4: } 5: }

55 How to handle data structures Quantitative functions over data structures 1: void worklist(list<t> l) { 2: c0 := 0; 3: while (!l.empty()) { 4: l.pop_back(); 5: c0++; 6: inv: c0<=old(size(l)); 4: } 5: }

56 How to handle data structures Quantitative functions over data structures No need for heap shape analysis 1: void worklist(list<t> l) { 2: c0 := 0; 3: while (!l.empty()) { 4: l.pop_back(); 5: c0++; 6: inv: c0<=old(size(l)); 4: } 5: }

57 How to handle data structures Quantitative functions over data structures No need for heap shape analysis Provide quantitative information like: size of a list, current position of an iterator 1: void worklist(list<t> l) { 2: c0 := 0; 3: while (!l.empty()) { 4: l.pop_back(); 5: c0++; 6: inv: c0<=old(size(l)); 4: } 5: }

58 How to handle data structures Quantitative functions over data structures No need for heap shape analysis Provide quantitative information like: size of a list, current position of an iterator Programmer has to define for each procedure of the data structure, how it affects the value of the quantitative function 1: void worklist(list<t> l) { 2: c0 := 0; 3: while (!l.empty()) { 4: l.pop_back(); 5: c0++; 6: inv: c0<=old(size(l)); 4: } 5: }

59 How to handle data structures 1: class List<T> { 2: bool empty() { 3: ensures 4: size(this) = old(size(this)); 5: } 6: 7: void pop_back() { 8: ensures 9: size(this) = old(size(this))-1; 10: } 11: 12: void clear() { 13: ensures size(this) = 0; 14: } 15: }

60 How to handle data structures methods: empty() pop_back() clear() 1: class List<T> { 2: bool empty() { 3: ensures 4: size(this) = old(size(this)); 5: } 6: 7: void pop_back() { 8: ensures 9: size(this) = old(size(this))-1; 10: } 11: 12: void clear() { 13: ensures size(this) = 0; 14: } 15: }

61 How to handle data structures methods: empty() pop_back() clear() Quantitative function: size(list) 1: class List<T> { 2: bool empty() { 3: ensures 4: size(this) = old(size(this)); 5: } 6: 7: void pop_back() { 8: ensures 9: size(this) = old(size(this))-1; 10: } 11: 12: void clear() { 13: ensures size(this) = 0; 14: } 15: }

62 How to handle data structures methods: empty() pop_back() clear() Quantitative function: size(list) Programmer must make sure to get the semantics right! 1: class List<T> { 2: bool empty() { 3: ensures 4: size(this) = old(size(this)); 5: } 6: 7: void pop_back() { 8: ensures 9: size(this) = old(size(this))-1; 10: } 11: 12: void clear() { 13: ensures size(this) = 0; 14: } 15: }

63 How to handle procedure calls Non-recursive procedure call: 1: void foo(x) { 2: c0 := 0; 3: for (i:=0; i<x; i++) { 4: inv: i<x; 5: bar(i); 6: c0++; 7: inv: c0<=x; 8: } 9: }

64 How to handle procedure calls Non-recursive procedure call: First compute symbolic bounds of procedure bar(x) 1: void foo(x) { 2: c0 := 0; 3: for (i:=0; i<x; i++) { 4: inv: i<x; 5: bar(i); 6: c0++; 7: inv: c0<=x; 8: } 9: }

65 How to handle procedure calls Non-recursive procedure call: First compute symbolic bounds of procedure bar(x) Compute bounds on argument i in terms of input parameter x 1: void foo(x) { 2: c0 := 0; 3: for (i:=0; i<x; i++) { 4: inv: i<x; 5: bar(i); 6: c0++; 7: inv: c0<=x; 8: } 9: }

66 How to handle procedure calls Non-recursive procedure call: First compute symbolic bounds of procedure bar(x) Compute bounds on argument i in terms of input parameter x Multiply bound on call complexity with the bound on statement execution count 1: void foo(x) { 2: c0 := 0; 3: for (i:=0; i<x; i++) { 4: inv: i<x; 5: bar(i); 6: c0++; 7: inv: c0<=x; 8: } 9: }

67 How to handle procedure calls Non-recursive procedure call: First compute symbolic bounds of procedure bar(x) Compute bounds on argument i in terms of input parameter x Multiply bound on call complexity with the bound on statement execution count Bounds on call arguments limited to linear bounds! 1: void foo(x) { 2: c0 := 0; 3: for (i:=0; i<x; i++) { 4: inv: i<x; 5: bar(i); 6: c0++; 7: inv: c0<=x; 8: } 9: }

68 How to handle procedure calls Recursive procedure call: 1: void foo(x) { 2: if (x>0) { 3: foo(x-1); 4: } 5: }

69 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

70 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

71 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables Initializes single global counter 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

72 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables Initializes single global counter Calls actual procedure foo(x) 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

73 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables Initializes single global counter Calls actual procedure foo(x) Increment counter before call 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

74 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables Initializes single global counter Calls actual procedure foo(x) Increment counter before call Compute bound for c0 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

75 How to handle procedure calls Recursive procedure call: Generate a procedure foo'(x): Stores parameters to global variables Initializes single global counter Calls actual procedure foo(x) Increment counter before call Compute bound for c0 Single global counter restricts to linear bounds of recursion! 1: void foo(x) { 2: if (x>0) { 3: c0++; 4: foo(x-1); 5: inv: c0 <= x' 6: } 7: } 8: 9: int c0; 10: int x'; 11: 12: void foo'(x) { 13: x':= x; c0 := 0; 14: foo(x); 15: }

76 Results of SPEED

77 Results of SPEED Tested on parts of c++ std and Microsoft products

78 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops

79 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops Able to get correct linear bounds on procedures with a nesting depth of up to 5

80 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops Able to get correct linear bounds on procedures with a nesting depth of up to 5 Most failures because of:

81 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops Able to get correct linear bounds on procedures with a nesting depth of up to 5 Most failures because of: Bounds depend on environment assumptions: class invariants, preconditions, concurrency and os-call properties

82 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops Able to get correct linear bounds on procedures with a nesting depth of up to 5 Most failures because of: Bounds depend on environment assumptions: class invariants, preconditions, concurrency and os-call properties Generation of bounds requires path sensitive invariant generation if (p) x++; else x--;

83 Results of SPEED Tested on parts of c++ std and Microsoft products Successful creation of bounds for more than half of the tested loops Able to get correct linear bounds on procedures with a nesting depth of up to 5 Most failures because of: Bounds depend on environment assumptions: class invariants, preconditions, concurrency and os-call properties Generation of bounds requires path sensitive invariant generation if (p) x++; else x--; Integrated into the code check-in process of some Microsoft product groups

84 Conclusion

85 Conclusion Everything built on the linear invariant generation tool

86 Conclusion Everything built on the linear invariant generation tool Black box, no guaranties

87 Conclusion Everything built on the linear invariant generation tool Black box, no guaranties Elegant solution with multiple counters

88 Conclusion Everything built on the linear invariant generation tool Black box, no guaranties Elegant solution with multiple counters Very limiting on procedure calls and recursion

89 Conclusion Everything built on the linear invariant generation tool Black box, no guaranties Elegant solution with multiple counters Very limiting on procedure calls and recursion Will be obsolete when non-linear invariant generators appear

90 Conclusion Everything built on the linear invariant generation tool Black box, no guaranties Elegant solution with multiple counters Very limiting on procedure calls and recursion Will be obsolete when non-linear invariant generators appear Few and intransparent results

91 Questions?