Logic-Flow Analysis of Higher-Order Programs

Transcription

1 Logic-Flow Analysis of Higher-Order Programs Matt Might POPL 2007

2 Why? Tim Sweeney, POPL 2006 Static array-bounds checking. Example... a[i]... Will 0 i < a.length always hold?

3 3 Why? Tim Sweeney, POPL 2006 Static array-bounds checking. Example... a[i]... Will 0 i < a.length always hold? Wait... Hasn t this been done already? (Range analysis, etc.)

4 Why? Tim Sweeney, POPL 2006 Static array-bounds checking. Example... a[i]... Will 0 i < a.length always hold? Wait... Hasn t this been done already? (Range analysis, etc.) Yeah, but not for permutation/vertex array code.

5 5 Why? Tim Sweeney, POPL 2006 Static array-bounds checking. Example... a[i]... Will 0 i < a.length always hold? Wait... Hasn t this been done already? (Range analysis, etc.) Yeah, but not for permutation/vertex array code. Not the big idea LFA is not about array-bounds checking.

6 6 The Idea Theorem Proving Flow Analysis

7 The Idea Theorem Proving Flow Analysis

8 8 The Idea Theorem Proving Flow Analysis

9 9 How? Abstract interpretation Mechanical (flow): ς ς ς Propositional (logic): Π Π Π

10 10 How? Abstract interpretation Mechanical (flow): ς ς Propositional (logic): Π Π ς Π

11 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold?

12 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array.

13 13 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array from line 10 or line 30.

14 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array from line 10 or line 30. i is an integer.

15 15 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array from line 10 or line 30. i is a non-negative integer.

16 16 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array from line 10 or line 30. i is a non-negative integer. a.length is a positive integer.

17 Higher-order flow analysis fails Example... a[i]... Will 0 i < a.length always hold? Flow analysis results a is an array from line 10 or line 30. i is a non-negative integer. a.length is a positive integer. Insufficiently rich information Frequently can t show i < a.length.

18 18 First attempt: Enrich flow values with relations Example... a[i]... Will 0 i < a.length always hold?

19 19 First attempt: Enrich flow values with relations Example... a[i]... Will 0 i < a.length always hold? Hypothetical results a is an array from line 10 or line 30. i is a non-negative integer. a.length is a positive integer.

20 20 First attempt: Enrich flow values with relations Example... a[i]... Will 0 i < a.length always hold? Hypothetical results a is an array from line 10 or line 30. i is in {x : 0 x < a.length}. a.length is a positive integer.

21 First attempt: Enrich flow values with relations Example... a[i]... Will 0 i < a.length always hold? Hypothetical results a is an array from line 10 or line 30. i is in {x : 0 x < a.length}. a.length is a positive integer. Problems 1. What does a.length mean where a is out of scope? 2. How did this set get there in the first place?

22 First attempt: Enrich flow values with relations Example... a[i]... Will 0 i < a.length always hold? Hypothetical results a is an array from line 10 or line 30. i is in {x : 0 x < a.length}. a.length is a positive integer. Problems 1. What does a.length mean where a is out of scope? 2. How did this set get there in the first place?

23 23 Problem 1: Meaning of variables Ambiguity problem Need environment-independent identities for values.

24 Problem 1: Meaning of variables Ambiguity problem Need environment-independent identities for values. Solutions Constants. E.g. 5 means 5 anywhere.

25 25 Problem 1: Meaning of variables Ambiguity problem Need environment-independent identities for values. Solutions Constants. E.g. 5 means 5 anywhere. Heap locations. E.g. Heap location 10 offset 3.

26 26 Problem 1: Meaning of variables Ambiguity problem Need environment-independent identities for values. Solutions Constants. E.g. 5 means 5 anywhere. Heap locations. E.g. Heap location 10 offset 3. Bindings. [Shivers 1988]

27 Bindings Definition A binding is a variable-time pairing, e.g. (x, 3).

28 28 Bindings Definition A binding is a variable-time pairing, e.g. (x, 3). Example (Variable v. binding) Value of variable x depends on environment. Value of x bound at time 3: Same in any environment/state.

29 29 Bindings Definition A binding is a variable-time pairing, e.g. (x, 3). Example (Variable v. binding) Value of variable x depends on environment. Value of x bound at time 3: Same in any environment/state. Example (Abstract bindings) 0CFA: All values of x bound at any time. 1CFA: All values of x bound at a time while calling foo.

30 30 Strategy Build binding-sensitive flow analysis.

31 31 Strategy Build binding-sensitive flow analysis. Build binding-sensitive logic.

32 32 Strategy Build binding-sensitive flow analysis. Build binding-sensitive logic. Build binding-sensitive propositional abstract interpretation.

33 33 Strategy Build binding-sensitive flow analysis. Build binding-sensitive logic. Build binding-sensitive propositional abstract interpretation. Weave.

34 34 Tool 1: Continuation-passing style (CPS) Contract Calls never return. Continuations are passed to receive the result. Example Direct-style identity function: (define (id x) x) Example CPS identity function: (define (id x return) (return x))

35 35 CPS simplifies In CPS, fun call, fun return, conditional branch, sequencing, iteration, exception throw, coroutine switch, continuation invocation...

36 36 CPS simplifies In CPS, fun call, fun return, conditional branch, sequencing, iteration, exception throw, coroutine switch, continuation invocation......all become call to λ

37 37 CPS simplifies In CPS, fun call, fun return, conditional branch, sequencing, iteration, exception throw, coroutine switch, continuation invocation... Machine state without CPS...all become call to λ ς State = CALL Env Store Stack Time

38 38 CPS simplifies In CPS, fun call, fun return, conditional branch, sequencing, iteration, exception throw, coroutine switch, continuation invocation... Machine state without CPS...all become call to λ ς State = CALL Env Store Stack Time

39 39 CPS simplifies In CPS, fun call, fun return, conditional branch, sequencing, iteration, exception throw, coroutine switch, continuation invocation... Machine state with CPS...all become call to λ ς State = CALL Env Store Time

40 40 Tool 2: Machine-based abstract interpretation Idea Abstract machine states component-wise. Machine state A An A A call site. environment for variable lookup. heap. time. More formally ς State = CALL Env Store Time

41 Tool 2: Machine-based abstract interpretation Idea Abstract machine states component-wise. Abstract machine state An abstract call site. An abstract environment for variable lookup. An abstract heap. An abstract time. More formally ς Ŝtate = ĈALL Ênv Ŝtore Time

42 Tool 2: Machine-based abstract interpretation Idea Abstract machine states component-wise. Abstract machine state An abstract call site. An abstract environment for variable lookup. An abstract heap. An abstract time. More formally ς Ŝtate = ĈALL Ênv Ŝtore Time

43 43 Binding-factored environment Definition An environment maps variables to values.

44 Binding-factored environment Definition An environment maps variables to values. Definition A binding-factored environment (β, ve) [Shivers 1988] maps: variables to their binding times. (β) and then, variables plus times (bindings) to values. (ve)

45 45 Tool 3: Restricted first-order logic for states Features Propositions are facts about concrete machine states. Ground terms are identities (bindings, locations, constants).

46 46 Tool 3: Restricted first-order logic for states Features Propositions are facts about concrete machine states. Ground terms are identities (bindings, locations, constants). Restrictions No existential quantifiers. Only outer-level universal quantifiers. Quantifiers range over abstract identities.

47 Example: Proposition Example Every value of x bound while calling foo is less than the length of every array bound to a.

48 48 Example: Proposition Example Every value of x bound while calling foo is less than the length of every array bound to a. In longhand: (forall x : (x, t foo ) (forall a : (a, ) (< x (alen a))))

49 49 Example: Proposition Example Every value of x bound while calling foo is less than the length of every array bound to a. In longhand: (forall x : (x, t foo ) (forall a : (a, ) (< x (alen a)))) Or, in convenient (but incomplete) shorthand: (< (x, t foo ) (alen (a, )))

50 50 Logic syntax Features S-Expressions. Just or, not. Relations encoded as functions.

51 51 Logic semantics Question How do we know when proposition φ is true for state ς?

52 52 Logic semantics Question How do we know when proposition φ is true for state ς? Answer When ς = φ holds.

53 53 Logic semantics Question How do we know when proposition φ is true for state ς? Answer When ς = φ holds. Means exactly what you think it means.

54 Filtered concretization Set of conrete states (State) 54

55 Filtered concretization ς Set of conrete states (State) {ς : ς ς } 55

56 Filtered concretization Π Set of conrete states (State) {ς : ς = Π} 56

57 Filtered concretization ς ς/π Π Set of conrete states (State) {ς : ς ς and ς = Π} 57

58 58 Deriving new propositions Example If ς = (= x y) and ς = (= y z), does ς = (= x z) hold?

59 59 Deriving new propositions Example If ς = (= x y) and ς = (= y z), does ς = (= x z) hold? Answer Yes, if {(= x y),(= y z)} (= x z) holds.

60 60 Deriving new propositions Example If ς = (= x y) and ς = (= y z), does ς = (= x z) hold? Answer Yes, if {(= x y),(= y z)} (= x z) holds. (Assm) ψ Π Π ψ ( Ant) Π {φ 1 } φ 3 Π {φ 2 } φ 3 Π {(or φ 1 φ 2 )} φ 3 (Subst) Π (= ι ι ) Π ψ[ι/x] Π ψ[ι /x] (Ant) Π φ Π Π Π φ (Cases) Π {φ 1 } φ 2 Π {(not φ 1 )} φ 2 Π φ 2 (Contr) Π {(not φ 1 )} φ 2 Π {(not φ 1 )} (not φ 2 ) Π φ 1 (Eq) Π (= ι ι) ( Cons) ( Intro) Π φ 1 Π (or φ 1 φ 2 ),(or φ 2 φ 1 ) Π ψ x free(ψ) Π (forall x : ι ψ) ( Swap) (Int) Π (forall x : ι φ) {φ} φ Π (forall x : ι (and φ φ )) Π (forall x 1, x 2 : ι 1, ι 2 ψ) Π (forall x 2, x 1 : ι 2, ι 1 ψ)

61 61 Trusting the theorem prover Summary = : What a proposition means. : What a proposition implies.

62 62 Trusting the theorem prover Summary = : What a proposition means. : What a proposition implies. Question How can we trust an external theorem prover?

63 63 Trusting the theorem prover Summary = : What a proposition means. : What a proposition implies. Question How can we trust an external theorem prover? Theorem (Syntactic soundness) If Π φ holds, then Π = φ holds.

64 All together now 64

65 65 Woven state Example (Machine, ς) call site (f x k) local env f t foo k t foo x t foo global env (f, t foo ) (k, t foo ) (x, t foo ) positive (z, t bar ) positive Example (Assumptions, Π) (forall x : (x, t foo ) (forall z : (z, t bar ) (< x z))) time t f

66 66 Woven state Example (Machine, ς) call site (f x k) local env f t foo k t foo x t foo global env (f, t foo ) (k, t foo ) (x, t foo ) positive (z, t bar ) positive Example (Assumptions, Π) (forall x : (x, t foo ) (forall z : (z, t bar ) (< x z))) time t f

67 67 Woven transition relation Old machine state New machine state ( ς,π) => ( ς,π ) Old assumption base New assumption base

68 68 Example: Transition Example call site (f x k) time t f

69 69 Example: Transition Example call site (f x k) local env f t foo time t f

70 70 Example: Transition Example call site (f x k) local env f t foo global env (f, t foo ) a closure over (λ (a q)...) time t f

71 Example: Transition Example call site (f x k) local env f t foo x t foo global env (f, t foo ) a closure over (λ (a q)...) time t f

72 Example: Transition Example call site (f x k) local env f t foo x t foo global env (f, t foo ) a closure over (λ (a q)...) time t f New fact? (forall x,a : (x, t foo ),(a, t f ) (= x a))

73 It depends. 73

74 Chaining equal values Candidate for Π φ = (forall x,a : (x, t foo ),(a, t f ) (= x a)) Prerequisites Can add it if Π φ. Chicken and egg How can φ be in there already?

75 75 ΓCFA: Abstract counting Idea Keep count of concrete counterparts to abstract identities.

76 76 ΓCFA: Abstract counting Idea Keep count of concrete counterparts to abstract identities. Mechanism Add counter to every abstract machine state. Counter maps each binding to times allocated. Stop counting after 1.

77 ΓCFA: Abstract counting Idea Keep count of concrete counterparts to abstract identities. Mechanism Add counter to every abstract machine state. Counter maps each binding to times allocated. Stop counting after 1. Theorem If {binding 1 } = {binding 2 }, then binding 1 = binding 2.

78 78 Chaining equal values Candidate for Π φ = (forall x,a : (x, t foo ),(a, t f ) (= x a)) Prerequisites Can add it if Π φ. Or, if count of (x, t foo ) is 1 and count of (a, t f ) is 0.

79 79 ΓCFA: Abstract garbage collection Idea Discard unreachable bindings.

80 80 ΓCFA: Abstract garbage collection Idea Discard unreachable bindings. Mechanism Start with bindings touched by current state. Take transitive closure. Can reset unreachable bindings counts to 0.

81 81 Chaining equal values Candidate for Π φ = (forall x,a : (x, t foo ),(a, t f ) (= x a)) Prerequisites Can add it if Π φ. Or, if count of (x, t foo ) is 1 and count of (a, t f ) is 0. Or, if count of (x, t foo ) is 1 and (a, t f ) is unreachable. (More in paper.)

82 82 Example: Invertible rebinding Example call site (f (+ x 1) k) local env f t foo x t f global env (f, t foo ) a closure over (λ (x q)...) time t f Updating assumption base Can replace (x, t f ) with (- (x, t f ) 1) in Π?

83 Example: Invertible rebinding Example call site (f (+ x 1) k) local env f t foo x t f global env (f, t foo ) a closure over (λ (x q)...) time t f Updating assumption base Can replace (x, t f ) with (- (x, t f ) 1) in Π? Yes, if (x, t f ) is unreachable and its count is 1. (E.g. tail recursion, for loops.) (More on this in the paper.) 83

84 84 Example: Conditional Example call site (if (< i (alen a))......)

85 85 Example: Conditional Example call site (if (< i (alen a))......) Case 1 Π can (dis)prove (< i (alen a)). Branch one way. ς true ς ς false

86 86 Example: Conditional Example call site (if (< i (alen a))......) Case 2 (< i (alen a)) has one counterpart. Branch both ways & assert. ς true (< i (alen a)) ς (not (< i (alen a))) ς false

87 87 Example: Conditional Example call site (if (< i (alen a))......) Case 3 None of the above. Branch both ways. Don t touch Π. ς ς true ς false

88 88 Walkthrough: Simple for loop Example for (i = 0; i < a.length; i++) print(a[i]) ; Example (CPS) (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Parameters 0CFA contour set. (Bindings = Variables.)

89 89 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a))

90 90 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a))

91 91 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 i)

92 92 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 i)

93 93 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 i) Safe 0 i < (alen a) holds!

94 94 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 i)

95 95 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 i)

96 96 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), (= 0 (- i 1))

97 97 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i)

98 98 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i), (< i (alen a)) Safe 0 i < (alen a) holds!

99 99 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i), (< i (alen a))

100 100 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i), (< i (alen a))

101 101 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 (- i 1)), (< (- i 1) (alen a))

102 102 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i), (< (- i 1) (alen a)), (< i (alen a)) Safe 0 i < (alen a) holds!

103 103 Walkthrough: Simple for loop (letrec ((loop (λ (i) (if (< i (alen a)) (print (aget a i) (λ () (loop (+ i 1))))...)))) (loop 0)) Assumption base, Π (< 0 (alen a)), ( 0 i), (< i (alen a)) Finished State already visited.

104 104 More in the paper Formal treatment. Flow analysis as oracle inference rules. More rules for assumption base management. Rules for handling arrays. Three-page worked example for vertex arrays.

105 105 Related & inspiring work Cousot & Cousot. (Abstract interpretation)

106 106 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis)

107 107 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing)

108 108 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains)

109 109 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains) Shivers. (Control-flow analysis w/ factored environment)

110 110 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains) Shivers. (Control-flow analysis w/ factored environment) Hudak. (Abstract reference counting)

111 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains) Shivers. (Control-flow analysis w/ factored environment) Hudak. (Abstract reference counting) Nanevski & Morrisett. (Soundness of theorem prover interaction)

112 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains) Shivers. (Control-flow analysis w/ factored environment) Hudak. (Abstract reference counting) Nanevski & Morrisett. (Soundness of theorem prover interaction) Ball. (Predicate abstraction)

113 113 Related & inspiring work Cousot & Cousot. (Abstract interpretation) Cousot & Cousot. (Invariant synthesis) Cousot & Cousot. (Widening & narrowing) Cousot & Cousot. (Relational abstract domains) Shivers. (Control-flow analysis w/ factored environment) Hudak. (Abstract reference counting) Nanevski & Morrisett. (Soundness of theorem prover interaction) Ball. (Predicate abstraction)...

114 Related & future work CFA LFA ΓCFA

115 115 Related & future work CFA LFA ΓCFA

116 116 Related & future work CFA LFA ΓCFA ΘCFA

117 Conclusion HOFA + FOL = LFA

118 118 Conclusion HOFA + FOL = LFA Merci

119 119 Question What are some other applications of LFA? Answer Improving flow precision. Static checks of assert statements. Pre- and post-condition checks.

120 120 Question Do you have an implementation? Answer Half of one: Modified ACL/2 for theorem prover. Modified ΓCFA. Goal: Meet in the middle.

121 Question Does a backward version exist? Answer Not yet. Start by widening into constraint-solving form.

122 Question What is the complexity of LFA? Answer Exponential in theory. Usually much friendlier in pratice. It depends on......the contour set....the degree of widening to assumption base & abstract heap.

123 123 Question Do you support floats? Answer Patrick, not yet.