An Introduction to Heap Analysis. Pietro Ferrara. Chair of Programming Methodology ETH Zurich, Switzerland

Transcription

1 An Introduction to Heap Analysis Pietro Ferrara Chair of Programming Methodology ETH Zurich, Switzerland Analisi e Verifica di Programmi Universita Ca Foscari, Venice, Italy

2 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

3 Syntax

4 Concrete and Abstract Domain Concrete domain: Abstract non-relational domain:

5 Non-Relational Domains A non-relational domain has to provide: > Operators on lattices Partial ordering Upper and lower bound Top and bottom elements > Abstraction and concretization functions > > >

6 Summary on Non-Relational Domains Many different domains: > Sign > Parity > Congruences > Integers > Intervals Work directly on values Evaluation of expressions and conditions No relational information!

7 Summary on Relational Domains Generic state > We do not have anymore an environment! A relational domain has to provide: > Operators on lattices Partial ordering Upper and lower bound Top and bottom elements > Abstraction and concretization functions > >

8 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

9 Extended language We extend the language to support references > Dynamic allocation of memory > Dereferencing pointers We can have > Integer variables > Pointers to integers variables > Pointers to pointers to We do not consider arithmetic of pointers

10 Syntax

11 Syntax

12 Syntax

13 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

14 Previously Concrete domain > environment that relats variables to values Runtime behaviors described by: > Set of environments > Lattice with set operators:

15 Naïve extension Trivial extension of this domain > Variables can be Integers values Pointers Let s define pointers as variables > [x -> y] means that x points to y Formally

16 Counterexample This approach is not expressive enough! x = allocint; The pointer created by allocint was not previously assigned to a variable > Imagine it as the creation of a new object We do not have a way to represent it We need something more!

17 Environment and Store Common approach in programming languages: > Environment Local variables related to addresses > Heap Addresses related to values

18 More Expressive Languages For our language such domain is enough But what about objects? > State of an object Environment! It relates variables (fields!) to values (integers or pointers to other objects)

19 class A { int node= ; A next=null; } An example Variable Value head #1 Address Value #1 head #2 next null #2 5 A head=new A(5); head.next=new A(4); Variable head #1 Value Address Value #1 head #2 next #3 #2 5 #3 head #4 next null #4 4

20 Our approach It is enough for our simple language For many statements, same semantics > Concatenation, if, while > Arithmetic operations > Evaluation of conditions

21 Statements We have to (re)define the semantics of

22 Old expressions

23 New expressions

24 Statements

25 An example int* it=allocint; *it=1; int sum=0; Variable Value it #1 Variable Value it #1 Variable Value it #1 sum #2 Address Value #1? Address Value #1 1 Address Value #1 1 #2 0 while(sum<2) { } sum=sum+*it; Variable Value it #1 sum #2 Address #1 1 #2 12 Value

26 Summary We distinguish between actions > on pointers > on numerical values Abstract domains on > Numerical information Sign Intervals > Heap structure Todo now!

27 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

28 Approach The core is how to abstract the heap: > Variables points to something on it Previously we had > In fact, we did not have pointers and store Let us be generic w.r.t. what an identifiers of the heap can be

29 Heap We suppose that an abstraction of the heap is provided, that is, we have For numerical domains, we had > Similar approach! Now we investigate what semantics primitives we need > Like eval_const, eval_arithm, etc

30 Statements

31 Old expressions Values are stored in the heap We do not know what is the abstract heap We need a primitive that defines it!

32 New expressions

33 Statements

34 Summary The heap analysis has to provide: > returns a new abstract heap id > returns the numerical value related to a heap identifier > assigns the evaluation of a numerical expression to an heap identifier Mmm something is wrong here! > We have not to consider numerical values!!!

35 Numerical Relational Domains A relational domain has to provide: > Operators on lattices > Abstraction and concretization functions > >

36 Numerical Values

37 Approach In expressions: > replace variables with (abstract) pointers Relational numerical domain: > track relations on abstract pointers instead of variables! Combination of > Heap analysis symbolically represents references > Numerical domains track numerical information

38 Arithmetic Expressions We replace variables with heap ids > through the environment We pass such expression to We do not need and We simply deals with and

39 Statements

40 New expressions

41 Statements corresponds to replaced with where we

42 Summary The heap analysis provides: > Heap identifiers Lattice structure! > An internal state > Lattice structure A numerical relational domain provides > Lattice structure > and

43 Example int x=0, y=1, z=2; x=&y; z=&y; while(x>0) y--; return z; Environment x y z Heap analysis Numerical domain We return [0..0]

44 Too easy! The situation is more complex: > Rough evaluation of boolean conditions A variable may point to different references? int x=0, y=1; if(random>0.5) x=&y Environment x y Heap analysis Numerical domain

45 Heap identifiers An heap id is no more a single element > Set of elements! Issue: > If we assign a variable pointing to many addresses, what happens? Weak updates: lub between the previous value and the assigned one We preserve the soundness! We introduce approximation!

46 In the concrete 0.6! int x=0, y=1; if(random>0.5) x=&y x++; true Environment x y Heap analysis Numerical domain

47 In the concrete 0.1! int x=0, y=1; if(random>0.5) x=&y x++; false Environment x y Heap analysis Numerical domain

48 In the abstract? int x=0, y=1; if(random>0.5) x=&y x++; Environment x y Heap analysis Numerical domain

49 Final state Concrete x y 1 2 x y Abstract x y

50 Heap analysis Nodes > the state of the heap represented by a graph Environment > Part of the heap analysis More complex data structures like lists > edges between nodes while(head!=null) head=head.next head next 1 2 next 3 next 4

51 Simple language Nodes > No edges between nodes > Only edges from variables to nodes Heap identifiers > Set of nodes > Lattice operators: set operators! Internal states > Set of nodes (references) > Lattice operators: set operators!

52 Are you sure??? The end As always, things are much more complex Let s start with a naïve solution: > Nodes identified by an integer number > Counter incremented each time we allocated a new integer As in the concrete context it should not work

53 Allocation inside while loop x int x= ; while(random>0.5) x=allocint; x Unbounded number of nodes > Lattice using set operators of infinite height > The analysis may not converge!

54 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

55 Top domain Let s start with a simple domain > The top domain > approximates all the concrete references! > Lattice structure is trivial: Upper bound: Lower bound: Top: Bottom: Partial order: true > Semantics:

56 Abstraction and concretization

57 Abstraction and concretization #77 #12 #43 #03

58 Concretization and abstraction #77 #12 #43 #03

59 Allocation of memory #77 allocint #12 #98 #43

60 A (counter)example int x=0, y=1; if(random>0.5) x=&y x++; Environment x y Heap analysis Numerical domain r r At the end, x and y may be both 1! > Not sound Why? > r represents many concrete references! > We have to perform weak updates!

61 Weak vs. strong updates Concrete semantics of assignments: (Unsound) Abstract semantics: Note: unsound iff x is an heap identifier!

62 (wrong) Assignments with heap id

63 Summary nodes What happens if? > > > Environment: > Initial state: > Result

64 Concrete and Abstract x=0 Not sound!!! x=0

65 Weak vs. Strong updates Weak updates: > assign lub(old value, assigned value) Strong updates: > Assign exactly the assigned value We can perform strong updates iff > The assigned variable points to one heap id > The id represents one concrete reference Otherwise we are unsound!

66 int x=0, y=1; if(random>0.5) x=&y x++; An example Environment x y Heap analysis Numerical domain At the end, x and y may be > both 1 > both 2 > Sound! > but rough (they are never 0!) r r

67 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Top domain 2. Program point-bounded references 3. Shape analysis

68 Program point-bounded reference Intuition: > Approximate together all the concrete references allocated by the same statement > Abstract reference: the program point of the statement Inside a loop or a recursive procedure: > One abstract -> many concrete > Important to track it for weak updates

69 Approximation: Heap identifiers > Different executions may assign references allocated by different statements To the same variable Heap identifiers > sets of abstract references Lattice operators: > Common set operators

70 Lattice structure. {(l2,c4),(l1,c3),(l3,c2)}... {(l2,c4),(l1,c3)} {(l1,c3),(l3,c2)}. {(l2,c4)} {(l1,c3)} {(l3,c2)}.

71 Abstraction and concretization

72 Abstraction and concretization

73 Abstraction and concretization #77 #12 #43 #03 1: void main(string[] args) { 2: int a=allocint; 3: }

74 Abstraction and concretization #77 1: void main(string[] args) { 2: int a; 3: if(random>0.5) 4: a=allocint; 5: else a=allocint; 6: } #12 #43 #03

75 Abstraction and concretization #77 #12 #43 #03 1: void main(string[] args) { 2: int a; 3: while(random>0.5) 4: a=allocint; 5: }

76 Allocation of memory #77 #12 #43 #03 1: void main(string[] args) { 2: int a=allocint; 3: }

77 Allocation of memory #77 #12 #43 #03 1: void main(string[] args) { 2: int a; 3: while(random>0.5) 4: a=allocint; 5: }

78 An example 1: int x=0, 2: y=1; 3: if(random>0.5) 4: x=&y; 5: x++; Environment x y Environment x y Heap analysis Numerical domain l1 l2 l1 l2 Heap analysis r Numerical domain r

79 Another example 1: while(random>0.5) { 2: x=allocint 3: x=0 4: } 5: x++; Environment Heap analysis x l2 Sound! Numerical domain but rough (x cannot be 0 at the end!) l2

80 Recall: Weak vs. Strong updates > We can perform strong updates iff The assigned variable points to one heap id The heap id represents one concrete reference We require that: > heap id composed by one abstract element > The reference is not allocated inside a loop If we do not have method calls! > Otherwise, different calls to the same method > Track if a method is recursive

81 Refining the analysis Create more abstract references > that represent one concrete reference > Increase the precision > Increase the complexity For instance, > Number of iteration of the while loop (pp, i): i-th iteration of the loop Require a widening operator, infinite references! > Stack of the called method

82 Limits Serious limits with data structures > Complex relations between data > More complex properties Acyclic list Trees Etc.. > Abstract reference bounded to program point Not enough > Need to track relations between heap nodes Something like relational numerical domains

83 An example 1: List list=new List(); 2: List it=list; 3: while(random>0.5) { 4: it.next=new List(); 5: it=it.next; 6: } class List { int node=0; List next=null; } Environment list it Heap analysis l2 next l4

84 An example 1: List list=new List(); 2: List it=list; 3: while(random>0.5) { 4: it.next=new List(); 5: it=it.next; 6: } Environment list it Heap analysis l2 next l4 next class List { int node=0; List next=null; } list may be cyclic it may not point to the last cell

85 An example - Concretization Environment list it Heap analysis l2 next l4 next list it list it list next it #1 next #2 #1 next #2 next #3 #1 next #2 next #3

86 Outline 1. Recall of numerical domains 2. Extended language with references 3. Concrete semantics 4. Abstract semantics 5. Abstract domains 1. Program point-bounded references 2. Shape analysis

87 Shape analysis Main trend in the field of static analysis Developed mostly by Mooly Sagiv & al. > Started in 1995 > Still ongoing research efforts Many case studies and applications Strong practical interest Huge literature on the topic Mooly Sagiv, Thomas W. Reps, Reinhard Wilhelm: Solving Shape- Analysis Problems in Languages with Destructive Updating ACM Trans. Program. Lang. Syst. 20(1): 1-50 (1998)

88 Intuition Describe the shape of the concrete heaps > Concrete heaps: potentially unbounded > Abstract heaps: bounded a-priori One abstract node many concrete references As in the previous approach! More flexible > Materialization > Explicit representation of sharing Focused on lists > But still approximated!

89 Shape graphs Concrete shape graph #1 #2 #3 #4 > Represented by cons-cells > Potentially unbounded n{x} nφ Abstract shape graphs > nφ : summary nodes

90 Approach n{x} nφ list Until here, it is not more refined nor really different > Now: Nodes bounded to variables > Before: Nodes bounded to program points > No difference between cyclic and acyclic lists > Still using summary nodes l2 next l4 next

91 First refinement: Sharing > Track an is-shared predicate on each node > means that all the cells represented by a node n are distinct That is, there is not sharing > The length of the list is abstracted away but in this way the analysis terminates! So we know something more > But how is this used?

92 Materialization without sharing Access the next element > Summary node > Materialize a new single node ad hoc Unsound! Precisely track nodes pointed by variables > Always single nodes! > More precise than the previous approach Sharing information is essential x=x.next n{x,y} nφ n{y} n{x} nφ

93 Materialization with sharing Unshared summary node > x=x.next n{x,y} nφ n{y} n{x} nφ Shared summary node > x=x.next n{x,y} nφ n{y} n{x} nφ

94 Problem > Identifiers of nodes Renaming variables that point to them > What happens if we assign an existing node to an existing variable? Node x=y; n{y} n{x,y} Renaming of the node

95 Another problem: Liquidization > If the assigned variable point to something? n{x} x=y; n{?} n{y} n{x,y} n{y} n{x} nφ x=y; n{x,y} n{?} nφ Garbage collection or liquidization

96 An example 1: List list=new List(); 2: List it=list; 3: while(random>0.5) { 4: it.next=new List(); 5: it=it.next; 6: } class List { int node=0; List next=null; } list it n{list} n{list,it} n{it} n{?}

97 An example 1: List list=new List(); 2: List it=list; 3: while(random>0.5) { 4: it.next=new List(); 5: it=it.next; 6: } class List { int node=0; List next=null; } list n{list} n{it} n{?} nφ n{it} n{?} it

98 An example - Concretization list it list it n{list} nφ n{it} l2 next l4 next list it list it list next it #1 next #2 #1 next #2 next #3 #1 next #2 next #3

99 Conclusion on shape analysis Shape analysis improves the precision > But it is not the final solution > Shape analysis appeared 13 years ago > Many improvements in the meanwhile > New improvements appear each year Not easy to infer information > Automatically > Efficiently > Precisely

100 We have Conclusion > extended the language with references > formalized concrete and abstract semantics > presented several heap analyses Reasoning on heap analysis > without taking into account numerical issues Quite similar to numerical domains > Different levels of precision and efficiency > Non-relational and relational information

101 Conclusion We can reason separately on > Numerical domains > Heap analyses Different plugins of the same analysis > Different levels of complexity efficiency Generic analyzers plugged with different > numerical domains > heap analyses > properties > etc