TVLA: A Framework for Kleene Logic Based Static Analyses

Transcription

1 TVLA: A Framework for Kleee Logic Based Static Aalyses Tal Lev-Ami Departmet of Computer Sciece, Tel-Aviv Uiversity, Israel May 28, 2000 Ackowledgmets First ad foremost I would like to thak Dr. Mooly Sagiv for his guidace, support ad drive. Without it, this thesis would ever have bee writte. I would like to thak Nurit Dor, Mauel Fahdrich, Noam Rietskey, Tom Reps, ad Reihard Wilhelm for readig the drafts ad for their helpful commets. I ejoyed havig valuable discussios with Hae Riis ad Flemmig Nielso. Thaks also to Guy Lade, Ra Shaham ad Oded Shmueli. I would like to thak my parets Liora ad Uzi for listeig ad givig a kid word where is was eeded. May thaks to the Acadamy of Sciece for their Fiacial support. Abstract We preset TVLA (Three-Valued-Logic Aalyzer). TVLA is a YACC -like framework for automatically costructig static-aalysis algorithms from a operatioal sematics, where the operatioal sematics is specified usig logical formulae. TVLA has bee implemeted i Java ad was successfully used to perform shape aalysis o programs maipulatig liked data structures (sigly ad doubly liked lists), to prove safety properties of Mobile Ambiets, ad to verify the partial correctess of several sortig programs. tla@math.tau.ac.il 1

2 Cotets 1 Itroductio Mai Results Applicatios Techical Cotributios Outlie of the Thesis A Primer o 3-Valued-Logic-Based Aalysis Represetig Memory States via Logical Structures Coservative Represetatio of Sets of Memory States via 3-valued Structures Embeddig Summary odes Formulae Subclasses of formulae Sematics System Descriptio TVP Declaratios Actios CFG Process Focus Precoditios Update Formulae Coerce Blur Output Additioal Features Applicatios Sigly Liked Lists Doubly Liked Lists Sortig Algorithms Specifyig a Aalysis for Observig ADT Properties Specifyig ad Checkig Partial Correctess of ADT Operatios Mobile Ambiets

3 5 The Coerce Algorithm Costraits Automatic Geeratio of Costraits Order of Costraits Memoizig Trasitive Closures Icremetal Evaluatio Icremetal Formula Evaluatio The Focus Algorithm Normalizig Focus Formulae Focusig o Cojuctios of Literals Focusig o Literals Usig Fuctioal Properties i Focus The Actual Implemetatio Active Nodes Geeralized Embeddig Formula Evaluatio Coerce Focus Actios New Retai Sigle Structure Usig Nullary Predicates to Improve Precisio Coclusio The Essece of Istrumetatio Compariso to Related Work Further Work A Empirical 80 B Proof of the Geeralized Embeddig Theorem 80 C User s Maual 85 C.1 Graphical Represetatio C.2 TVP C.2.1 Predicates C.3 Formulae C.3.1 Cosistecy Rules

4 C.3.2 Actios C.4 Cotrol Flow Graph C.5 Usability C.5.1 Commets C.5.2 Preprocessig C.5.3 Sets C.5.4 Foreach C.5.5 Composite Operatios C.6 TVS C.7 Commad Lie Optios Itroductio The abstract-iterpretatio techique [CC79] for static aalysis allows oe to summarize the behavior of a statemet o a ifiite set of possible memory states. This is sometimes called a abstract sematics for the statemet. With this methodology it is ecessary to show that the abstract sematics is coservative, i.e., it summarizes the (cocrete) operatioal sematics of the statemet for every possible memory state. Ituitively speakig, the operatioal sematics of a statemet is a formal defiitio of a iterpreter for this statemet. This operatioal sematics is usually quite atural. However, desigig ad implemetig soud ad reasoably precise abstract sematics is quite cumbersome (the best iduced abstract sematics defied i [CC79] is usually ot computable). This is particularly true i problems like shape aalysis ad poiter aalysis (e.g., see [Deu94, SRW00, SRW98]), where the operatioal sematics ivolves destructive memory updates. I this paper, we preset TVLA (Three-Valued-Logic Aalyzer), a system for automatically geeratig a static-aalysis algorithm from the operatioal sematics of a give program. The operatioal sematics is writte i a special form, based o first-order predicate logic with trasitive closure. A additioal iput to TVLA is a abstract represetatio of all the possible memory states at the begiig of the aalyzed program. TVLA automatically geerates the abstract sematics, ad, for each program poit, produces a coservative abstract represetatio of the memory states at that poit. 1.1 Mai Results TVLA is iteded as a proof of cocept for itra-procedural shape aalysis, ad other static-aalysis algorithms. It is a test-bed i which it is 4

5 quite easy to try out ew ideas. The theory behid TVLA is based o [SRW99, SRW00] (see Sectio 8.2). The system is publicly available from tla. TVLA was implemeted i Java ad has bee successfully used to perform shape aalysis o programs maipulatig liked data structures (sigly ad doubly liked lists), to prove safety properties of Mobile Ambiets, ad to verify partial correctess of several programs. We also report o some programs that are too complex for the curret system. The system was tested o a Petium II 400 MHz ruig Liux with JDK 1.2. All the timig iformatio about the system refers to this computer Applicatios TVLA has bee utilized to aalyze a variety of small but itricate programs from the groups described below. Sigly Liked Lists: We performed shape aalysis o the set of programs maipulatig sigly liked lists used i [DRS00], icludig oes for searchig, elemet isertio, ad elemet deletio. These programs perform destructive updatig. Some of these programs are (deliberately) sematically icorrect, ad we are able to locate the bugs i them. The aalysis times are reported i AppedixA. Doubly Liked Lists: Doubly liked lists are more challegig tha sigly liked lists because they create shared memory cells ad cycles. We have aalyzed a program that iserts a ew elemet ito a arbitrary place i a doubly liked list, ad the aalysis was able to coclude that the isertio results i a doubly liked list. Sortig Programs: A differet kid of applicatio of TVLA is for program verificatio. We applied TVLA to several implemetatios of sortig algorithms, ad proved that, give a possibly usorted liked list as iput, we always ed up with a sorted list. This is prove without the eed for programmer-specified loop ivariats. Istead, the operatioal sematics also keeps track of iequalities betwee the list elemets. We are ecouraged by the fact that we have successfully verified both isert sort ad bubble sort o sigly liked lists. Mobile Ambiets: We implemeted the aalysis of [NNS00] ad foud out that it is imprecise ad quite slow. This motivated us to geeralize the techiques preseted i [SRW99, SRW00] i order to guaratee that oly a costat umber of structures arise at each program poit (see Sectio 3.4). 1 Our experiece idicates that usig JVM o Widows, the system rus about 20% faster. 5

6 With this extesio, TVLA was able to successfully aalyze a slight variat of the origial specificatio used i [NNS00]. This took 336 CPU secods ad the aalysis proved the ecessary properties (uiqueess of ambiet istace ad mutual exclusio) precisely. 1.2 Techical Cotributios The TVLA system itroduces several ew cotributios, which are described i this thesis. Focus: We preset a otrivial algorithm for focusig o a geeral formula. Thus, ulike [SRW00], our Focus is ot limited to the formulae specific to shape aalysis. This geeralizatio is crucial i order to go beyod shape aalysis. For example, to verify sortig programs we use more complex formulae tha the oes eeded i shape aalysis. The Focus algorithm preseted i this thesis is also more efficiet tha the algorithm from [SRW99] for the formulae that they both hadle. Coerce: The Coerce operatio is very time cosumig. TVLA itroduces a ew algorithm for Coerce (see Sectio 5) which is more efficiet tha the o give i [SRW00, SRW99], for empirical results, see Appedix A. Automatic geeratio of cosistecy rules: Oe of the complicated aspects of usig the three-valued logic approach is the desig of cosistecy rules. This is particularly complicated because two logically equivalet sets of cosistecy rules may result i icomparable aalyses (both of which are coservative). Furthermore, usig cosistecy rules that are ot global ivariats may lead to a icorrect aalysis. TVLA icorporates a algorithm that automatically geerates cosistecy rules from the specificatio, ad thus the user of the system does ot usually eed to add explicit cosistecy rules (see Sectio 5). Costat umber of structures: TVLA allows the use of a eve more compact abstract represetatio i which oly a costat umber of abstract structures arise at each program poit. I some cases (such as the aalysis preformed o Mobile Ambiets), this makes a otherwise ifeasible aalysis possible. 1.3 Outlie of the Thesis The rest of the thesis is orgaized as follows. I Sectio 2, we give a primer of the use of 3-valued logic i static aalysis. Sectio 3 cotais a overview of the TVLA system ad its capabilities. Sectio 4 gives a descriptio of the aalyses doe with the system. We the give a descriptio of the 6

7 /* list.h */ typedef struct ode { struct ode *; it data; *L; (a) /* reverse.c */ #iclude list.h L reverse(l x) { L y, t; y = NULL; while (x!= NULL) { t = y; y = x; x = x->; y-> = t; t = NULL; retur y; (b) Figure 1: (a) Declaratio of a liked-list data type i C. (b) A C fuctio that uses destructive updatig to reverse the list poited to by parameter x. mai algorithms developed for the system: a efficiet Coerce algorithm (Sectio 5) ad a geeral Focus algorithm (Sectio 6). Sectio 7 explais the more advaced topic of active odes. We coclude by summarizig related work ad further research directios (Sectio 8). Appedix A presets the empirical results for test rus of the system. Appedix C is a user s maual for the TVLA system. A program that destructively reverses a sigly liked list is show i Figure 1. The shape aalysis of this program serves as a ruig example i this thesis. 2 A Primer o 3-Valued-Logic-Based Aalysis Kleee s 3-valued logic is a extesio of ordiary 2-valued logic with the special value of 1/2 (ukow) for cases that ca be either 1 or 0. Kleee s iterpretatio of the propositioal operators is give i Table 1. We say that the values 0 ad 1 are defiite values ad that 1/2 is a idefiite value. We say that the values 0 ad 1 are defiite values ad that 1/2 is a idefiite value, ad defie a partial order o truth values to reflect iformatio 7

8 0 1 1/ /2 1/2 0 1/2 1/ / / /2 1/2 1 1/ /2 1/2 Table 1: Kleee s 3-valued iterpretatio of the propositioal operators. cotet: l 1 l 2 deotes that l 1 has more defiite iformatio tha l 2 : Defiitio 2.1 [Iformatio Order]. For l 1,l 2 {0, 1/2, 1, we defie the iformatio order o truth values as follows: l 1 l 2 if l 1 = l 2 or l 2 =1/2. The symbol (joi) deotes the least-upper-boud operatio with respect to, i.e., l 1 l 2 = l 1, if l 1 = l 2 ad 1/2 otherwise. Kleee s sematics of 3-valued logic is mootoic i the iformatio order. 2.1 Represetig Memory States via Logical Structures Our vocabulary icludes a set of predicate symbols partitioed ito two disjoit sets: core ad istrumetatio predicates. Istrumetatio predicates are used to observe derived properties based o core predicates. A 2-valued logical structure S is comprised of a set of idividuals (odes) called a uiverse, deoted by U S, ad a iterpretatio over that uiverse for a set of predicate symbols. The iterpretatio of a predicate symbol p i S is deoted by p S. For every (core ad istrumetatio) predicate p of arity k, p S is a fuctio p S :(U S ) k {0, 1. 2-valued structures are used to represet memory states used i the operatioal sematics of the program. TVLA makes a explicit assumptio that the set of predicate symbols used throughout the aalysis is fixed. (The umber of idividuals i structures ca vary throughout the aalysis.) TVLA oly supports predicates of arity 2; such logical structures ca be thought of as directed graphs. A directed edge labeled by p from u 1 to u 2 deotes that p S (u 1,u 2 ) = 1. Also, we draw p iside a ode u whe p S (u) = 1. Example 2.2 I the ruig example, a 2-valued structure represets a memory state (also called a store); a idividual correspods to a list elemet. The iteded meaig of the core predicates is give i Table 2, ad the iteded meaig of the istrumetatio predicates is give i Table 3 8

9 Predicate Iteded Meaig x(v) Is v poited to by variable x? y(v) Is v poited to by variable y? t(v) Is v poited to by variable t? (v 1,v 2 ) Does the -field of v 1 poit to v 2? Table 2: The core predicates used i the aalysis of the ruig example. Predicate Iteded Meaig Defiig Formula r[, x](v) Is v reachable from program v 1 :(x(v 1 ) (v 1,v)) variable x usig field? r[, y](v) Is v reachable from program v 1 :(y(v 1 ) (v 1,v)) variable y usig field? r[, t](v) Is v reachable from program v 1 :(t(v 1 ) (v 1,v)) variable t usig field? c[](v) Does v reside o a directed + (v, v) cycle via derefereces alog -fields? is[](v) Is v poited to by more v 1,v 2 : (v 1,v) (v 2,v) v 1 v 2 tha oe -field Table 3: The istrumetatio predicates used i the aalysis of the ruig example ad their meaig. Similar istrumetatio predicates are used i all of our shape aalyses for sigly liked lists. The defiig formulae are explaied i Sectio 2.3. (for the momet igore the third colum). The store i Figure 2 is represeted by the 2-valued structure S 3 show i Figure 3. The structure S 3 has four odes, u 0, u 1, u 2, ad u 3 represetig the four list elemets. This represetatio itetioally igores the values of the data field, which are usually immaterial for the aalysis. Poiter variables are represeted by uary predicates (i.e., x S (u) =1 if the variable x poits to the list elemet represeted by u). I Figure 3, the variable x is represeted by the uary predicate x, which is 1 oly for u 0. Notice that TVLA allows the user to specify that a uary predicate is draw as a box with a arrow ito each ode for which it holds. I Figure 3, x is draw as a box ad has a arrow to u 0. Poiter fields withi the list elemets are represeted as biary predicates (i.e., S (u 1, u 2 ) = 1 if the 9

10 x NULL Figure 2: A possible store for the ruig example. x u0 r[,x] u1 r[,x] u2 r[,x] u3 r[,x] x y t r[, x] r[, y] r[, t] is[] c[] u u u u u0 u1 u2 u3 u u u u Figure 3: A logical structure S 3 represetig the store show i Figure 2 i a graphical ad tabular represetatio. -field of u 1 poits to u 2 ). The istrumetatio predicate r[, x] holds for list elemets that are reachable from program variable x, possibly usig a sequece of accesses through the -field. The structure S 3 i Figure 3 has r[, x] S 3 set to 1 for all the odes because they are all reachable from x. A importat aspect of explicitly storig r[, x] is that we ca icremetally compute the appropriate values for the predicates after executio of the program statemet (see [SRW00, Sectio 6.1]). For example, for the statemet y=x, the odes reachable from y after the statemet executes are the same as the odes reachable from x. The istrumetatio predicate is[] holds for odes shared by -fields (a ode is shared by -fields, if it is poited to by more tha oe list elemet usig the field ). I Figure 3, all the elemets of the list are ushared, ad thus is[] S 3 is 0 for all of them. The istrumetatio predicate c[] holds for odes o a cycle of accesses alog -fields. We use the cyclicity istrumetatio to avoid performig a trasitive-closure operatio whe updatig the reachability iformatio. I Figure 3, the list is acyclic, ad thus c[] S 3 is 0 for all of the odes. I fact, throughout the aalysis of the ruig example, is[] S ad c[] S are 0 for all of the odes. 10

11 2.2 Coservative Represetatio of Sets of Memory States via 3-valued Structures Like 2-valued structures, a 3-valued logical structure S is also comprised of a uiverse U S, ad a iterpretatio p S for every predicate symbol p. But, for every predicate p of arity k, p S is a fuctio p S :(U S ) k {0, 1, 1/2, where 1/2 explicitly captures ukow predicate values. 3-valued logical structures are also draw as directed graphs. Defiite values are draw as i the 2-valued structures. Biary idefiite (1/2) predicate values are draw as dotted directed edges. Uary idefiite predicate values are draw iside the odes ad marked as idefiite (this does ot occur i the ruig example) Embeddig Although structures may have differet idividuals, we ca defie a order o structures, deoted by based o the cocept of embeddig. The goal is to guaratee that if S S the the value of every formula i S is less or equal to its value i S. I particular, wheever the formula evaluates to a defiite value i S the the formula has the same value i S. Formally, Defiitio 2.3 Let S ad S be two structures. Let f : U S U S be surjective. We say that f embeds S i S (deoted by S f S ) if (i) for every predicate p (icludig sm) of arity k ad all u 1,..., u k U S, ad (ii) for all u U S p S (u 1,..., u k ) p S (f(u 1 ),..., f(u k )) (1) ( {u f(u) =u > 1) sm S (u ) (2) We say that S ca be embedded i S (deoted by S S ) if there exists a fuctio f such that S f S. A special kid of embeddig is a tight embeddig, i which iformatio loss is miimized whe multiple idividuals of S are mapped to the same idividual i S : Defiitio 2.4 A structure S is a tight embeddig of S if there exists a surjective fuctio blur : U S U S such that, for every p sm of arity k, p S (u 1,..., u k )= p S (u 1,..., u k ) (3) blur(u i )=u i,1 i k 11

12 x u0 r[,x] u r[,x] x y t r[, x] r[, y] r[, t] is[] c[] u u u0 u u0 0 1/2 u 0 1/2 Figure 4: A 3-valued structure S 4 represetig lists of legth 2 or more that are poited to by program variable x (e.g., S 3 ). ad for every u U S, sm S (u ) = ( {u blur(u) =u > 1) blur(u)=u sm S (u) (4) Because blur is surjective, equatios (3) ad (4) uiquely determie S (up to isomorphism); therefore, we say that S = blur(s). Example 2.5 I the ruig example, the 3-valued structure S 4 show i Figure 4 represets the 2-valued structure S 3 for f(u 0 )=u 0 ad f(u 1 )= f(u 2 )=f(u 3 )=u. I fact, the structure show i Figure 4 represets all the lists with two or more elemets. The uary predicate symbol x has x S 4 (u 0 ) = 1, idicatig that the program variable x is kow to poit to the list elemet represeted by u 0, ad x S 4 (u) = 0, idicatig that x is kow ot to poit to ay of the list elemets represeted by u. The biary predicate symbol has S 4 (u 0,u)=1/2, idicatig that the -field of the list elemet represeted by u 0 may poit to a list elemet represeted by u amely the secod list elemet (u 1 i Figure 3) but does ot poit to all the list elemets represeted by u (e.g. u 2 i Figure 3). Also, S 4 (u, u) = 1/2, idicatig that the -field of a list elemet represeted by u may poit to aother list elemet represeted by u or eve to itself but does ot poit to all the list elemets represeted by u (e.g., i Figure 3 the -field of u 2 poits to u 3, but ot to u 1 ). 12

13 2.2.2 Summary odes Nodes i a 3-valued structure that may represet more tha oe idividual from a give 2-valued structure are called summary odes. For example, i the structure show i Figure 3, the odes u 1, u 2, ad u 3 are represeted by the sigle ode u i Figure 4. TVLA uses a special desigated uary predicate sm to maitai summaryode iformatio. Such a summary ode w has sm S (w) = 1/2, idicatig that it may represet more tha oe ode i the embedded 2-valued structures. These odes are graphically draw as dotted ellipsis. I cotrast, if sm S (w) = 0 the w is kow to represet a uique ode. Oly odes with sm S (w) = 1/2 ca have more tha oe ode mapped to them by the embeddig fuctio. The exact choice of which odes should be summarized is crucial for the precisio of the aalysis ad is discussed i Sectio Formulae Properties of structures ca be extracted by evaluatig formulae. We use first-order logic with trasitive closure ad equality, but without fuctio symbols ad costat symbols. For example, the formula v 1 :(x(v 1 ) (v 1,v)) (5) extracts reachability iformatio. Here, deotes the reflexive trasitive closure of the predicate. Therefore, i every structure S, x(v 1 ) evaluates to 1 if v 1 is the ode poited to by x ad (v 1, v) evaluates to 1 i S if there exists a path of zero or more -edges from v 1 to v. The third colum of Table 3 displays the defiig formula of all the istrumetatio predicates used i the ruig example Subclasses of formulae Atomic formulae are oe of the followig (i) p(v 1,..., v k ), (ii) v 1 = v 2, ad (iii) 0 or 1. Without loss of geerality oly ullary, uary, ad biary predicates are supported. A literal is a atomic formula or a egatio of a atomic formula. Defiitio 2.6 A Hor clause is a formula of the form m 1 ( i=1 ϕ i ) ϕ m, 13

14 where m>1, ad ϕ i is a atomic formula, We ow geeralize the defiitio. Toward this ed, for a formula ϕ, we defie ϕ 1 ϕ ad ϕ 0 ϕ. A exteded Hor clause is a formula ϕ of the form (ϕ i ) B i ) (ϕ m ) Bm, m 1 ( i=1 where m>1, ϕ i is a atomic formula, B i {0, Sematics Defiitio 2.7 A assigmet Z is a fuctio that maps free variables to idividuals (i.e., a assigmet has the fuctioality Z : {v 1,v 2,... U S ). A assigmet that is defied o all free variables of a formula ϕ is called complete for ϕ. I the sequel, we assume that every assigmet Z that arises i coectio with the discussio of some formula ϕ is complete for ϕ. The meaig of a formula ϕ, deoted by [[ϕ]] S (Z), yields a truth value i {0, 1, 1/2. The meaig of ϕ is defied iductively as follows: Atomic For a logical literal l {0, 1, 1/2, [[l]] S (Z) =l (where l {0, 1, 1/2). For a atomic formula p(v 1,..., v k ), [[p(v 1,..., v k )]] S (Z) =p S (Z(v 1 ),..., Z(v k )) For a atomic formula (v 1 = v 2 ), 0 Z(v 1 ) Z(v 2 ) [[v 1 = v 2 ]] S (Z) = 1 Z(v 1 )=Z(v 2 ) ad sm S (Z(v 1 )) = 0 1/2 otherwise Logical Coectives For logical formulae ϕ 1 ad ϕ 2 [[ϕ 1 ϕ 2 ]] S (Z) = mi([[ϕ 1 ]] S (Z), [[ϕ 2 ]] S (Z)) [[ϕ 1 ϕ 2 ]] S (Z) = max([[ϕ 1 ]] S (Z), [[ϕ 2 ]] S (Z)) [[ ϕ 1 ]] S (Z) = 1 [[ϕ 1 ]] S (Z) Quatifiers If ϕ is a logical formula, [[ v 1 : ϕ]] S (Z) = mi u U S [[ϕ 1]] S (Z[v 1 u]) [[ v 1 : ϕ]] S (Z) = max u U S [[ϕ 1]] S (Z[v 1 u]) 14

15 Trasitive Closure For (TC v 1,v 2 : ϕ)(v 3,v 4 ), [[(TC v 1,v 2 : ϕ)(v 3,v 4 )]] S (Z) = max 1,u 1,..., u +1 U, Z(v 3 )=u 1,Z(v 4 )=u +1 mi i=1 [[ϕ]]s (Z[v 1 u i,v 2 u i+1 ]) We say that S ad Z potetially satisfy ϕ (deoted by S, Z = ϕ) if [[ϕ]] S (Z) = 1/2 or [[ϕ]] S (Z) = 1. Fially, we write S = ϕ if for every Z: S, Z = ϕ. The Embeddig Theorem: The Embeddig Theorem (see [SRW99, Theorem 3.7]) states that ay formula that evaluates to a defiite value i a 3-valued structure evaluates to the same value i all the 2-valued structures embedded ito that structure. The Embeddig Theorem is the foudatio for the use of 3-valued logic i static-aalysis: it esures that it is sesible to reiterpret o the 3-valued structures the formulae, that whe iterpreted i 2-valued logic, defie the operatioal sematics. TVLA requires each istrumetatio predicate to be associated with a formula over the core predicates defiig its meaig. For example, evaluatig formula (5) o the 3-valued structure show i Figure 4, yields 1 for v u 0, which idicates that the list elemet represeted by u 0 is reachable from variable x, ad 1/2 for v u, which idicates that the list elemets represeted by u may or may ot be reachable from program variable x. Notice that r[, x] S 4 (u) = 1, which is more precise. This is a geeral priciple with istrumetatio predicates (referred to as the istrumetatio priciple i [SRW99]). The stored iformatio ca be more precise tha the result of evaluatig the correspodig formula. 3 System Descriptio The iput to TVLA cosists of two files: (i) a TVS (Three Valued logical Structure) file cotaiig a textual represetatio of the iput structures (see Figure 5), ad (ii) a TVP (Three Valued Program) file, which icludes the operatioal sematics ad the associatio of the operatioal sematics with the edges of the cotrol flow graph (CFG) of the aalyzed program (see Figs. 6 ad 7). To simplify the specificatio, we allow the operatioal sematics to be specific to the aalyzed data type (e.g., sigly liked lists i the ruig example). I the coversio of a C program ito a TVP file, some 15

16 % = {u, u0 %p = sm = {u :1/2 = {u u :1/2,u0 u :1/2 x = {u0 : 1 r[, x] = {u :1,u0 : 1 Figure 5: A TVS structure describig a sigly liked list poited to by x. ormalizig trasformatios are applied (see [CWZ90, SRW98]). For example, the assigmet y->=t is broke ito two statemets: (i) y->=null, followed by (ii) y->=t assumig that y->==null. The full operatioal sematics for programs maipulatig sigly-liked-lists of type L is give i Sectio TVP There are two challegig aspects to writig a good TVP specificatio: oe is the desig of the istrumetatio predicates, which is importat for the precisio of the aalysis; the other is writig the operatioal sematics maipulatig these predicates. A importat observatio is that the TVP specificatio should always be thought of i the terms of the cocrete 2-valued world rather tha the abstract 3-valued world: the Embeddig Theorem guaratees the soudess of the reiterpretatio of the formulae i the abstract world. This is a applicatio of the well-kow credo of Patrick ad Radhia Cousot that the desig of a static aalysis always starts with a cocrete operatioal sematics. The TVP file is divided ito sectios separated by %%, give i the order described below Declaratios The first sectio of the TVP file cotais all the declaratios eeded for the aalysis. Sets: The first declaratio i the TVP file is the set PVar, which specifies the variables used i the program (here x, y, ad t). I the remaider of the specificatio, set otatio allows the user to defie the operatioal sematics for all programs maipulatig a certai data type, i.e., it is parametric i PVar. 16

17 /* Declaratios */ %s PVar {x, y, t // The set of program variables #iclude sll pred.tvp // Core ad Istrumetatio Predicates %% /* A Operatioal Sematics */ #iclude ptr cod.tvp // Operatioal Sematics of Coditios #iclude sll stat.tvp // Operatioal Sematics of Statemets %% /* The program s CFG ad the effect of its edges */ 1 Set Null L(y) 2 // y = NULL; 2 Is Null Var(x) exit // x == NULL 2 Is Not Null Var(x) 3 // x!= NULL 3 Copy Var L(t, y) 4 // t = y; 4 Copy Var L(y, x) 5 // y = x; 5 Get Next L(x, x) 6 // x = x->; 6 Set Next Null L(y) 7 // y-> = NULL; 7 Set Next L(y, t) 8 // y-> = t; 8 Set Null L(t) 2 // t = NULL; Figure 6: The TVP file for the ruig example show i Figure 1. Files sll pred.tvp, sll cod.tvp, ad sll stat.tvp are give i Figures 7, 10, ad 11 respectively. 17

18 /* sll pred.tvp */ foreach (z i PVar) { %p z(v 1 ) uique box // Core predicates correspodig to program variables %p (v 1,v 2 ) fuctio // -field core predicate %i is[](v) = v 1,v 2 :((v 1,v) (v 2,v) v 1 v 2 ) // Is shared istrumetatio foreach (z i PVar) { %i r[, z](v) = v 1 :(z(v 1 ) (v 1,v)) // Reachability istrumetatio %i c[](v) = v! : (v, v 1 ) (v 1,v) // Cyclicity istrumetatio Figure 7: The TVP predicate declaratios for maipulatig liked lists as declared i Figure 1 (a). The core predicates are take from Table 2. Istrumetatio predicates are take from Table 3. Predicates: The predicates for maipulatig sigly liked lists as declared i Figure 1(a) are give i Figure 7. The foreach clause iterates over all the program variables i the set PVar ad for each of them defies the appropriate core predicate the uary predicates x, y, ad t (box tells TVLA to display the predicate as a box). The biary predicate represets the poiter field. For readability, we use some mathematical symbols here that are writte i C-like sytax i the actual TVP file (see [LA00, Appedix B]). The secod foreach clause (i Figure 7) uses PVar to defie the reachability istrumetatio predicates for each of the variables of the program (as opposed to Table 3, which is program specific). Thus, to aalyze other programs that maipulate sigly liked lists the oly declaratio that is chaged is that of PVar. The fact that the TVP file is specific for the data type L declared i Figure 1(a) allows us to explicitly refer to. Fuctioal properties: TVLA also supports a cocept of fuctioal properties borrowed from the database commuity. Sice program variables ca poit to at most oe heap cell at a time, they are declared as uique. The biary predicate represets the poiter field ; the -field of each list elemet ca oly poit to at most oe target list elemet, ad thus is declared as a (partial) fuctio. 18

19 3.1.2 Actios I the secod sectio of the TVP file, we defie actios that specify the operatioal sematics of program statemets ad coditios. A actio defies a 2-valued structure trasformer. The actios are associated with CFG edges i the third sectio of the TVP file. A actio specificatio cosists of several parts, each of which is optioal (the meaig of these costructs is explaied i Sectio 3.2). There are three major parts to the actio: (i) Focus formulae (explaied i Sectio 3.2.1), (ii) precoditio formula specifyig whe the actio is evaluated, ad (iii) update formulae specifyig the actual structure trasformer. For example, the actio Is Null Var(x1) (see Figure 10) specifies whe the true brach of the coditio x1 == NULL, is eabled by meas of the formula v : x1(v), which holds if x1 does ot poit to ay list elemet. Sice this coditio has o side effects there are o update formulae associated with this actio ad thus the structure remais uchaged. As aother example, the actio Copy Var L(x1, x2) (see Figure 11) specifies the sematics the statemet x1 = x2. It has o precoditio, ad its side effect is to set the x1 predicate to x2 ad the r[, x1] predicate to r[, x2] CFG The third sectio of the TVP specificatio is the CFG with actios associated with each of its edges. The edges are specified as source actio target. The first CFG ode that appears i the specificatio is the etry ode of the CFG. The CFG specificatio for the ruig example, is give i Figure Process This sectio presets a more detailed explaatio, usig the example show i Figure 8, of how the effect of a actio associated with a CFG edge is computed. To complete the picture, a iterative (fixed-poit) algorithm to compute the result of static-aalysis is preseted i Sectio Focus First, the Focus operatio coverts the iput structure ito a more refied set of structures that represets the same 2-valued structures as the iput structure. Give a formula, Focus guaratees that the formula ever evaluates to 1/2 i the focused structures. Focus (ad Coerce) are sematic 19

20 iput structure x y u0 r[,x] r[,y] u r[,x] r[,y] focus formulae x u0 r[,x] r[,y] y x u0 r[,x] r[,y] S i { v 1 : x(v 1 ) (v 1,v) y x y u0 r[,x] r[,y] u.1 r[,x] r[,y] focused structures update formulae u r[,x] r[,y] y u0 r[,y] u r[,x] r[,y] u.0 r[,x] r[,y] S f0 S f1 S f2 Predicate Update Formula x(v) v 1 : x(v 1 ) (v 1,v) r[, x](v) r[, x](v) (c[](v) x(v)) x y u0 r[,y] y x u0 r[,y] u.1 r[,x] r[,y] output structures u r[,y] u r[,x] r[,y] u.0 r[,x] r[,y] S o0 S o1 S o2 y x x u0 r[,y] y u0 r[,y] u.1 r[,x] r[,y] coerced structures u r[,x] r[,y] 20 S c1 u.0 r[,x] r[,y] S c2 Figure 8: The first applicatio of abstract iterpretatio for the statemet x = x-> i the reverse fuctio show i Figure 1.

21 reductios (see [CC79]), i.e., they trasfer a 3-valued structure ito a set of 3-valued structures represetig the same memory states. A algorithm for Focus of a geeral formula is give i [LA00]. I the ruig example, the most iterestig focus formula is v 1 : x(v 1 ) (v 1,v), which determies the value of the variable x after the Get Next L(x, x) actio (which correspods to the statemet x = x->). Focusig o this formula esures that x S (u) is defiite at every ode u i every structure S after the actio. Figure 8 shows how the structure S i is focused for this actio. Three cases are cosidered i refiig S i : (i) The -field of u 0 does ot poit to ay of the list elemets represeted by u (S f0 ); (ii) The -field of u 0 poits to all of the list elemets represeted by u (S f1 ); ad (iii) The -field of u 0 poits to oly some of the list elemets represeted by u (S f2 ): u is bifurcated ito two odes odes poited to by the -field of u 0 are represeted by u.1, ad odes ot poited to by the -field of u 0 are represeted by u.0. As explaied later, the result ca be improved (e.g., S f0 ca be discarded sice u is ot reachable from x, ad yet r[, x] S f0(u) = 1). This is solved by the Coerce operatio, which is applied after the abstract iterpretatio of the statemet (see Sectio 3.2.4) Precoditios After Focus, precoditios are evaluated. If the precoditio formula is potetially satisfied, the the actio is performed; otherwise, the actio is igored. This mechaism comes i hady for (partially) iterpretig program coditios. I the ruig example, the loop while (x!= NULL) has two outgoig edges i the CFG: oe with the precoditio ( v : x(v)), specifyig that if x is NULL the statemet followig the loop is executed (the exit i our case). The other edge has the precoditio v : x(v), specifyig that if x is ot NULL the loop body is executed Update Formulae The effect of the operatioal sematics of a statemet is described by a set of update formulae defiig the value of each predicate after the statemet s actio. The Embeddig Theorem eables us to reevaluate the formulae o the abstract structures ad kow that the result provides a coservative abstract sematics. If o update formula is specified for a predicate, it is left uchaged by the actio. 21

22 I Figure 8, the effect of the Get Next L actio (x = x->) is computed usig the followig update formulae: (i) x(v) = v 1 : x(v 1 ) (v 1,v), (ii) r[, x](v) = r[, x](v) (c[](v) x(v)). The first formula updates the x variable to be the -successor of the origial x. The secod formula updates the iformatio about which odes are reachable from x after the actio: A ode is reachable from x after the actio if it is reachable from x before the actio, except for the ode directly poited to by x (uless x appears o a -cycle, i which case the ode poited to by x is still reachable eve though we advaced to its -successor). For S f2, the update formula for x evaluates to 1 for v u.1 ad to 0 for all odes other tha u.1. Therefore, after the actio, the resultig structure S o2 has x S o2 (u.1) = 1 but x S o2 (u.0) = 0 ad x S o2 (u 0 ) = Coerce The last stage of the computatio is the Coerce operatio, which uses a set of cosistecy rules (defied i [SRW99, SRW00, LA00]) to make structures more precise by removig uecessary idefiite values ad discardig ifeasible structures. The set of cosistecy rules used is idepedet of the curret actio beig performed. See [LA00] for a detailed descriptio of the Coerce algorithm used i TVLA ad how TVLA automatically geerated cosistecy rules from the istrumetatio predicates ad the fuctioal properties of predicates. For example, Figure 8 shows how the Coerce operatio improves precisio. The structure S o0 is ifeasible because the ode u must be reachable from y (sice r[, y] S o0 (u) = 1) ad this is ot the case i S o0. I the structure S o1, u is o loger a summary ode because x is uique; u s self-loop is removed because u already has a icomig -field ad it does ot represet a shared list elemet (is[] S o1 (u) = 0). For the same reaso, i S o2, u.1 is o loger a summary ode; Also, the list elemet represeted by u.1 already has a icomig -field ad it is ot shared (is[] S o2 (u.1) = 0), ad thus u.1 s self-loop is removed. For a similar reaso, the idefiite -edge from u.0 to u.1 is removed Blur To guaratee that the aalysis termiates o programs cotaiig loops, we require the umber of potetial structures for a give program to be fiite. Toward this ed, we defie the cocept of a bouded structure. For each aalysis, we choose a set of uary predicates called the abstractio 22

23 predicates. 2 I the bouded structure, two odes u 1, u 2 are merged if p S (u 1 )=p S (u 2 ) for each abstractio predicate p. Whe odes are merged, the predicate values for their o-abstractio predicates are joied (i.e., the result is 1/2 if their values are differet). This is a form of wideig (see [CC79]). The operatio of computig this kid of bouded structure is called Blur. The choice of abstractio predicates is very importat for the balace betwee space ad precisio. TVLA allows the user to select the abstractio predicates. By default, all the uary predicates are abstractio predicates, as i the ruig example. Example 3.1 I Figure 4, the odes u 0 ad u are differetiated by the fact that x S 4 (u 0 ) = 1, whereas x S 4 (u) = 0. (All other predicates are 0.) If x was ot a abstractio predicate, the the appropriate bouded structure S 4 would have had a sigle ode, say u, with xs 4 (u) =1/2 ad S 4 (u, u) = 1/2. After the actio is computed ad Coerce applied, the Blur operatio is used to trasform the output structures ito bouded structures, thereby geeratig more compact, but potetially less precise structures. 3.3 Output Now that we have a method for computig the effect of a sigle actio, what remais is to compute the effect of the whole program, i.e., to compute what structures ca arise at each CFG ode if the program was used o the give iput structures. We use a stadard iterative algorithm (e.g., see [Muc99]) with a set of bouded structures as the abstract elemets. A ew structure is added to the set if the set does ot already cotai a member that is isomorphic to the ew structure. I the ruig example, the aalysis termiates whe the structures created i the fourth iteratio are isomorphic to the oes created i the third iteratio (see Figure 9). We ca see that the aalysis precisely captures the behavior of the reverse program. 3.4 Additioal Features The system allows several customizatios o the stadard iterative algorithm for optimizig the aalysis (as commad lie optios). The user ca choose whether the actios are evaluated i depth first search post-order or reverse depth first search post-order. Eve though reverse depth first search 2 I [SRW99, SRW00] the abstractio predicates are all the uary predicates. 23

24 Iter Structures x 0 r[,x] x r[,x] x y r[,x] y r[,x] 1 r[,x] y r[,y] r[,y] x r[,y] r[,y] r[,x] y x y r[,x] 2 r[,y] r[,y] r[,x] y r[,y] x r[,y] r[,y] r[,y] r[,x] r[,x] y x y 3 r[,y] r[,y] r[,x] r[,y] r[,y] Figure 9: The structures arisig i the reverse fuctio show i Figure 1 at CFG ode 2 for the iput structure show i Figure 4. 24

25 post order is usually more efficiet (sice a actio is evaluated oly after its predecessors are evaluated), usig post order causes structures to reach the ed of the program more quickly. This is very useful i case the aalysis is ot feasible ad yet we wat to see a glimpse of what is expected, a example for such a aalysis is the merge fuctio without reachability as defied i Sectio 4.1. Aother form of customizatio is the choice of CFG odes i which the set of structures is saved. At the miimum at least oe such ode should reside o each loop i the CFG. Three forms are available: (i) at every CFG ode, (ii) at every merge poit (i.e., CFG ode with two icomig edges), ad (iii) at every back edge of the depth first search tree. The more CFG odes i which the structures are saved the faster the aalysis is (sice structures eed ot be recreated). However, more space is eeded (by factor of 10 eve for simple programs). Oe of the mai features of TVLA is the support of sigle structure aalysis. Sometimes whe the umber of structures that arise at each program poit is too large, it is better to merge these structures ito a sigle structure that represets at least the same set of 2-valued structures. TVLA ehaces this feature eve more by allowig the user to specify that some chose costat umber of structures will be associated with each program poit. More specifically, ullary predicates (i.e., predicates of 0-arity) are used to discrimiate betwee differet structures. For example, for liked lists we use the predicate [x]() = v : x(v) which discrimiates betwee structures i which x actually poits to a list elemet from structures i which it does ot. For example, cosider a structure S 1 i which both x ad y poit to list elemets, ad aother structure S 2 i which both x ad y are NULL. Mergig S 1 ad S 2 will loose the iformatio that x ad y are simultaeously allocated or ot allocated. Notice that S 1 has [x] =[y] = 1 ad S 2 has [x] =[y] = 0 therefore S 1 ad S 2 will ot be merged together. I some cases (such as safety aalysis of Mobile Ambiets, see [NNS00]) this optio makes a otherwise ifeasible aalysis ru i a reasoable time. However, there are other cases i which the sigle-structure method is less precise or eve more time cosumig tha the usual method, which uses sets of structures. TVLA also supports modelig statemets that hadle dyamically allocated ad freed memory. 25

26 /* ptr cod.tvp */ %actio Is Not Null Var(x1) { %t x1 +!= NULL %f { x1(v) %p v : x1(v) %actio Is Null Var(x1) { %t x1 + == NULL %f { x1(v) %p ( v : x1(v)) %actio Is Eq Var(x1, x2) { %t x1 + == + x2 %f { x1(v), x2(v) %p v : x1(v) x2(v) %actio Is Not Eq Var(x1, x2) { %t x1 +!= + x2 %f { x1(v), x2(v) %p v : x1(v) x2(v) Figure 10: A operatioal sematics i TVP for hadlig poiter coditios. 4 Applicatios 4.1 Sigly Liked Lists We used the fuctios aalyzed i [DRS00] with sharig ad reachability istrumetatios (see Table 4). The specificatio for all the fuctios was writte oce ad used with each of the CFGs. The actios for hadlig program coditios that cosists of poiter equalities ad iequalities are give i Figure 10. The actios for maipulatig the struct ode declaratio from Figure 1(a) are give i Figure 11. The actios Set Next Null L ad Set Next L model destructive updatig (i.e., assigmet to x1->), ad therefore have a otrivial specificatio. We use the otatio ϕ 1?ϕ 2 : ϕ 3 for a if-the-else clause. If ϕ 1 is 1 the the result is ϕ 2, if ϕ 2 is 0 the the result is ϕ 3. If ϕ 1 is 1/2 the the result is ϕ 2 ϕ 3. We use the otatio TC(v 1,v 2 )(v 3,v 4 ) for the trasitive-closure operator. The variables v 3 ad v 4 are the free variables of the sub-formula over which the trasitive closure is performed, ad v 1 ad v 2 are the variables used o the resultig biary relatio. Most of the aalyses were very precise with ruig times of up to 8 secods for the most complex fuctio (merge). 26

27 /* sll stat.tvp */ %actio Set Null L(x1) { %t x1 + = NULL { x1(v) =0 r[, x1](v) = 0 %actio Copy Var L(x1, x2) { %t x1 + = + x2 %f { x2(v) { x1(v) =x2(v) r[, x1](v) =r[, x2](v) %actio Malloc L(x1) { %t x1 + = (L) malloc(sizeof(struct ode))) %ew { x1(v) =isn ew(v) r[, x1](v) =isn ew(v) %actio Free L(x1) { %t free(x1) %f {x1(v) %message v 1,v 2 : x1(v 1 ) (v 1,v 2 ) -> Iteral error! assume that + x1 + -> + + ==NULL %retai x1(v) %actio Get Next L(x1, x2) { %t x1 + = + x2 + -> + %f { v 1 : x2(v 1 ) (v 1,v) { x1(v) = v 1 : x2(v 1 ) (v 1,v) r[, x1](v) =r[, x2](v) (c[](v) x2(v)) %actio Set Next Null L(x1) { %t x1 + -> + + = NULL %f { x1(v) { (v 1,v 2 )=(v 1,v 2 ) x1(v 1 ) is[](v) =is[](v) ( ( v 1 : x1(v 1 ) (v 1,v)) v 1,v 2 :((v 1,v) x1(v 1 )) ((v 2,v) x1(v 2 )) v 1 v 2 ) r[, x1](v) =x1(v) foreach(z i PVar-{x1) { r[, z](v) =(c[](v) r[, x1](v)? z(v) v 1 : z(v 1 ) TC(v 1,v)(v 3,v 4 )((v 3,v 4 ) x1(v 3 )) : r[, z](v) (r[, x1](v) x1(v) v 1 : r[, z](v 1 ) x1(v 1 ))) c[](v) =c[](v) ( v 1 : x1(v 1 ) c[](v 1 ) r[, x1](v)) %actio Set Next L(x1, x2) { %t x1 + -> + + = + x2 %f { x1(v),x2(v) %message v 1,v 2 : x1(v 1 ) (v 1,v 2 ) -> Iteral error! assume that + x1 + -> + + ==NULL { (v 1,v 2 )=(v 1,v 2 ) x1(v 1 ) x2(v 2 ) is[](v) =is[](v) v 1 : x2(v) 27 (v 1,v) foreach(z i PVar) { r[, z](v) =r[, z](v) r[, x2](v) v 1 : r[, z](v 1 ) x1(v 1 ) c[](v) =c[](v) (r[, x2](v) v 1 : x1(v 1 ) r[, x2](v 1 )) Figure 11: A operatioal sematics i TVP for hadlig the poitermaipulatio statemets of liked lists as declared i Figure 1(a).

28 program search ull deref delete del all isert create merge reverse fumble rotate swap getlast descriptio searches for a elemet i a liked list searches a liked list but with a typical error of ot checkig for the ed of the list deletes a give elemet from a liked list deletes a etire liked list iserts a elemet ito a sorted liked list preped a varyig umber of ew elemets to a liked list merges two sorted liked lists ito oe sorted list reverses a liked list via destructive updates a erroeous versio of reverse which loses the list performs a cyclic rotatio whe give poiters to the first ad last elemets swaps the first ad secod elemets of a list, fails whe the list is 1 elemet log returs the last elemet of the list Table 4: Descriptio of the aalyzed sigly liked list programs. These programs are collectios of iterestig programs from LCLit [Eva96], [JJNS97], Thomas Ball ad from first-year studets. They are available at urr. x x u0 r[,x] u r[,x] last u1 r[,last] r[,x] u3 r[,x] u2 r[,x]=1/2 last u4 r[,last] r[,x]=1/2 Figure 12: The structure before ad after the rotate fuctio. 28

29 The rotate fuctio gives a example of a aalysis which is ot as precise as possible (see Figure 12). The idefiite edge [u.1, [u.0, u0].0], u1 is superfluous ad all the list should be kow to be reachable from x. The imprecisio arises because the list becomes cyclic i the process ad the reachability update formula i the actio Set Next Null L show i Figure 11 is ot very precise i case of cyclic lists. The merge fuctio is a good example of how precisio problems i the aalysis ca create too may structures. Aalyzig the merge fuctio without the reachability istrumetatio creates tes of thousads of graphs ad takes too much space for the machie we were usig. Addig the reachability iformatio reduces the umber of graphs to 327 ad the time to about 8 secods. 4.2 Doubly Liked Lists Doubly liked lists are a example of a more complex abstract datatype which ca still be aalyzed accurately i may cases. The aalysis is also a good example for how uaces i the istrumetatio predicates used ca chage the accuracy of the aalysis. We use a stroger istrumetatio predicate tha the oe described i [SRW99], i.e., we keep a stroger program ivariat. We show how usig the ew istrumetatio icreases the accuracy of the aalysis. It is hard to predict how these small differeces are goig to affect the aalysis, this demostrates the importace of the system as a platform for developig ad testig ew aalysis algorithms. The splice fuctio aalyzed ad the appropriate data structure are give i Figure 13. The TVP for splice is specified i figures 14, 16, ad 15. The istrumetatio predicate that eables us to aalyze the DLL programs precisely is the cacel f by b (c[f, b]), a uary predicate statig that v->f->b == v. A similar istrumetatio is maitaied for cacel b by f. The formula used i [SRW99] for the istrumetatio is, c[f, b](v) = v 1,v 2 : f(v, v 1 ) b(v 1,v 2 ) v 1 = v 2 (6) whe tryig to ru the aalysis o the splice fuctio utilizig this defiig formula, we foud out that the aalysis was ot as precise as we wated. We came up with the followig defiitio of the istrumetatio predicate, c[f, b](v) = v 1 : f(v, v 1 ) b(v 1,v) (7) The differece may seem isigificat. However, (7) is stroger tha (6), i.e., a ode that satisfies (7) must satisfy (6) but ot vice versa. The costraits 29

30 /* dlist.h */ typedef struct DNode { struct DNode *f, *b; it data; DNode, *DL; (a) /* splice.c */ #iclude dlist.h void splice(it v, DL p) { DL e, t; e = (DL)malloc(sizeof(DNode)); e->data = v; t = p->f; e->f = t; (b) if (t!= NULL) t->b = e; p->f = e; e->b = p; Figure 13: (a) Declaratio of a doubly liked-list data type i C. (b) A program that splices a elemet with a data value v ito a doubly liked list with a head poited by l, after a elemet poited to by p. geerated by the system from this istrumetatio, ad, c[f, b](v) f(v, v 1 ) b(v 1,v) c[f, b](v) b(v 1,v) f(v, v 1 ) are very importat i keepig the aalysis of the splice program precise. For example, Lets look at the simple t = p->f istructio with both versios of the istrumetatio o the structure i Figure 17. Figure 17 depicts a doubly liked list poited to by p. Figure 18 illustrates a 4.3 Sortig Algorithms I this sectio, we describe how the 3-valued-logic aalysis framework ca be used to prove that a implemetatio of a abstract datatype (ADT) is partially correct. Here we will be cocered with a ADT of sorted liked lists i.e., a subset of the full set of data structures allowed accordig to the C typedef show i Figure 1(a), cosistig of those structures that 30