Towards imperative modules: reasoning about invariants and sharing of mutable state

Towards imperative modules: reasoning about invariants and sharing of mutable state David A. Naumann Joint work with Mike Barnett and Anindya Banerjee Stevens Institute of Technology Supported by NSF CCR-0208984, CCF-0429894, and Microsoft. NJPLS 1 Oct 2004 / 1

Outline Difficulty in reasoning about object invariants due to callbacks and heap sharing programmer s view logician s view The boogie and friends disciplines: state based encapsulation (with Mike Barnett [LICS]) Representation independence (with Anindya Banerjee) Related work and Phd/postdoc advert NJPLS 1 Oct 2004 / 2

Programmer s intro: object invariants class Subject { private x,y: int := 0,1; invariant I(self) where I is defined by I(o) = o.x method m() { self.x := self.x+1; o.y self.y := self.y+1; }... } NJPLS 1 Oct 2004 / 3

Programmer s intro: object invariants class Subject { private x,y: int := 0,1; obs: Observer :=... ; invariant I(self) where I is defined by I(o) = o.x method m() { self.x := self.x+1; obs.notify(); self.y := self.y+1; }... } o.y NJPLS 1 Oct 2004 / 3-a

Programmer s intro (2): sharing class Subject2 { private x: Integer := new Integer(0); private y: Integer := new Integer(1); invariant I(self) where I(o) = o.x.val o.y.val method m() { self.x.incr(); self.y.incr(); } method leak(): Integer { result := x; } } class Main { s: Subject2; i: Integer;... i := s.leak(); i.incr(); s.m()... } NJPLS 1 Oct 2004 / 4-a

Programmer s intro (2): sharing class Subject2 { private x: Integer := new Integer(0); private y: Integer := new Integer(1); invariant I(self) where I(o) = o.x.val method m() { self.x.incr(); self.y.incr(); } method leak(): Integer { result := x; } } class Main { s: Subject2; i: Integer;... i := s.leak(); i.incr(); s.m()... } o.y.val How can we encapsulate not just fields but also referenced objects? NJPLS 1 Oct 2004 / 4-b

I Logician s intro P Q I I is encapsulated for P call Q Declaration:. Specification P fields; I depends on the internal representation. Q involves public NJPLS 1 Oct 2004 / 5

I Q Logician s intro P Q I I is encapsulated for P call Q Declaration:. Specification P fields; I depends on the internal representation. Q involves public P does not interfere* with I P I Q I * does not write variables read in I (hazard: aliased vars) * does not update objects read in I (hazard: heap sharing) NJPLS 1 Oct 2004 / 5-b

call call Logicians intro (2) R R P P P I Q. Q I Q NJPLS 1 Oct 2004 / 6

call Logicians intro (2) R P R I P I R I no interference P P I Q I call I Q. I Q I NJPLS 1 Oct 2004 / 6-a

Q call call Logicians intro (2) R R R P R I I P I R I no interference P P I Q I call I Q. I Q I Q I NJPLS 1 Oct 2004 / 6-b

Q call call Logicians intro (2) R R R P R I I P I R I no interference P P I Q I Q I How generalize to multiple instantiation, i.e., I call I Q. I Q I for all? NJPLS 1 Oct 2004 / 6-c

Logicians intro (2) R R P R I R no interference I P I R I call call Q P P I Q I I I. Q call Q I Q I How generalize to multiple instantiation, i.e., I for all? What about outcalls, i.e. method invocations on in other objects, which may lead to reentrant callbacks? NJPLS 1 Oct 2004 / 6-d

An assertion-based discipline Problems: due to reentrant callbacks, precondition P I for is unsound unless I re-established before outcalls NJPLS 1 Oct 2004 / 7

An assertion-based discipline Problems: due to reentrant callbacks, precondition P I for is unsound unless I re-established before outcalls need to protect I Subject from interference by code in other classes and by other instances of Subject Solution uses a single everywhere-invariant, PI. PI P Rule:. P Q Q NJPLS 1 Oct 2004 / 7-b

Auxiliary field to make explicit when invariant holds: boolean Maintain program invariant PI Itype NJPLS 1 Oct 2004 / 8

Auxiliary field to make explicit when invariant holds: boolean Maintain program invariant PI Itype class Subject {... invariant I method m() { (self) where I (o) = o.x assert self.inv (* precondition *) unpack self; (* self.inv := false *) self.x := self.x+1; obs.notify(); self.y := self.y+1; (* I(self) *) pack self; (* self.inv := true *) }... } class Main... method notify() { assert z.inv?; z.m(); } o.y NJPLS 1 Oct 2004 / 8-a

PI Auxiliary field to make explicit when invariant holds: boolean Maintain program invariant PI Itype class Subject {... invariant I method m() { (self) where I (o) = o.x assert self.inv (* precondition *) unpack self; (* self.inv := false *) self.x := self.x+1; obs.notify(); self.y := self.y+1; (* I(self) *) pack self; (* self.inv := true *) }... } class Main... method notify() { assert z.inv?; z.m(); } o.y Absence of interf., as a precond.: PI NJPLS 1 Oct 2004 / 8-b

Auxiliary field to delimit heap dependence of invariant: Def: iff either or. NJPLS 1 Oct 2004 / 9

Auxiliary field to delimit heap dependence of invariant: Def: iff either or. Def: I is admissible iff when I depends on then either or. NJPLS 1 Oct 2004 / 9-a

Auxiliary field to delimit heap dependence of invariant: Def: iff either or. Def: I is admissible iff when I depends on then either or. Absence of interference, as a precondition: PI PI Ownership provides stateful encapsulation: control is inside the boundary for. means NJPLS 1 Oct 2004 / 9-b

Last auxiliary field for ownership discipline: boolean PI Itype NJPLS 1 Oct 2004 / 10

Last auxiliary field for ownership discipline: boolean PI Itype Absence of interf., as a precond.: PI PI NJPLS 1 Oct 2004 / 10-a

with Last auxiliary field for ownership discipline: boolean PI Itype Absence of interf., as a precond.: Precondition and effect of unpack assert := false; forall ; : do PI PI := false; NJPLS 1 Oct 2004 / 10-b

with with Last auxiliary field for ownership discipline: boolean PI Itype Absence of interf., as a precond.: Precondition and effect of unpack assert ; := false; forall Precondition and effect of pack assert Itype := true; forall ; : : do do PI PI := false; := true; NJPLS 1 Oct 2004 / 10-c

PI Ownership transfer PI Special command setowner to highlight that it only manipulates auxiliary state (like unpack/pack. State-based encapsulation (vs. type systems): avoids restriction on existence or reading of references allows transfer of objects across boundaries examples: lexer/stream, AST (into); tasks (between); database connections (in and out) NJPLS 1 Oct 2004 / 11

PI Q Q Stateful encapsulation I. Def: is properly annotated iff each pack, unpack, setowner, and field update has stipulated precondition. Theorem: PI for any properly annotated Justifies rule: PI P P Proof: using a straightforward denotational semantics for a sequential language with mutually recursive class declarations and methods etc. NJPLS 1 Oct 2004 / 12

Stateful encapsulation II friends. A List owns its nodes. A node does not own its neighbors. class List { head: ListNode; invariant self.head=null class ListNode { next, prev: ListNode; invariant self.next=null (self.next.prev=self self.head.prev=null;... } self.next.own=self.own);... } NJPLS 1 Oct 2004 / 13

depends on Absence of interference, as a precondition: I I NJPLS 1 Oct 2004 / 14

depends on Absence of interference, as a precondition: I I Auxiliary field for friendship discipline: set of I NJPLS 1 Oct 2004 / 14-a

depends on depends on Absence of interference, as a precondition: I I Auxiliary field for friendship discipline: set of I Admissibility: when I then either, or for some declared pivot and, NJPLS 1 Oct 2004 / 14-b

Program equivalence: two-state invars class Subject2 { private x: Integer := new Integer(0); private y: Integer := new Integer(1); invariant I(self) where I(o) = o.x.val o.y.val method m() { self.x.incr(); self.y.incr(); } class Subject2 { // Alternate version private x: int := 0; private z: Integer := new Integer(1); invariant I(self) where I(o) = 0 o.z.val method m() { self.x := self.x + 1; } Coupling relation: o.x = o.x.val o.z = o.y.val o.x.val NJPLS 1 Oct 2004 / 15

Towards simulation: admissibility revisited Let be an instance of the class to be revised. Partition, where and the set of objects transitively owned by in. Then I iff I. NJPLS 1 Oct 2004 / 16

Towards simulation: admissibility revisited Let be an instance of the class to be revised. Partition, where and the set of objects transitively owned by in. Then I iff I. PI implies: If we choose top-level instances of in, have for I for all where. NJPLS 1 Oct 2004 / 16-a

Coupling for two versions of Heaps coupled just if there are same-length partitions NJPLS 1 Oct 2004 / 17

by a given coupling. Coupling for two versions of Heaps coupled just if there are same-length partitions such that for each pair, relates to and implies NJPLS 1 Oct 2004 / 17-a

Coupling for two versions of Heaps coupled just if there are same-length partitions such that for each pair, and implies relates to by a given coupling. Moreover (modulo bijective renaming of locations). corresponds to NJPLS 1 Oct 2004 / 17-b

Coupling for two versions of Heaps coupled just if there are same-length partitions such that for each pair, and implies relates to by a given coupling. Moreover (modulo bijective renaming corresponds to of locations). Identity on visible state (fields in, interface of ). NJPLS 1 Oct 2004 / 17-c

Abstraction theorem Theorem If the induced coupling is a simulation, i.e., is preserved by the methods of the revised class, then it is preserved by all contexts. If coupling holds at boundaries of everywhere that does. then outside it holds Reentrant callbacks and invariants: a method that does not require cannot rely on I; that s all. Reentrant callbacks and simulation: a method that does not require must still preserve how? (vs. invariant case where some precondition can help)? Need modifies spec. NJPLS 1 Oct 2004 / 18-b

(e.g., callbacks from notify can inspect the Subject but not alter the datastruct tracking Observers). NJPLS 1 Oct 2004 / 19

(e.g., callbacks from notify can inspect the Subject but not alter the datastruct tracking Observers). NJPLS 1 Oct 2004 / 19-a

(e.g., callbacks from notify can inspect the Subject but not alter the datastruct tracking Observers). NJPLS 1 Oct 2004 / 19-b

Conclusion Discipline for control of dependence for object invariants. Controls use of pointers rather than their existence. Handles difficult design patterns that are common in practice. No restrictions on heap structure. No committment to particular program logic or verification system. Uses verification conditions; not special type annotation but not fully automated. NJPLS 1 Oct 2004 / 20

Related work Leino et al [JoT, ECOOP04, CASSIS04] Boogie, Spec# with concurrency O Hearn et al [POPL04]; Mijajlović et al [FSTTCS 04] static modularity for separation logic Parkinson & Bierman [POPL05] instantiable abstraction in sep. logic using scope of predicate definitions Hongseok Yang [TCS?] relational sep. logic full logic and mechanization Pierik and de Boer NJPLS 1 Oct 2004 / 21

Future work precise comparison with Separation Logic: P Q P Q P I Q I P I implementation and case studies Spec# project friends and subclassing; generalization to multi-class patterns Barnett and Naumann integrate with ownership typing, extend simu to concurrent Banerjee and Naumann machine check soundness proof Naumann Q I NJPLS 1 Oct 2004 / 22

Advert Seeking PhD student or postdoc to develop these ideas in context of JML, a specification language used by ESC/Java and several other systems e.g. smartcard verif. (Joint project with Iowa State (Gary Leavens) and UFPE, Recife, Brazil.) NJPLS 1 Oct 2004 / 23

References Barnett, DeLine, Fähndrich, Leino, Wolfram Schulte: Verification of object-oriented programs with invariants (Journal of Object Technology 04) Leino and Müller: Object invariants in dynamic contexts (ECOOP 04) Barnett and D.N.: Friends need a bit more (MPC 04) O Hearn, Yang, Reynolds: Separation and Info Hiding (POPL 04) Banerjee and D.N.: State based ownership and encapsulation for generic classes NJPLS 1 Oct 2004 / 24