Concurrent library abstraction without information hiding

Universidad Politécnica de Madrid Escuela técnica superior de ingenieros informáticos Concurrent library abstraction without information hiding Artem Khyzha Advised by Manuel Carro and Alexey Gotsman Madrid 2014

Concurrent libraries Encapsulate efficient concurrent algorithms. - Java: java.util.concurrent - C++: Intel Threading Building Blocks - C#: System.Collections.Concurrent Implement stacks, queues, skip lists, hash tables, etc. But it is not easy to understand why they are correct. 2

Concurrent library correctness Non-blocking stack: t1: (t1, call push(42)) (t1, ret push) (t1, call isempty) (t1, ret isempty(yes)) t2: (t2, call pop) (t2, ret pop(42)) (t2, call push(11)) (t2, ret push) 3

Concurrent library correctness Non-blocking stack: Why is it OK to return empty here? t1: (t1, call push(42)) (t1, ret push) (t1, call isempty) (t1, ret isempty(yes)) t2: (t2, call pop) (t2, ret pop(42)) (t2, call push(11)) (t2, ret push) 4

Concurrent library correctness Non-blocking stack: It can also return no t1: (t1, call push(42)) (t1, ret push) (t1, call isempty) (t1, ret isempty(no)) t2: (t2, call pop) (t2, ret pop(42)) (t2, call push(11)) (t2, ret push) 5

A binary relation L L on libraries L and L. Usually, L is an impl. and L is a spec. 6

Concurrent library specification Non-blocking stack Atomic stack ADT struct Node { Node *next; int val; } *Top; void push(int v) { Node *t, *x; x = new Node; x->val = v; do { t = Top; x->next = t; } while(!cas(&top,t,x)); } Sequence S; void push(int v) { atomic { S = v S; } }

Library abstraction Abstraction theorem: it is sound to replace a library with its specification in reasoning about its client. If L L, then C(L ) can recreate executions of C(L). Proofs about C(L ) work for C(L). EXAMPLE OF THE PROPERTY. 8

Problem Linearizability hasn t been defined even for settings concurrent libraries get used in 9

Information hiding in linearizability C call m(42) ret m(0) L 10

Information hiding in linearizability C ret m(0) call m(42) Linearizability assumes a complete information hiding between the library and its client. L The client and the library exchange data via library interface. 10

Information hiding in fine-grained concurrency C read x ++x; x >? read x x = 1 ++x L 11

Information hiding in fine-grained concurrency C ++x; read x x >? x = 1 ++x L read x Clients and libraries share address space and communicated via shared memory for efficiency. Following the contract between components makes the program correct. 11

Information hiding in fine-grained concurrency C increases x read x ++x; x >? read x x = 1 ++x initialises x increases x L 12

Information hiding in fine-grained concurrency C increases x read x Clients and libraries share address space and communicated via shared ++x; x >? x = 1 ++x initialises x increases x L read x memory for efficiency. Following the contract between components makes the program correct. 12

Information hiding in fine-grained concurrency C increases x read x ++x; x >? read x x = 1 ++x initialises x increases x L once assigned, x can only increase 13

Information hiding in fine-grained concurrency C increases x read x Clients and libraries share address space and communicated via shared ++x; memory for efficiency. x >? read x x = 1 ++x Following the contract between components makes the program initialises x increases x L correct. once assigned, x can only increase 13

Information hiding in fine-grained concurrency C increases x read x ++x; x >? read x x = 1 ++x initialises x increases x L 14

Information hiding in fine-grained concurrency C increases x read x Clients and libraries share address space and communicated via shared ++x; x >? x = 1 ++x initialises x increases x L read x 14 memory for efficiency. Following the contract between components makes the program correct. Example in the paper.

In this thesis 1. No information hiding assumed. 2. Formalisation of the contract between clients and libraries. 3. A different library abstraction relation; 4. The Abstraction theorem proved. C C L1 L2 15

In this thesis We want to be able to replace L1 with L2 while reasoning about C(L1). The contract between components has to be considered. C C L1 L2 16

The contract views The Views Framework is a generalisation of Hoare-style program logics. Parameters: Views, Reification, Axioms. Framework can be instantiated into Concurrent Separation Logic, Rely- Guarantee etc. POPL 13, Dinsdale-Young et al 17

The contract views Formally: (View, *) a semi-group of assertions with commutative operation. Views represent information about the contract, and * combines contracts. Reification maps y views > 42to sets of concrete states; x - i -> _ Axioms specify x - i -> how _ * y commands > 42 change 18

The contract views Axioms specify how commands change views: soundly w.r.t. a concrete semantics; considering permissions. { x > 42 } ++x { x > 43 } { x > 42 } --x { x > 41 } 19

The contract views Axioms specify how commands change views: soundly w.r.t. a concrete semantics; considering contract. { x - i -> _ } ++x { x - i -> _ } { x - i -> _ } --x { x - i ->! _ } 20

The contract views Reification p gives a meaning to a view p Programs work with machine states σ; Views are abstracts states: reification maps views to sets of machine states; Axioms specify σ how p commands change views. a machine state σ satisfies p 21

The contract semantics Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 Semantics in terms of traces sequences of labelled transitions between machine states. Idea: represent the information about the contract with views. LICS 93, Brookes 22

The contract semantics Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 p1 p2 p2 p3 p3 p3 p4 q1 q1 q2 q2 q3 q4 q4 Client and library have initial views p1 and q1 (parts of the contract to follow); Components only change their own views; 23

The contract semantics Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 p1 p2 p2 p3 p3 p3 p4 = Z = X # q1 q1 q2 q2 q3 q4 q4 = Y # Internally safe semantics of C(L) w.r.t. p1 and q1 the set of all possible (Z, X #, Y # ) 26

Semantics of programs Sint C(L) (pc, pl) = {(Z, X #, Y # ) where: } Z is a concrete trace (α1,σ1,σ 1) (αn,σn,σ n) X # is a client s abstract trace (α1, p1, p2) (αn, pn, pn+1) Y # is a library s abstract trace Views in X# and Y# are picked according to axioms Machine states satisfy views: σ1 p1*q1 X

The contract semantics Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 p1 p2 p2 p3 p3 p3 p4 q1 q1 q2 q2 q3 q4 q4 Client s vision: by C by C by C σ1 σ2 σ3 σ4 σ6 σ7 p1 p2 p2 p3 p3 p4 Library s vision: by L by L by L σ2 σ3 σ4 σ5 σ6 q1 q2 q2 q3 q4 X

The contract semantics Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 Client s vision: by C by C by C σ1 σ2 σ3 σ4 σ6 σ7 Library s vision: by L by L by L σ2 σ3 σ4 σ5 σ6 X

Abstraction theorem If L L, then executions of C(L) can be recreated by C(L ). 27

Abstraction theorem If L1 q1 L2 and initial contracts are p1 and q1,???????? for all internally safe traces (Z1, X # 1, Y # 1) of C(L1), C(L2) has (Z2, X # 2, Y # 2) such that: client s execution does not change: client(z1) = client(z2) library L 2 recreates the same state changes: states(lib(z1)) = states(lib(z2)) 28

Abstraction theorem If????????? for all internally safe traces ( C L1 q1 L2 must establish correspondence between client s execution possible does uses of not libraries change: without considering a specific client library Has to work for all clients 30

Abstraction theorem If L1 q1 L2 and initial contracts are p1 and q1,????????? for all internally safe traces (Z1, X # 1, Y # 1) of C(L1), C(L2) has (Z2, X # 2, Y # 2) such that: client s execution does not change: client(z1) = client(z2) library L 2 recreates the same state changes: states(lib(z1)) = states(lib(z2)) X

Library abstraction relation A binary relation q: every concrete behaviour is preserved; the contract is not stricter; the same methods are used. L2 L1 method calls contract any C 31

Library abstraction relation A binary relation v p : ensures that every concrete behaviour is preserved; ensures that the contract is not more restrictive. v 8 2 SJ K 9 2 SJ K (Y 1,Y # 1 ) v (Y 2,Y # 2 ), states(y v 1)=states(Y 2 )^views(y # 1 ) views(y # 2 ) ^interface(y 1 )=interface(y 2 ). L 1 v p L 2, 8(Y 1,Y # 1 ) 2 SJL 1K(p). 9(Y 2,Y # 2 ) 2 SJL 2K(p). (Y 1,Y # 1 ) v (Y 2,Y # 2 ) X

Library-local contract A possible use of L: by L by L by L σ2 σ3 σ4 σ5 σ6 q1 q2 q2 q3 q4 Client may change machine state any time but then library s view remains invariant 32

Library-local contract A possible use of L: α by L by L σ2 σ3 σ4 σ5 σ6 q1 q2 q2 q3 q4 Client may change machine state any time but then library s view remains invariant which is enabled by changing views w.r.t. axioms and the Frame property of them for all (client s) views r, {q1} α {q2} implies {q1 * r} α {q2 * r} 33

Library-local contract A possible use of L: by L by L by L σ2 σ3 σ4 σ5 σ6 q1 q2 q2 q3 q4 Client may change machine state any time but then library s view remains invariant Execution of C(L): by C by L by C by L by L by C σ1 σ2 σ3 σ4 σ5 σ6 σ7 p1 p2 p2 p3 p3 p3 p4 q1 q1 q2 q2 q3 q4 q4 34

Library-local semantics Execution of L: by L by L by L σ2 σ3 σ4 σ5 σ6 q1 q2 q2 q3 q4 Library has initial view q1 as the contract; Follows the contract: σi qi Changes views w.r.t. the axioms; 35

Library-local executions Library behaviours are generated by n special clients: MGCn(L) = C1 C2 Cn Ci = (call arbitrary method of L)* Client may change the machine state arbitrarily 36

Library-local executions Library behaviours are generated by n special clients: Library s vision: MGCn(L) = C1 C2 Cn Ci = (call arbitrary method of L)* Client may change the machine state arbitrarily by L by L by L σ2 σ3 σ4 σ5 σ6 37

Library abstraction relation A binary relation q: every concrete behaviour is preserved; the contract is not stricter; the same methods are used. L2 L1 method calls contract any C 38

Abstraction theorem If L1 q1 L2 and initial contracts are p1 and q1, for all internally safe traces (Z1, X # 1, Y # 1) of C(L1), C(L2) has (Z2, X # 2, Y # 2) such that: client s execution does not change: client(z1) = client(z2) library L 2 recreates the same state changes: states(lib(z1)) = states(lib(z2)) 39

Library-local contract Safety of L w.r.t. p checks that library follows its contract itself. safe(l, pl) requires safe(mgcn(l), pl) safe(p, p) is: L safe(l,p), V n 1 safe(mgc n(l),p) 8, P 0,, 0. 2bpc^hP, i = )hp 0, 0 i =) 9p 0.p 0 2 SP(,p) ^ safe(p 0,p 0 ) X

Abstraction theorem Proof is constructive: decomposition : split a trace of C(L1) on library s and client s parts; C L1 C L1 X

Abstraction theorem Proof is constructive: decomposition : split a trace of C(L1) on library s and client s parts; replace L1 with L2; L1 L2 p X

Abstraction theorem Proof is constructive: decomposition : split a trace of C(L1) on library s and client s parts; replace L1 with L2; composition : merge parts of a trace together. C C + L2 L2 X

Summary 1. Library abstraction relation for programs interacting only on the shared memory; 2. Formalisation of the contract between clients and libraries with internally safe traces. 3. Proof of the Abstraction theorem. 40

Future work Linearizability Complete information hiding Communication via interface of the library? Partial information hiding? Library abstraction (in this work) No information hiding Communication via shared memory 41

Future work 1. Library abstraction with partial information hiding; 2. Real-world examples; 3. Relational logic for establishing q. 42

transition trace ofsafe L2 can mumbled to semantics fitto the ch eained able we are to prove able to safe prove and internally and be internally safe semantics safe be to b two consequent transitions. Consideration of this Abstraction Theorem that justifies abstracting f slosed that in transition traces of a closed program, states are not ch rams.programs Closed programs work in the work absence in the absence of the environment, of the environment safe semantics as follows: uent transitions. Consideration of this adjusts definitions of saft specification while reasoning about its nces of library abstraction is sound by establishing theclient. itionoftraces a closed of aprogram, closed program, states are states not changed are not changed between betwee as follows: bstracting from an a library with itsjpk(p) a adjusts client a19. concurrent library is still reprodu Definition Closed semantics Sclosed of deration s.ics Consideration of of this of implementation thisofdefinitions adjusts definitions ofofsafe and of safe internally and internall its theoremsstates that certain executions one provided that they are related with our not 19.client. ClosedThe semantics JPK(p) of a program P w.r.t. a vi closed n Sclosed JPK(p) = {Z Z 2 Twith JPK ^another 9n, { i }i=1, { still reproducible, if we replace a library Assume: mantics Sclosed JPK(p) Sclosed ofjpk(p) a program of13 a program P(Abstraction). w.r.t. Pa w.r.t. view is: view p is: C(L2 n+1 paif n Theorem program (p) =our {Z notion Z 2 TofJPK ^ 9n, abstraction. { i }i=1, { i }i=1. 1 2 bpc ^ ( 1, 1, with library Z= ), p, p ), safeint(c(l ), p, p ) and L n+1 n+1 n safeint(c(l n 1 c 2 c l l 9n, T JPK { ^ 9n,,2{ {)Corollary,.{ 1i }2i=1 bpc. ^ 2 bpc ^ Z = (,, )(,, )... (, i }C(L i }i=1libraries 1 1 1 2 2 2 3 n n i=1 ogram and L and L are such that 20. If a client C and libraries L 1 2 1 an # # # (, Z =, ( )(,,,, )( ).,.. (,, )..,. (, )}, [ {"} )} [ {"} p20. )If and L v L hold, then: safeint(c(l ), p p ) and L v L hold, then: 1 1 2 1 2 1 2 2 3 2 2 n 3 n n+1 n n n+1 c,zpl= p 2 c 1 p 2 l a client Cl,and L2 )K(p are such l 8(Z X libraries, Y ) 2 LS1 and JC(L, p ).that 9(Z safein,x Abstraction theorem (closed programs) 1 1 1 int 1 c l 2 2 ),Cpcand, pl )libraries and L v L hold, then: 2libraries 1 p 2 L and L L are and such L are that such safeint(c(l that safeint(c(l ), p, p ), ), p, p ) l # # 1 2 1 2 1 c 1 c l l Then: 9Z2 2 Sclosed1JC(L 122SS 1 )K(p c pl ). 9(Z2, X2, Y28Z)client(Z JC(L l ).states(lib(z =2 )K(p client(z )) = c, pl ). intclosed 1 ) JC(L 2 )p^ L2 1hold, vpl Lthen: 2 hold, then: # JC(L# # # JC(L )K(p p ). 9Z 2 S )K(p p ). client(z ) = c 1 c 2 2 c 1 osed l closed l s(lib(z1 )) =Definition states(lib(z214. )) ^ X ^ Ythe Y2of. best axiom 1 = 1 set We sayx2that 2 pl ). Sclosed 9Z2 JC(L 2 Sclosed JC(L ). client(z ) = client(z =2client(Z )^ ^= s 2 2 )K(p c p2l)k(p c pl ).1client(Z 1 ) states(lib(z 21))) Frame property, when the is following holds: of best axioms of the Views Framework closed under states(lib(z states(lib(z 5.1 Proof outline 1 )) = states(lib(z 1 )) = states(lib(z 2 )). 2 )). holds: oof outline P(, p) =) q r 2 SP(, p r). X 8p, q,. q 2 SP(, p) =)