Load-reserve / Store-conditional on POWER and ARM

Transcription

1 Load-reserve / Store-conditional on POWER and ARM Peter Sewell (slides from Susmit Sarkar) 1 University of Cambridge June 2012

2 Correct implementations of C/C++ on hardware Can it be done?...on highly relaxed hardware? What is involved? Mapping new conructs to assembly Optimizations: which ones legal? Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

3 Correct implementations of C/C++ on hardware Can it be done?...on highly relaxed hardware? e.g. Power What is involved? Mapping new conructs to assembly Optimizations: which ones legal? Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

4 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

5 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c lwsync; lwsync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

6 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c Fence acquire Fence release Fence seq-c lwsync; lwsync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync lwsync lwsync sync (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

7 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c Fence acquire Fence release Fence seq-c lwsync; lwsync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync lwsync lwsync sync CAS relaxed loop: lwarx; cmp; bc exit; wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

8 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c Fence acquire Fence release Fence seq-c lwsync; lwsync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync Is that mapping correct? lwsync lwsync sync CAS relaxed loop: lwarx; cmp; bc exit; wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

9 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c lwsync; lwsync; sync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync Fence acquire Fence release Fence seq-c CAS relaxed lwsync lwsync sync loop: lwarx; cmp; bc exit; Answer: No! wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

10 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c lwsync; sync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync Is that mapping correct? Fence acquire lwsync Fence release lwsync Fence seq-c sync CAS relaxed loop: lwarx; cmp; bc exit; Answer: Yes! wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

11 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c Fence acquire Fence release Fence seq-c CAS relaxed lwsync; sync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync Is that the only correct mapping? lwsync lwsync sync loop: lwarx; cmp; bc exit; Answer: No! wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: (From Paul McKenney and Raul Silvera) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

12 Implementing C/C++11 on POWER: Pointwise Mapping C/C++11 Operation POWER Implementation Store (non-atomic) Load (non-atomic) Store relaxed Store release Store seq-c Load relaxed Load consume Load acquire Load seq-c lwsync; sync; (and preserve dependency) ; cmp; bc; isync sync; ; cmp; bc; isync Alternative sync; ; sync; ; sync Fence acquire Fence release Fence seq-c lwsync lwsync sync CAS relaxed loop: lwarx; cmp; bc exit; wcx.; bc loop; exit: CAS seq-c sync; loop: lwarx; cmp; bc exit; wcx.; bc loop; isync; exit: All compilers mu agree for separate compilation Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

13 Machine Synchronisation Operations x86: atomic synchronization operations, e.g. atomic add, CAS,... RISC-friendly alternative: Load-reserve/Store-conditional (aka LL/SC, larx/cx and lwarx/wcx, LDREX/STREX) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

14 Machine Synchronisation Operations x86: atomic synchronization operations, e.g. atomic add, CAS,... RISC-friendly alternative: Load-reserve/Store-conditional (aka LL/SC, larx/cx and lwarx/wcx, LDREX/STREX) Can be used to implement CAS, atomic add, spinlocks,... Universal (like CAS) [Herlihy 93] (but no ABA problem) Atomic Addition loop: lwarx r, d; add r,v,r; wcx r, d; bne loop; Informally, wcx succeeds only if no other write to the same address since la lwarx, setting a flag iff it succeeds Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

15 What is no write since...? In machine time? Neither necessary, nor sufficient Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

16 What is no write since...? In machine time? Neither necessary, nor sufficient Microarchitecturally (simplified): if cache-line ownership not lo since la lwarx (but we don t want to model the microarchitecture...) Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

17 Modeling not lo since Abractly: ownership chain modeled by buiing up coherence order Coherence: order relating ores to the same location (eventually linear) A wcx succeeds only if it is (or at lea, if it can become) coherence-next-to the write read from by lwarx...and no other write can later come in between Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

18 Modeling not lo since Abractly: ownership chain modeled by buiing up coherence order Coherence: order relating ores to the same location (eventually linear) A wcx succeeds only if it is (or at lea, if it can become) coherence-next-to the write read from by lwarx...and no other write can later come in between Isolate key concept: write reaching coherence point coherence is linear below this write, and no new edges will be added below Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

19 Coherence points and a successful wcx Atomic Addition loop: lwarx r, x; add r,3,r; wcx r, x; bne loop; Coherence order for x: c:w x=4 i:w x=0 j:w x=1 a:w x=2 b:w x=3 Suppose lwarx reads from the a:w x:2 Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

20 Coherence points and a successful wcx Atomic Addition loop: lwarx r, x; add r,3,r; wcx r, x; bne loop; Coherence order for x: c:w x=4 i:w x=0 j:w x=1 a:w x=2 b:w x=3 Suppose lwarx reads from the a:w x:2 wcx can succeed if this becomes possible: writes that have reached coherence point i:wx=0 j:wx=1 a:wx=2 d:w x=5 c:w x=4 b:w x=3 Warning: wcx can fail spuriously Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

21 Load-reserve/ore-conditional and ordering Same-thread load-reserve/ore-conditionals ordered by program order If all memory accesses are l-r/s-c sequences Then: only SC behaviour But... normal loads/ores (to different addresses) not ordered; the l-r/s-c do not act as a barrier Confusion here led to Linux bug... bad barrier placement in atomic-add-return Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

22 Correctness of the Mapping Theorem: For any sane, non-optimising compiler following the mapping: compilation DRF C/C++ prog POWER prog C/C++11 semantics POWER semantics C/C++11 execution observations POWER execution observations Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

23 Correctness of the Mapping Theorem: For any sane, non-optimising compiler following the mapping: C/C++11 semantics DRF C/C++ prog C/C++11 execution observations Preserves memory accesses; compilation Uses the mapping table; Respects the thread local semantics of C/C++, preserving dependencies POWER semantics POWER execution POWER prog observations Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

24 Correctness of the Mapping Theorem: For any sane, non-optimising compiler following the mapping: compilation DRF C/C++ prog C/C++11 semantics C/C++11 execution observations POWER semantics POWER execution POWER prog observations From POWER trace, bui key relations (happens-before, SC order) Required properties from abs. machine properties If trace looks like it produces data race, bui the C/C++ data race Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10

25 For details... see Synchronising C/C++ and POWER, Sarkar et al., PLDI In the paper: A formal model of load-reserve/ore-conditional (in Lem) An executable model with exploration tool (ppcmem) Simplifications to the C/C++11 lock model Models tight again each other: relaxing the Power model wou make C/C++11 unimplementable Peter Sewell (Cambridge) Load-reserve / Store-conditional on POWER and ARM June / 10