On the C11 Memory Model and a Program Logic for C11 Concurrency

Transcription

1 On the C11 Memory Model and a Program Logic for C11 Concurrency Manuel Penschuck Aktuelle Themen der Softwaretechnologie Software Engineering & Programmiersprachen FB 12 Goethe Universität - Frankfurt am Main 04. Feb Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

2 Topics 1 Motivation 2 C11 Memory Model Atomic Accesses 3 Logic The Hoare Triple Separation Logic Relaxed Separation Logic 4 Conclusion Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

3 Common Weakness Enumeration 1 TOP 25 Most Dangerous Software Errors 41 CWE-456: Missing Initialisation 40 CWE-825: Expired Pointer Dereference 36 CWE-476: NULL Pointer Dereference 33 CWE-362: Concurrent Execution using Shared Resource with Improper Synchronisation Race Condition int init = 1; int uninit; int result = init + uninit; 1 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

4 Common Weakness Enumeration 1 TOP 25 Most Dangerous Software Errors 41 CWE-456: Missing Initialisation 40 CWE-825: Expired Pointer Dereference 36 CWE-476: NULL Pointer Dereference 33 CWE-362: Concurrent Execution using Shared Resource with Improper Synchronisation Race Condition freep; *p = 47; 1 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

5 Common Weakness Enumeration 1 TOP 25 Most Dangerous Software Errors 41 CWE-456: Missing Initialisation 40 CWE-825: Expired Pointer Dereference 36 CWE-476: NULL Pointer Dereference 33 CWE-362: Concurrent Execution using Shared Resource with Improper Synchronisation Race Condition p = mallocsizeofint; // out of memory! *p = 42; 1 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

6 Common Weakness Enumeration 1 TOP 25 Most Dangerous Software Errors 41 CWE-456: Missing Initialisation 40 CWE-825: Expired Pointer Dereference 36 CWE-476: NULL Pointer Dereference 33 CWE-362: Concurrent Execution using Shared Resource with Improper Synchronisation Race Condition tmp = shared; tmp = tmp&1? 3*tmp +1 : tmp>>1; shared = tmp; 1 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

7 Hidden Races int data = 0; void worker forint i=0; i < N; i++ data++; } } int main.. pardo [worker worker] and join; printf"data=%d", data; } Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

8 Hidden Races void worker // gcc -O0 forint i=0; i < N; i++ register int tmp = data; // RAM: expensive tmp++; data = tmp; // RAM: expensive } } void worker // gcc -O1 register int tmp = data; forint i=0; i < N; i++ tmp++; data = tmp; } void worker // gcc -O3 register int tmp = data; tmp += N; data = tmp; } Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

9 Situation until now C99/C0X standard: purely sequential Thread and synchronisation support via libraries PThread, TBB, MS Windows Runtime Lib, C++/BOOST,... Limited compiler support Optimisation unsound in concurrent programs? Synchronisation inefficient sync in GCC Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

10 Situation until now C99/C0X standard: purely sequential Thread and synchronisation support via libraries PThread, TBB, MS Windows Runtime Lib, C++/BOOST,... Limited compiler support Optimisation unsound in concurrent programs? Synchronisation inefficient sync in GCC Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

11 Situation until now C99/C0X standard: purely sequential Thread and synchronisation support via libraries PThread, TBB, MS Windows Runtime Lib, C++/BOOST,... Limited compiler support Optimisation unsound in concurrent programs? Synchronisation inefficient sync in GCC Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

12 Some Extensions of C11 Optional concurrency mode Threads via threads.h similar to POSIX threads thread management, mutex, barrier Atomic Memory Access via stdatomic.h Memory Model! Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

13 Some Extensions of C11 Optional concurrency mode Threads via threads.h similar to POSIX threads thread management, mutex, barrier Atomic Memory Access via stdatomic.h Memory Model! Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

14 Some Extensions of C11 Optional concurrency mode Threads via threads.h similar to POSIX threads thread management, mutex, barrier Atomic Memory Access via stdatomic.h Memory Model! Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

15 Some Extensions of C11 Optional concurrency mode Threads via threads.h similar to POSIX threads thread management, mutex, barrier Atomic Memory Access via stdatomic.h Memory Model! Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

16 C11 Memory Model Extends existing evaluation rules Allow compiler and processor as many optimisations as possible Support for Sequential consistency Acquire-Release Consume-Release Relaxed just atomic Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

17 C11 Memory Model Extends existing evaluation rules Allow compiler and processor as many optimisations as possible Support for Sequential consistency Acquire-Release Consume-Release Relaxed just atomic Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

18 C11 Memory Model Extends existing evaluation rules Allow compiler and processor as many optimisations as possible Support for Sequential consistency Acquire-Release Consume-Release Relaxed just atomic Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

19 Acquire-Release All effects of instructions preceding a release R have be observable before R All reads succeeding an acquire A have to happen after A Very efficient on x86 plain reads/writes load store Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

20 Acquire-Release All effects of instructions preceding a release R have be observable before R All reads succeeding an acquire A have to happen after A Very efficient on x86 plain reads/writes [v]na 47 [f ] rel 1 v alloc; r alloc; f alloc; [f ] rlx 0; repeat [f ]acq end r [v] na Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

21 Acquire-Release All effects of instructions preceding a release R have be observable before R All reads succeeding an acquire A have to happen after A Very efficient on x86 plain reads/writes [v]na 47 [f ] rel 1 v alloc; r alloc; f alloc; [f ] rlx 0; repeat [f ]acq end r [v] na Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

22 stdatomic.h atomic_load, -store atomic_fetch_add, -sub, -or, -xor, -and atomic_exchange atomic_compare_exchange_weak, -strong atomic_flag_clear, -test_and_set... Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

23 stdatomic.h atomic_load, -store atomic_fetch_add, -sub, -or, -xor, -and atomic_exchange atomic_compare_exchange_weak, -strong atomic_flag_clear, -test_and_set... forint i=0; i<n; i++ // data++; atomic_fetch_add&data, 1; } Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

24 stdatomic.h atomic_load, -store atomic_fetch_add, -sub, -or, -xor, -and atomic_exchange atomic_compare_exchange_weak, -strong atomic_flag_clear, -test_and_set... atomic_exchange*p, val{ old = *p; *p = val; return old; } Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

25 stdatomic.h atomic_load, -store atomic_fetch_add, -sub, -or, -xor, -and atomic_exchange atomic_compare_exchange_weak, -strong atomic_flag_clear, -test_and_set... atomic_compare_exchange*p,ref,val{ old = *p; if old == ref *p = val; return true; } return false; } spinlock*p, threadid whileat_com_xchg_weak p, 0, threadid; } Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

26 stdatomic.h atomic_load, -store atomic_fetch_add, -sub, -or, -xor, -and atomic_exchange atomic_compare_exchange_weak, -strong atomic_flag_clear, -test_and_set... Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

27 Program Logics Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

28 The Hoare Triple P Q R If the assertion P is true before [the] initiation of a program Q, then the assertion R will be true on its completion [if the program terminates]. x 1 x x + 1 x 2 x 1 while true; x 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

29 The Hoare Triple P Q R If the assertion P is true before [the] initiation of a program Q, then the assertion R will be true on its completion [if the program terminates]. x 1 x x + 1 x 2 x 1 while true; x 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

30 The Hoare Triple P Q R If the assertion P is true before [the] initiation of a program Q, then the assertion R will be true on its completion [if the program terminates]. x 1 x x + 1 x 2 x 1 while true; x 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

31 The Hoare Triple P Q R If the assertion P is true before [the] initiation of a program Q, then the assertion R will be true on its completion [if the program terminates]. x 1 x x + 1 x 2 x 1 while true; x 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

32 Hoare Logic: Rule of Composition P Q1 R R Q2 T P Q1 ; Q 2 T x 1 x x + 1 x 2 x 2 y x x 2 y 2 x 1 x x + 1; y x x 2 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

33 Hoare Logic: Rule of Composition P Q1 R R Q2 T P Q1 ; Q 2 T x 1 x x + 1 x 2 x 2 y x x 2 y 2 x 1 x x + 1; y x x 2 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

34 Separation Logic Targets shared data structures pointers, references Efficient fully-automated verifier Distinguishes between Registers represented by the Stack s : Vars Values with doms < Addressable Memory represented by the Heap h : Addresses Values with domh < Separates independent code sections using the separation conjunction Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

35 Separation Logic Targets shared data structures pointers, references Efficient fully-automated verifier Distinguishes between Registers represented by the Stack s : Vars Values with doms < Addressable Memory represented by the Heap h : Addresses Values with domh < Separates independent code sections using the separation conjunction Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

36 Separation Logic Targets shared data structures pointers, references Efficient fully-automated verifier Distinguishes between Registers represented by the Stack s : Vars Values with doms < Addressable Memory represented by the Heap h : Addresses Values with domh < Separates independent code sections using the separation conjunction Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

37 Separation Logic Targets shared data structures pointers, references Efficient fully-automated verifier Distinguishes between Registers represented by the Stack s : Vars Values with doms < Addressable Memory represented by the Heap h : Addresses Values with domh < Separates independent code sections using the separation conjunction Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

38 Separation Logic: Framing Rule p Q1 q p r Q1 q r x y x 2 y 2 [x] 3 x 3 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

39 Separation Logic: Framing Rule p Q1 q p r Q1 q r x y x 2 y 2 [x] 3 x 3 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

40 Separation Logic: Framing Rule p Q1 q p r Q1 q r x y x 2 y 2 [x] 3 x 3 y 2 x = y x 2 y 2 [x] 3 x 3 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

41 Separation Logic: Framing Rule p Q1 q p r Q1 q r x y x 2 y 2 [x] 3 x 3 y 2 x = y x 2 y 2 [x] 3 x 3 y 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

42 Relaxed Separation Logic A complete C-language subset Rules to infer a Hoare-Triple annotation Central Theorem Coq-proven: If annotation is successful program is memory sound Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

43 Relaxed Separation Logic P, Q := complete first order logic {}}{ false P k Q x. P known from SL {}}{ emp e k e P Q Rele, Q Acqe, Q RMWAcqe, Q Inite Uninite Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

44 Relaxed Separation Logic P, Q := complete first order logic {}}{ false P k Q x. P known from SL {}}{ emp e k e P Q Rele, Q Acqe, Q RMWAcqe, Q Inite Uninite Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

45 Relaxed Separation Logic P, Q := complete first order logic {}}{ false P k Q x. P known from SL {}}{ emp e k e P Q Rele, Q Acqe, Q RMWAcqe, Q Inite Uninite Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

46 Relaxed Separation Logic [v]na 47 [f ] rel 1 emp v alloc; r alloc; f alloc; [f ] rlx 0; repeat [f ]acq end [r] na [v] na Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

47 Relaxed Separation Logic [v]na 47 [f ] rel 1 emp v alloc; r alloc; Uninitv Uninitr f alloc; [f ] rlx 0; repeat [f ]acq end [r] na [v] na Non-Atomic Allocation emp alloc x. Uninitx Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

48 Relaxed Separation Logic v alloc; r alloc; Uninitv Uninitr f alloc; Uninitv Uninitr Relf, Q Acqf, Q [v]na 47 [f ] rel 1 [f ] rlx 0; repeat [f ]acq end [r] na [v] na Atomic Allocation emp alloc x. Relx, Q Acqx, Q Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

49 Relaxed Separation Logic v alloc; r alloc; f alloc; Uninitv Uninitr Relf, Q Acqf, Q [f ] rlx 0; Uninitv Uninitr Relf, Q Acqf, Q Initf [v]na 47 repeat [f ]acq end [f ] rel 1 [r] na [v] na Relaxed Write simplified Rel.. [l] v Initl Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

50 Relaxed Separation Logic v alloc; r alloc; f alloc; [f ] rlx 0; Uninitv Uninitr Relf, Q Acqf, Q Initf Uninitv Relf, Q Uninitr Acqf, Q Initf [v] na 47 repeat [f ] acq end [f ] rel 1 [r] na [v] na Parallel Composition P1 Q1 R1 P2 Q2 R2 P1 P 2 Q1 Q 2 R1 R 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

51 Relaxed Separation Logic Uninitv Relf, Q [v] na 47 v 47 Relf, Q [f ] rel 1 v alloc; r alloc; f alloc; [f ] rlx 0; Uninitr Acqf, Q Initf repeat [f ] acq end [r] na [v] na Non-Atomic Write 1 l _ Uninitl [l]na v 1 l v Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

52 Relaxed Separation Logic [v] na 47 v} {{ 47} Relf, Q Q [f ] rel 1 Relf, Q v alloc; r alloc; f alloc; [f ] rlx 0; Uninitr Acqf, Q Initf repeat [f ] acq end [r] na [v] na Atomic Write Qv Rell, Q [l]rel v Initl Rell, Q Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

53 Relaxed Separation Logic [v] na 47 [f ] rel 1 Relf, Q v alloc; r alloc; f alloc; [f ] rlx 0; Uninitr Acqf, Q Initf repeat [f ] acq end v 1 47 Uninitr true }{{} Q [r] na [v] na Atomic Read simplified Initl Acql, Q [l]acq v.qv Acq... Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

54 Relaxed Separation Logic [v] na 47 [f ] rel 1 v alloc; r alloc; f alloc; [f ] rlx 0; repeat [f ] acq end v } {{ 1 47} Uninitr true Q [r] na [v] na true r 47 v 47 Non-Atomic Read k k l v [l]na x.x = v l v Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

55 Relaxed Separation Logic [v] na 47 [f ] rel 1 Relf, Q v alloc; r alloc; f alloc; [f ] rlx 0; repeat [f ] acq end [r] na [v] na true r 47 v 47 true r 47 Parallel Composition P1 Q1 R1 P2 Q2 R2 P1 P 2 Q1 Q 2 R1 R 2 Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

56 Conclusion & Outlook C11 is now officially fit for concurrent programs RSL shows how to perform formal arguments on subset of C11-MM Fully automated verifier? Thank you for your attention! Questions? Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19

57 Conclusion & Outlook C11 is now officially fit for concurrent programs RSL shows how to perform formal arguments on subset of C11-MM Fully automated verifier? Thank you for your attention! Questions? Uni Frankfurt FB 12 C11-MM / Relaxed Separation Logic 04. Feb / 19