Undefinedness and Non-determinism in C

Transcription

1 1 Undefinedness and Non-determinism in C Nabil M. Al-Rousan Nov. 21, UBC Based on slides from Robbert Krebbers Aarhus University, Denmark

2 2 What is this program supposed to do? The C quiz, question 1 int main() { int x; int y = (x = 3) + (x = 4); printf("x=%d,y=%d\n", x, y); }

3 2 What is this program supposed to do? The C quiz, question 1 int main() { int x; int y = (x = 3) + (x = 4); printf("x=%d,y=%d\n", x, y); } Let us try some compilers Clang prints x=4,y=7, seems just left-right

4 2 What is this program supposed to do? The C quiz, question 1 int main() { int x; int y = (x = 3) + (x = 4); printf("x=%d,y=%d\n", x, y); } Let us try some compilers Clang prints x=4,y=7, seems just left-right GCC prints x=4,y=8, does not correspond to any order

5 2 What is this program supposed to do? The C quiz, question 1 int main() { int x; int y = (x = 3) + (x = 4); printf("x=%d,y=%d\n", x, y); } Let us try some compilers Clang prints x=4,y=7, seems just left-right GCC prints x=4,y=8, does not correspond to any order This program violates the sequence point restriction due to two unsequenced writes to x resulting in undefined behavior thus both compilers are right

6 3 Underspecification in C11 Unspecified behavior: two or more behaviors are allowed For example: order of evaluation in expressions (+57 more) Implementation defined behavior: like unspecified behavior, but the compiler has to document its choice For example: size and endianness of integers (+118 more) Undefined behavior: the standard imposes no requirements at all, the program is even allowed to crash For example: dereferencing a NULL or dangling pointer, signed integer overflow,... (+201 more)

7 3 Underspecification in C11 Unspecified behavior: two or more behaviors are allowed For example: order of evaluation in expressions Non-determinism Implementation defined behavior: like unspecified behavior, but the compiler has to document its choice For example: size and endianness of integers Parametrization (+57 more) (+118 more) Undefined behavior: the standard imposes no requirements at all, the program is even allowed to crash For example: dereferencing a NULL or dangling pointer, signed integer overflow,... (+201 more) No semantics/crash state

8 4 Why does C use underspecification that heavily? Pros for optimizing compilers: More optimizations are possible High run-time efficiency Easy to support multiple architectures

9 4 Why does C use underspecification that heavily? Pros for optimizing compilers: More optimizations are possible High run-time efficiency Easy to support multiple architectures Cons for programmers/formal methods people: Portability and maintenance problems Hard to capture precisely in a semantics Hard to formally reason about

10 Approaches to underspecification CompCert (Leroy et al.) / VST (Appel et al.) Main goal: verification of/w.r.t. CompCert compiler in Coq Semantics only needs to be correct for CompCert compiler For example: integer overflow and aliasing violations not UB KCC (Ellison & Rosu, Hathhorn et al.) Main goal: compiler independent C11 semantics in K Describes most unspecified and undefined behavior No proof assistant support CH 2O (Krebbers & Wiedijk) Main goal: compiler independent C11 semantics in Coq Describes all unspecified and undefined behavior Describes some implementation-defined behavior For example: no legacy architectures with 1s complement Cerberus (Sewell et al.) Main goal: defacto C11 semantics in LEM Improve standard to match the way C is used in practice 5

11 15 Strict-aliasing What is aliasing? Aliasing: multiple pointers referring to the same object int x, *p = &x, *q = &x; // p and q are aliased

12 15 Strict-aliasing What is aliasing? Aliasing: multiple pointers referring to the same object int x, *p = &x, *q = &x; // p and q are aliased Tricky with functions: int f(int *p, int *q) { int x = *q; *p = 17; return x; } If p and q alias, the original value n of *p is returned n p q

13 15 Strict-aliasing What is aliasing? Aliasing: multiple pointers referring to the same object int x, *p = &x, *q = &x; // p and q are aliased Tricky with functions: int f(int *p, int *q) { int x = *q; *p = 17; return x *q; } If p and q alias, the original value n of *p is returned n p q Eliminating x is unsound: 17 would be returned

14 16 Strict-aliasing Alias analysis Alias analysis: to determine whether pointers can alias

15 16 Strict-aliasing Alias analysis Alias analysis: to determine whether pointers can alias Consider a similar function: short g(int *p, short *q) { short x = *q; *p = 17; return x; }

16 16 Strict-aliasing Alias analysis Alias analysis: to determine whether pointers can alias Consider a similar function: short g(int *p, short *q) { short x = *q; *p = 17; return x; } And call it with aliased pointers: union { int x; short y; } u; u.y = 3; g(&u.x, &u.y); &u.x y x &u.y

17 16 Strict-aliasing Alias analysis Alias analysis: to determine whether pointers can alias Consider a similar function: short g(int *p, short *q) { short x = *q; *p = 17; return x; } And call it with aliased pointers: union { int x; short y; } u; u.y = 3; g(&u.x, &u.y); &u.x y x &u.y C99/C11 allow type-based alias analysis: reads/writes with the wrong type cause undefined behavior = A compiler can assume that p and q do not alias

18 17 Strict-aliasing How to treat pointers Others (e.g. CompCert) Memory: a finite map of objects which consist of arrays of bytes Our approach Pointers: pairs (o, i) where o identifies the object, and i the offset into that object Too little information to formalize strict-aliasing

19 17 Strict-aliasing How to treat pointers Others (e.g. CompCert) Memory: a finite map of objects which consist of arrays of bytes Our approach A finite map of objects which consist of well-typed trees of bits Pointers: pairs (o, i) where o identifies the object, and i the offset into that object Too little information to formalize strict-aliasing

20 17 Strict-aliasing How to treat pointers Others (e.g. CompCert) Memory: a finite map of objects which consist of arrays of bytes Our approach A finite map of objects which consist of well-typed trees of bits Pointers: pairs (o, i) where o identifies the object, and i the offset into that object Pairs (o, r) where o identifies the object, and r the path through the tree in that object Too little information to formalize strict-aliasing

21 17 Strict-aliasing How to treat pointers Others (e.g. CompCert) Memory: a finite map of objects which consist of arrays of bytes Our approach A finite map of objects which consist of well-typed trees of bits Pointers: pairs (o, i) where o identifies the object, and i the offset into that object Pairs (o, r) where o identifies the object, and r the path through the tree in that object Too little information to formalize strict-aliasing A semantics for strict-aliasing

22 Strict-aliasing Example of the memory as a structured forest Consider: struct S { union U { signed char x[2]; int y; } u; void *p; } s = { {.x = {33,34} }, s.u.x + 2 } The object in memory looks like: o s.0 signed char: p = (o s : struct S, struct S 0 any : (ptr p) 0 (ptr p) 1... (ptr p) 31 union U 0 signed char[2] 0, 16) signed char> void 18

23 Problem: Type-Punning

24 19 Strict-aliasing Theorem (Strict-aliasing) Given: addresses Γ, a 1 : σ 1 and Γ, a 2 : σ 2 with annotations that do not allow type-punning σ 1, σ 2 unsigned char σ 1 not a subtype of σ 2 and vice versa Then there are two possibilities: 1. a 1 and a 2 do not alias 2. accessing a 1 after a 2 (and vice versa) is undefined

25 19 Strict-aliasing Theorem (Strict-aliasing) Given: addresses Γ, a 1 : σ 1 and Γ, a 2 : σ 2 with annotations that do not allow type-punning σ 1, σ 2 unsigned char σ 1 not a subtype of σ 2 and vice versa Then there are two possibilities: 1. a 1 and a 2 do not alias 2. accessing a 1 after a 2 (and vice versa) is undefined Corollary Compilers can perform type-based alias analysis

26 20 CH 2 O abstract C x string := Set of strings k cintrank ::= char short int long long long ptr si signedness ::= signed unsigned τ i cinttype ::= si? k τ ctype ::= void def x τ i τ τ # x»? τ τ[e] struct x union x enum x typeof e e cexpr ::= x const τi z string z sizeof τ alignof τ offsetof τ x τ min τ max τ bits &e e e. x e 1 α e 2 e( e) alloc τ e free e u e e 1 e 2 (τ)i e 1 && e 2 e 1 e 2 (e 1, e 2) e 1? e 2 : e 3 r crefseg ::= [e].x I cinit ::= e { # #»» r := I } sto cstorage ::= static extern auto s cstmt ::= e return e? goto x x : s break continue {s} sto #» τ x := I? ; s typedef x := τ ; s skip s 1 ; s 2 while(e) s do s while(e) for(e 1 ; e 2 ; e 3) s if (e) s 1 else s 2 d decl ::= struct τ #» x union τ #» x enum x # := e»? : τ i typedef τ global I? : sto #» τ fun s : sto #» τ Θ decls := list (string decl)

27 Problem: End-of-Array Pointers

28 Problem: Byte-Level Operations and Object Representation

29 Problem: Padding of Structs and Unions

30 Problem: Indeterminate Memory and Pointers

31 Conclusion Formal methods can be applied to real programming languages It is hard to write semantics to behaviors that were previously undefined