CS165 Computer Security. Understanding low-level program execution Oct 1 st, 2015

Size: px

Start display at page:

Download "CS165 Computer Security. Understanding low-level program execution Oct 1 st, 2015"

Coleen Harper
5 years ago
Views:

1 CS165 Computer Security Understanding low-level program execution Oct 1 st, 2015

2 A computer lets you make more mistakes faster than any invention in human history - with the possible exceptions of handguns and tequila 2

3 What will executing this program do? #include <stdio.h> void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); 42.c 3

4 To answer the question Is this program safe/secure/vulnerable? We need to know What will executing this program do? 4

5 To answer the question Is this program safe/secure/vulnerable? We need to know What will executing this program do? Understanding the compiler and machine semantics are key. 5

6 Agenda Compilation Workflow x86 Execution Model Basic Execution Memory Operation Control Flow Memory Organization 6

7 Agenda Compilation Workflow x86 Execution Model Basic Execution Memory Operation Control Flow Memory Organization NEXT 7

8 void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); 8

9 void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); Compilation

10 void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); Compilation Alice

$void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){$

11 void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); Compilation Alice Alice, the answer is

12 void answer(char *name, int x){ } printf( %s, the answer is: %d\n, name, x); void main(int argc, char *argv[]){ } int x; x = ; answer(argv[1], x); Compilation Alice The compiler and machine determines the semantics Alice, the answer is

13 Compiled Code Input Source Language Compilation Target Language Output Terminology: Target language = machine-readable code = binary instructions 13

14 Interpreted Code Source Language Input Interpretation Output 14

15 Source Language Compilation Target Language 15

16 Source Language Compilation Target Language 4393.c in C 4393 in x86 16

17 Source Language Compilation Target Language 4393.c in C 4393 in x86 Preprocessor (cpp) Compiler (cc1) Assembler (as) Linker (ld) 4393.c

18 Source Preprocessor (cpp) Source Compiler (cc1) Assembler (as) Linker (ld) $ cpp #include <stdio.h> void answer(char *name, int x){ }... printf( %s, the answer is: %d\n, name, x); #include expansion #define substitution 18

19 Preprocessor (cpp) Compiler (cc1) Source Source Assembly Assembler (as) Linker (ld) $ gcc -S #include <stdio.h> void answer(char *name, int x){ }... printf( %s, the answer is: %d\n, name, x); Creates Assembly 19

20 gcc S 4393.c outputs 4393.s _answer: Leh_func_begin1: pushq %rbp Ltmp0: movq %rsp, %rbp Ltmp1: subq $16, %rsp Ltmp2: movl %esi, %eax movq %rdi, -8(%rbp) movl %eax, -12(%rbp) movq -8(%rbp), %rax... 20

21 Preprocessor (cpp) Compiler (cc1) Assembler (as) Source Source Assembly Binary Linker (ld) $ as <options> _answer: Leh_func_begin1: pushq %rbp Ltmp0: movq %rsp, %rbp Ltmp1: subq $16, %rsp Ltmp2: movl %esi, %eax movq %rdi, -8(%rbp) movl %eax, -12(%rbp) movq -8(%rbp), %rax... Creates object code 4393.s 21

22 Preprocessor (cpp) Compiler (cc1) Assembler (as) Linker (ld) Source Source Assembly Binary Binary $ ld <options> o Links with other files and libraries to produce an exe 22

23 Disassembling Today: using objdump (part of binutils) objdump D <exe> If you compile with -g, you will see more information objdump D S 23

24 Binary Code Segment (.text) Data Segment (.data)... The program binary (aka executable) Final executable consists of several segments Text for code written Read-only data for constants such as hello world and globals... 24

25 Binary Code Segment (.text) Data Segment (.data)... The program binary (aka executable) Final executable consists of several segments Text for code written Read-only data for constants such as hello world and globals... $ readelf S <file> 25

26 Machine Instruction Example int t = x+y; addl 8(%ebp),%eax Similar to expression: x += y More precisely: int eax; int *ebp; eax += *(ebp[2]) 0x80483ca: C Code Add two signed integers Assembly Add 2 4-byte integers Long words in GCC parlance Same instruction whether signed or unsigned Operands: x: Register %eax y: Memory M[%ebp+8] t: Register %eax Object Code Return function value in %eax 3-byte instruction Stored at address 0x80483ca 26

27 Agenda Compilation Workflow x86 Execution Model Basic Execution Memory Operation Control Flow Memory Organization NEXT 27

28 Basic Execution Binary Code Data... Processor File system Process Memory 28

29 Basic Execution Binary Code Data... Processor Stack File system Heap Process Memory 29

30 Basic Execution Binary Code Data... Stack Fetch, decode, execute Processor File system Heap Process Memory 30

31 Basic Execution Binary Code Data... Stack Fetch, decode, execute Processor File system Heap Process Memory read and write 31

32 x86 Processor EIP EFLAGS EAX EDX ECX EBX ESP EBP ESI EDI 32

33 x86 Processor EAX EDX ECX EIP EBX EFLAGS ESP EBP ESI EDI Address of next instruction 33

34 x86 Processor EAX EDX ECX EIP EBX EFLAGS ESP EBP ESI EDI Address of next instruction Condition codes 34

35 x86 Processor EAX EDX ECX EIP EBX EFLAGS ESP EBP ESI EDI Address of next instruction Condition codes General Purpose 35

36 Registers have up to 4 addressing modes EAX 1. Lower 8 bits 2. Mid 8 bits 3. Lower 16 bits 4. Full register EDX ECX EBX ESP EBP ESI EDI 36

EAX, EDX, ECX, and EBX EAX EDX ECX EBX AH DH CH BH AL DL CL BL Bit 32 16 15 8 7 0 32 bit registers (three letters) Lower bits (bits 0-7) (two letters with

37 EAX, EDX, ECX, and EBX EAX EDX ECX EBX AH DH CH BH AL DL CL BL Bit bit registers (three letters) Lower bits (bits 0-7) (two letters with L suffix) Mid-bits (bits 8-15) (two letters with H suffix) EAX EDX ECX EBX AX DX CX BX Bit Lower 16 bits (bits 0-15) (2 letters with X suffix) 37

38 ESP, EBP, ESI, and EDI EAX AH AL ESP SP EDX DH DL EBP BP ECX CH CL ESI SI EBX BH BL EDI DI Bit Lower 16 bits (bits 0-15) (2 letters) 38

39 x86 Implementation instruction instruction instruction EIP is incremented after each instruction Instructions are different length EIP modified by CALL, RET, JMP, and cond. JMP data data data

40 x86 Instruction Set Instructions classes: Data Movement: MOV, PUSH, POP, Arithmetic: TEST, SHL, ADD, I/O: IN, OUT, Control: JMP, JZ, JNZ, CALL, RET String: REP, MOVSB, System: IRET, INT, Volume 2A: Instruction Set Reference, A-M Volume 2B: Instruction Set Reference, N-Z Intel syntax: OP DST, SRC AT&T (gcc/gas) syntax: OP SRC, DST 40

41 Basic Ops and AT&T vs Intel Syntax source first destination first Meaning AT&T Intel ebx = eax eax = eax + ebx movl %eax, %ebx mov ebx, eax addl %ebx, %eax add eax, ebx ecx = ecx << 2 shl $2, %ecx shl ecx, 2 AT&T is at odds with assignment order. It is the default for objdump, and traditionally used for UNIX. Intel order mirrors assignment. Windows traditionally uses Intel, as is available via the objdump -M intel command line option 41

42 Agenda Compilation Workflow x86 Execution Model Basic Execution Memory Operation Control Flow Memory Organization NEXT 42

43 x86: Byte Addressable 43

44 x86: Byte Addressable... Address 3 holds 1 byte Address 2 holds 1 byte Address 1 holds 1 byte Address 0 holds 1 byte 44

45 x86: Byte Addressable It s convention:... lower address at the bottom Address 3 holds 1 byte Address 2 holds 1 byte Address 1 holds 1 byte Address 0 holds 1 byte 45

46 x86: Byte Addressable... I can fetch bytes at any address Address 3 holds 1 byte Address 2 holds 1 byte Address 1 holds 1 byte Address 0 holds 1 byte 46

47 x86: Byte Addressable... I can fetch bytes at any address Address 3 holds 1 byte Address 2 holds 1 byte Address 1 holds 1 byte Address 0 holds 1 byte Memory is just like using an array! 47

48 x86: Byte Addressable... I can fetch bytes at any address Address 3 holds 1 byte Address 2 holds 1 byte Address 1 holds 1 byte Address 0 holds 1 byte Memory is just like using an array! Alternative: Word addressable Example: For 32-bit word size, it s valid to fetch 4 bytes from Mem[0], but not Mem[6] since 6 is not a multiple of 4. 48

49 x86: Addressing bytes Addresses are indicated by operands that have a bracket [] or paren (), for Intel vs. AT&T, resp. 0xff 0xee 0xdd 0xcc 0xbb 0xaa 0x00 Addr

50 x86: Addressing bytes Addresses are indicated by operands that have a bracket [] or paren (), for Intel vs. AT&T, resp. What does mov dl, [al] do? 0xff 0xee 0xdd 0xcc Addr 6 Register Value 0xbb eax edx ebx 0x3 0x0 0x5 0xaa 0x

51 x86: Addressing bytes Addresses are indicated by operands that have a bracket [] or paren (), for Intel vs. AT&T, resp. What does mov dl, [al] do? 0xff 0xee 0xdd Addr 6 Register Value Moves 0xcc into dl 0xcc 0xbb eax edx ebx 0x3 0x0 0x5 0xaa 0x

52 x86: Addressing bytes Addresses are indicated by operands that have a bracket [] or paren (), for Intel vs. AT&T, resp. What does mov edx, [eax] do? 0xff 0xee 0xdd 0xcc Addr 6 Register Value 0xbb eax edx ebx 0x3 0xcc 0x5 0xaa 0x

53 x86: Addressing bytes Addresses are indicated by operands that have a bracket [] or paren (), for Intel vs. AT&T, resp. What does mov edx, [eax] do? 0xff 0xee 0xdd 0xcc Addr 6 Register eax edx ebx Value 0x3 0xcc 0x5 Which 4 bytes get moved, and which is the LSB in edx? 0xbb 0xaa 0x

54 Endianess Endianess: Order of individually addressable units Little Endian: Least significant byte first so address a goes in littlest byte (e.g., AL), a+1 in the next (e.g., AH), etc. 0xff 0xee 0xdd 0xcc Addr 6 Register Value 0xbb eax edx ebx 0x3 0xcc 0x5 0xaa 0x

55 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 55 6

56 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 0xcc Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 56 6

57 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 0xdd 0xcc Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 57 6

58 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 0xee 0xdd 0xcc Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 58 6

59 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 0xff 0xee 0xdd 0xcc Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 59 6

60 mov edx, [eax] Register eax edx ebx EDX Value 0x3 0xcc 0x5 0xff 0xee 0xdd 0xcc EDX = 0xffeeddcc! Endianess: Ordering of individually addressable units Little Endian: Least significant byte first... so... address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, and so on. Bit 0 0xff 0xee 0xdd 0xcc 0xbb 0xaa 0x00 Addr

61 mov [eax], ebx Register eax edx ebx EBX Value 0x3 0xcc 0x Bit 0 0xff 0xee 0xdd 0xcc Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 61 6

62 mov [eax], ebx Register eax edx ebx EBX Value 0x3 0xcc 0x Bit 0 0xff 0xee 0xdd 0xcc 05 Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 62 6

63 mov [eax], ebx Register eax edx ebx EBX Value 0x3 0xcc 0x Bit 0 0xff 0xee 0xdd 00 0xcc 05 Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 63 6

64 mov [eax], ebx Register eax edx ebx EBX Value 0x3 0xcc 0x Bit 0 0xff 0xee 00 0xdd 00 0xcc 05 Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 64 6

65 mov [eax], ebx Register eax edx ebx EBX Value 0x3 0xcc 0x Bit 0 0xff 00 0xee 00 0xdd 00 0xcc 05 Addr Endianess: Ordering of individually addressable units 0xbb Little Endian: Least significant byte first... so... 0xaa address a goes in the least significant byte (the littlest bit) a+1 goes into the next byte, 0x00 0 and so on. 65 6

66 There are other ways to address memory than just [register]. These are called Addressing Modes. An Addressing Mode specifies how to calculate the effective memory address of an operand by using information from registers and constants contained with the instruction or elsewhere. 66

67 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) 67

68 Motivation: Addressing Buffers typedef uint32_t addr_t; 0 uint32_t w, x, y, z; 0 uint32_t buf[3] = {1,2,3}; 0 addr_t ptr = (addr_t) buf; w = buf[2]; x = *(buf + 2); buf[2] Memory What is x? what memory cell does it ref? buf 1 68

69 Motivation: Addressing Buffers typedef uint32_t addr_t; 0 uint32_t w, x, y, z; 0 uint32_t buf[3] = {1,2,3}; 0 addr_t ptr = (addr_t) buf; w = buf[2]; x = *(buf + 2); y = *( (uint32_t *) (ptr+8)); buf[2] Memory buf 1 69

70 Motivation: Addressing Buffers typedef uint32_t addr_t; 0 uint32_t w, x, y, z; 0 uint32_t buf[3] = {1,2,3}; 0 addr_t ptr = (addr_t) buf; w = buf[2]; x = *(buf + 2); y = *( (uint32_t *) (ptr+8)); buf[2] Memory 0 0 Equivalent 0 (addr_t) (ptr + 8) = (uint32_t *) buf+2 buf 1 70

71 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) 71

72 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) Say at imm +r 1 72

73 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) Say at imm +r 1 Constant scaling factor s, typically 1, 2, 4, or 8 73

74 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) Say at imm +r 1 Constant scaling factor s, typically 1, 2, 4, or 8 Say in Register r 2 74

75 Motivation: Addressing Buffers Type buf[s]; buf[index] = *(<buf addr>+sizeof(type)*index) Say at imm +r 1 Constant scaling factor s, typically 1, 2, 4, or 8 Say in Register r 2 imm + r 1 + s*r 2 AT&T: imm (r 1, r 2, s) Intel: r 1 + r 2 *s + imm 75

76 AT&T Addressing Modes for Common Codes Form imm (r) imm (r 1, r 2 ) imm (r 1, r 2, s) imm Meaning on memory M M[r + imm] M[r 1 + r 2 + imm] M[r 1 + r 2 *s + imm] M[imm] 76

77 Carnegie Mellon Address Computation Examples %edx %ecx 0xf000 0x0100 Expression Address Computation Address 0x8(%edx) 0xf x8 0xf008 (%edx,%ecx) 0xf x100 0xf100 (%edx,%ecx,4) 0xf *0x100 0xf400 0x80(,%edx,2) 2*0xf x80 0x1e080 77

78 Referencing Memory Loading a value from memory: mov <eax> = *buf; mov -0x38(%ebp),%eax (A) mov eax, [ebp-0x38] (I) <eax> = buf; Loading an address: lea lea -0x38(%ebp),%eax (A) lea eax, [ebp-0x38] (I) 78

79 Suppose I want to access address 0xdeadbeef directly Loads the address lea eax, 0xdeadbeef (I) Deref the address mov eax, 0xdeadbeef (I) Note missing $. This distinguishes the address from the value 79

80 Understanding Swap void swap(int *xp, int *yp) { int t0 = *xp; int t1 = *yp; *xp = t1; *yp = t0; } Register %edx %ecx %ebx %eax Value xp yp t0 t1 Offset yp xp Rtn adr Old %ebp Stack (in memory) %ebp -4 Old %ebx %esp movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 80

81 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 0x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 81

82 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 0x124 0x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 82

83 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 0x124 0x120 0x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 83

84 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 0x124 0x x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 84

85 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 456 0x124 0x x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 85

86 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap x124 0x x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 86

87 %eax %edx %ecx %ebx %esi %edi %esp %ebp Understanding Swap 456 0x124 0x x104 yp xp Offset %ebp x120 0x124 Rtn adr Address 0x124 0x120 0x11c 0x118 0x114 0x110 0x10c 0x108 0x104 0x100 movl 8(%ebp), %edx # edx = xp movl 12(%ebp), %ecx # ecx = yp movl (%edx), %ebx # ebx = *xp (t0) movl (%ecx), %eax # eax = *yp (t1) movl %eax, (%edx) # *xp = t1 movl %ebx, (%ecx) # *yp = t0 87

Turning C into Object Code Code in files p1.c p2.c Compile with command: gcc -O p1.c p2.c -o p Use optimizations (-O) Put resulting binary in file p

Turning C into Object Code Code in files p1.c p2.c Compile with command: gcc -O p1.c p2.c -o p Use optimizations (-O) Put resulting binary in file p text C program (p1.c p2.c) Compiler (gcc -S) text Asm