Be a Binary Rockst r. An Introduction to Program Analysis with Binary Ninja

Transcription

1 Be a Binary Rockst r An Introduction to Program Analysis with Binary Ninja

2 Agenda Motivation Current State of Program Analysis Design Goals of Binja Program Analysis Building Tools 2

3 Motivation 3

4 Tooling - Concrete -> Symbolic Increase speed & effectiveness of RE / VR Make Program Analysis more accessible & useful 4

5 Foundations Need to understand code semantics Could be done directly on the assembly An Intermediate Language (IL) is needed 5

6 Why IL? Architecture Abstraction Smaller number of instructions 6

7 Easy to lift Simple flags calculation As close to native instructions as possible Typeless - types inferred later 7

8 Easy to read Intuitive to read Tree-based infix notation No register abstraction Flags calculation only when necessary Avoid excessive temporaries 8

9 Easier to analyze Instruction Set Size Easier to lift IL Instruction Set Size 9

10 The Options 10

11 Existing Options for IL BAP VEX REIL LLVM IDA 11

12 BAP Tree-tree based :) Flags are explicit and inhibit readability :( Written in OCAML :( 12

13 add ebx, eax shl ebx, cl addr add %eax,%ebx t:u32 = REBX:u32 REBX:u32 = REBX:u32 + REAX:u32 RCF:bool = REBX:u32 < t:u32 addr shl %cl,%ebx t1:u32 = REBX:u32 >> 0x20:u32 (RECX:u32 & 0x1f:u32) RCF:bool = ((RECX:u32 & 0x1f:u32) = 0:u32) & RCF:bool ((RECX:u32 & 0x1f:u32) = 0:u32) & low:bool(t1:u32) 13

14 VEX Register names are abstracted :( Single assignment :( Over 1000 instructions! :( Yet they call it RISC-like Even Angr is planning a move away from it 14

15 subs R2, R2, #8 t0 = GET:I32(16) t1 = 0x8:I32 t3 = Sub32(t0,t1) PUT(16) = t3 PUT(68) = 0x59FC8:I32 15

16 REIL Tiny instruction set Horrible readability Makes abstractions nearly impossible Flags are explicit and inhibit readability :( 16

17 test eax, eax STR R_EAX:32,, V_00: STR 0:1,, R_CF: AND V_00:32, ff:8, V_01: SHR V_01:8, 7:8, V_02: SHR V_01:8, 6:8, V_03: XOR V_02:8, V_03:8, V_04: SHR V_01:8, 5:8, V_05: SHR V_01:8, 4:8, V_06: XOR V_05:8, V_06:8, V_07: XOR V_04:8, V_07:8, V_08: a SHR V_01:8, 3:8, V_09: b SHR V_01:8, 2:8, V_10: c XOR V_09:8, V_10:8, V_11: d SHR V_01:8, 1:8, V_12: e XOR V_12:8, V_01:8, V_13: f XOR V_11:8, V_13:8, V_14: XOR V_08:8, V_14:8, V_15: AND V_15:8, 1:1, V_16: NOT V_16:1,, R_PF: STR 0:1,, R_AF: EQ V_00:32, 0:32, R_ZF: SHR V_00:32, 1f:32, V_17: AND 1:32, V_17:32, V_18: EQ 1:32, V_18:32, R_SF: STR 0:1,, R_OF:1 17

18 LLVM Easy to analyze and has great tools already available It s a compiler! Reversers want a decompiler. Cannot be the only goal 18

19 LLVM Challenges Hard to lift well from compiled binaries Designed for compiler output Expects type information in the instructions SSA form - assembly is not Stack in assembly looks like a structure, but structures lose many advantages of SSA 19

20 IDA? 20

21 Binary Ninja s Answer Binary Ninja Intermediate Language (BNIL) 21

22 IL Goals & Design 22

23 Why Another IL? Popular existing ILs for compiled binaries are not very human readable. They are extremely low level and verbose. Existing ILs are single stage. Heavyweight analysis must be performed to get anywhere close to decompiled output. Writing a lifter for a new architecture is usually very time consuming. 23

24 Binary Ninja IL Create a family of ILs with multiple stages of analysis Lowest level is close to assembly After analysis and transformations, higher levels are closer to decompiled output and would be much easier to translate to good LLVM code Analysis involved in each transformation is easy to understand, fast, and directly aids further analysis 24

25 IL Design Goals Human readable Computer understandable (SSA, 3AF, etc.) Plugin understandable Easy to lift native architectures Translation to other ILs such as LLVM 25

26 Human Readable Reads like pseudocode, even in lowest level form Flags are resolved into readable expressions 26

27 Low Level IL Example lea lea push sub mov cmp ja rax, [0x201047] rdi, [0x201040] rbp rax, rdi rbp, rsp rax, 0xe 0x68d rax = 0x rdi = 0x push(rbp) rax = rax - rdi rbp = rsp if (rax u> 0xe) then 0x68d else 0x68b x86-64 Assembly Low Level IL 27

28 Low Level IL Example addiu $sp, $sp, -0x18 sw $ra, 0x14($sp) lw $a0, ($a1) jal atoi nop sltiu $at, $v0, 0x20 beqz $at, 0x4002d8 nop MIPS Assembly $sp = $sp - 0x18 [$sp + 0x14].d = $ra $a0 = [$a1].d call(atoi) $at = $v0 u< 0x20? 1 : 0 if ($at == 0) then 0x4002d8 else 0x Low Level IL 28

29 Computer Understandable Multiple IL forms Pick the right IL for the task at hand 29

30 IL Forms Lifted IL ASM -> IL Low Level IL Flags use resolved SSA / 3AF High Level IL Calls in high level form Expression folding Like decompiled output Medium Level IL Stack usage resolved Type propagation SSA / 3AF 30

31 Plugin Understandable All IL forms directly accessible from API Analysis performed on IL also accessible by API 31

32 Easy to Lift Expression tree Designed for quick, modular lifter implementations Semantic flags eases the burden of describing flag effects during lifting 32

33 Semantic Flags Architecture plugins define the set of flags and their semantic roles Instructions can define a set of flags they write Data flow analysis is performed to link flag uses to flag writes 33

34 Semantic Flags In most compiled code, flags are resolved to simple comparison expressions with no effort from the architecture plugin Special cases fall back to emitting concrete flag write expressions 34

35 Semantic Flags Example Writes to all ALU flags sub.q{*}(rax, 0xe) if (u>) then else Folded expression describing use of flags if (rax u> 0xe) then else Flag state representing unsigned greater than 35

36 Translating Upwards Semantic flags analysis gives Low Level IL with flag usage fully resolved Stack is represented as memory accesses, so data flow can be difficult to compute on stack variables in Low Level IL Need to analyze and translate to Medium Level IL 36

37 Low Level IL to Medium Level IL Low Level IL is translated to SSA form Use implicit data flow from SSA to resolve stack layout Data flow based stack layout resolution avoids problems with nonstandard frame pointer behavior Translate loads and stores on stack to stack variable uses and assignments 37

38 Medium Level IL Example push(ebp) ebp = esp esp = esp - 0x18 eax = [ebp + 8].d [esp].d = eax call(free) esp = ebp ebp = pop <return> jump(pop) var_4 = ebp eax = arg_4 var_1c = eax free(var_1c) ebp = var_4 return Medium Level IL 38

39 Medium Level IL Registers and stack usage are now both treated as variables Stack variables no longer use explicit memory access Translate to SSA form to obtain implicit data flow on both registers and stack variables Type propagation is performed on SSA form 39

40 Using Medium Level IL - Jump Tables 40

41 Using Medium Level IL - Jump Tables Jump table resolution based on path-sensitive data flow SSA conversion process also tracks control flow dependence for every block Data flow computations allow disjoint sets of possible values Reads from memory are simulated At jump site, possible values are the possible jump targets 41

42 x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Jump Table Example x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) Solve for this to get jump targets Medium Level IL SSA Form 42

43 x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Jump Table Example x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) Track flow backwards with SSA to find definitions 43

44 x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Jump Table Example x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) Memory read depends on value of x8#1 44

45 Jump Table Example x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Value used in branch comparison x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) 45

46 Jump Table Example x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Branch condition must be false to reach jump site x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) 46

47 x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Jump Table Example When false, we know that x0#2.d is between 0 and 0x1f inclusive x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) 47

48 x8#1 = zx.q(x0#2.d) if (x0#2.d u> 0x1f) then else Jump Table Example Resolve forward to obtain possible jump targets x8#2 = sx.q([table + (x8#1 << 2)].d) x8#3 = x8#2 + table jump(x8#3) Set of possible values here are the jump targets 48

49 Using Medium Level IL - Jump Tables More complex idioms need to combine multiple sources of information Value through SSA ϕ-functions is the set union of the inputs Value of a specific SSA variable is the set intersection of the information found in the definition and all uses of the variable 49

50 Leveraging the Jump Table Algorithm Single jump table algorithm works on all architectures with no additional effort from architecture plugin Control flow dependence information accessible from API Queries for set of possible values accessible from API 50

51 The Final Forms Medium Level IL has the type information, stack knowledge, and SSA form to translate easily into LLVM IR Further analysis can be performed to translate to High Level IL, the Binary Ninja IL that will be used to create its decompiler All aspects of every IL form are plugin accessible, so translating to other representations is straightforward 51

52 Binary Ninja for Profit 52

53 Python, C and C++ API s (headless) Binja API Branches: Basic block/ Function edges (incoming & outgoing) Get the register states, some naive range analysis api.binary.ninja/search.html 53

54 binja_memcpy.py: IL /bin/bash 54

55 binja_memcpy.py: IL /bin/bash 55

56 binja_memcpy.py: API 56

57 binja_memcpy.py: API 57

58 binja_memcpy.py: API 58

59 binja_memcpy.py: API 59

60 binja_memcpy.py: Output 60

61 SSA: Uninitialized variable for func in bv.functions: for block in func.medium_level_il.ssa_form: for instr in block: visit_instr(instr) 61

62 SSA: Uninitialized variable def visit_instr(inst): # Read of variable if inst.operation == MLIL_VAR_SSA: # Not written if inst.index == 0: if inst.src.type == StackVariableSourceType: # Local variables if inst.src.identifier < 0: print ("Uninitialized stack variable reference at " + hex(inst.address)) 62

63 SSA: Uninitialized variable else: for op in instr.operands: if isinstance(op, MediumLevelILInstruction): visit_instr(op) 63

64 Very accurate Symbolic Execution Takes time, data, and memory, often not feasible IDEA! Reasoning only about what we care about. Apply complex data to abstract domains! Domains: type, sign, range, color etc. 64

65 Abstract Interpretation Sets of concrete values are abstracted imprecisely Galois Connection formalizes Concrete <-> Abstract 65

66 Abstract Interpretation int x; int[] a = new int[10]; a[2 * x] = 3; X s value is imprecise Compilers perform imprecise abstraction 1. Add precision - i.e. declare 1. abstract value [0, 9] 2. Symbolically execute with abstract domain/ values Requires control-flow analysis 66

67 Abstract Domains & Sign Analysis int a,b,c; a = 42; b = 87; if (input) { c = a + b; } else { c = a - b; } Map variables to an abstract value 67

68 Abstract Domains & Sign Analysis Binary Ninja plugin Under approximate Path sensitive - construct lattices of abstract values One abstract state per CFG node Avoid loss in precision for fractions. 68

69 Demo! Analyze example program PHP CVE

70 UAF Analysis: PointsTo for Binja IL Before: Allocation -> Write UAF Analysis: Allocation -> Free -> Use Key Idea: Data flow graph, assignments, copies, dereferences, and frees of pointers Context and path sensitive (path API == soon!). blog.trailofbits.com/2016/03/09/the-problem-with-dynamic-program-analysis/ 70

71 Devirtualizing C++ VTable Function Call Example: mov eax, [ecx]; call [eax + 4] 71

72 Devirtualizing C++ 72

73 Devirtualizing C++ 73

74 Playing with Scripts! memcpy, headless python API script Depth-first-search, path sensitive CFG template trailofbits/binjascripts Sign analysis, abstract domain plugin, CFG traversal script And much much more. 74

75 Conclusion: Resources binary.ninja/ Abstract Interpretation talk: santos.cs.ksu.edu/schmidt/escuela03/wssa/talk1p.pdf Static Program Analysis Book! cs.au.dk/~amoeller/spa/spa.pdf 75

76 Conclusion: Binary Ninja 76

77 Contact Us Sophia d Antoine - - sophia@trailofbits.com Peter LaFosse Rusty Wagner binaryninja@vector35.com 77