T Reverse Engineering Malware: Static Analysis I

Transcription

1 T Reverse Engineering Malware: Static Analysis I Antti Tikkanen, F-Secure Corporation Protecting the irreplaceable f-secure.com

2 Representing Data 2

3 Binary Numbers Nibble B Byte B D Word B D 3 9 3

4 Endianness aka. Byte Order c Actual bytes 0x034133c9 Little Endian 0xc Big Endian Intel x86 Intel 8051 PowerPC (exc. G5) Sparc (exc. v9) 4

5 Strings ASCII Unicode (UCS-2) H e l l o C 6C 6F (BOM) H e l l o (ff fe) C 00 6C 00 6F 00 5

6 How do you know when the string ends? ASCIIZ H e l l o C 6C 6F 00 Null-terminated Unicode Pascal Delphi H e l l o C 00 6C 00 6F H e l l o C 6C 6F H e l l o C 6C 6F 6

7 Representing Code 7

8 CPU Architectures CISC Complex Instruction Set Computing Emphasis on hardware and assembly programming Complex multi-clock instructions High code density Examples: x86, m68k, IBM z/architecture RISC Reduced Instruction Set Computing Emphasis on software and high-level languages Simple, reduced instruction set Low code density Examples: ARM, PowerPC, MIPS 8

9 Introduction to Intel x86 Started with 8086 in 1978 Continued with 8088, 80186, 80286, 386, 486, Pentium, CISC architecture 32-bit is called x86-32 or IA bit is called x86-64, IA-32e, AMD64, EM64T, x64 or Intel 64 IA-64 is Itanium, which is different 9

10 Intel Introduced in 1986 Has a 32-bit word length Has 8 general-purpose registers Supports paging and virtual memory Addresses up to 4GiB of memory 10

11 Data Register Layout 11

12 Data Registers AL / AH / AX EAX BL / BH / BX EBX CL / CH / CX ECX DL / DH / DX EDX Accumulator Data index Loop counter Data register Arithmetic operations General data storage, index Loop constructs Arithmetics 12

13 Address Registers IP / EIP Instruction Pointer Program execution SP / ESP Stack Pointer Stack operation BP / EBP Base Pointer Stack frame SI / ESI DI / EDI Source Index Destination Index String operation String operation 13

14 Segment Registers CS Code Segment Program code DS Data Segment Program data ES / FS / GS Other Segments Other uses 14

15 EFLAGS 15

16 Examples of Mnemonics MOV EAX, 1 Move 1 to EAX ADD EDX, 5 Add 5 to EDX SUB EBX, 2 Subtract 2 from EBX AND ECX, 0 Bit-wise AND 0 to ECX XOR EDX, 4 Bit-wise exclusive OR 4 to EDX SHL ECX, 6 Shift ECX left by six ROR EBX, 3 Bit-wise rotate EBX right by 3 INC ECX Increment ECX 16

17 Examples of Mnemonics (contd) JNZ label JMP label CALL func RET LOOP label PUSH EAX POP EDI Jump if not zero (equal) Unconditional jump to label Call function Return from function ECX--, Jump to label if not zero Push EAX to stack Pop EDI from stack 17

18 Intel x86 Instruction Format (32-bit) Variable-size complex instruction format (CISC) Instruction format: OPERATION, [OPERAND 1, OPERAND 2, [OPERAND 3]] Operands can operate on register, memory address or immediate data Single instruction can be of any size from 1 byte up to 15 bytes (compare to RISC, where instruction size is constant, for example 4 bytes)!! 18

19 Instruction Format (64-bit) 64-bit mode is backward-compatible extension to 32-bit mode Adds optional REX prefix to 32-bit format Image Copyright Intel Corporation 19

20 Instruction Prefix Bytes Group 1: lock and repeat prefixes Lock ensures exclusive use of memory in multiple-processor environments Repeat is used in string operations (sort of a loop) Group 2: Segment overrides or branch hint Segment in memory access is implied by the instruction, but can be overridden using Group 2 prefix. Some bytes in this group indicates a branch hint, if the instruction is conditional jump Group 3: Operand size override Switch between 16 and 32-bit operand size Group 4: Address size override Switch between 16 and 32-bit addressing 20

21 REX Prefix Semantics for accessing extra registers in 64-bit mode Additional 64-bit operand size overrides 21

22 Opcode Bytes Byte(s) that presents the actual operation (MOV, INC, PUSH, etc.) Originally only single byte, but later specifications define 1-3 bytes (or 4, depending on definition) First byte can be any opcode in range 0-255, or escape byte (0F) indicating that another opcode byte follows Second opcode byte defines an opcode after escape byte Third opcode byte is a prefix (66, F2 or F3) to multimedia (SSE) instructions Latest instruction sets define escape bytes in the second opcode table (38 and 3A) Some opcodes imply the registers used, others need more information (MODRM/SIB) About 1200 different opcodes (!!!!) 22

23 MODRM and SIB These bytes define the registers and other data used by the instruction MODRM defines the registers used for addressing the memory or register Sometimes addressing is more complex than a simple register can offer SIB byte is used for complex memory addressing: indexing and scaling Example of complex addressing: MOV EBX, [EBP + EAX*2 + 4] Destination Register Base register Index register Scale factor Displacement offset 23

24 Displacement and Immediate Data Some complex addressing forms require offset within the memory reference, called displacement Displacement follows immediately the MODRM/SIB bytes (1, 2 or 4 bytes) Some instructions use immediate data as operand value (1, 2 or 4 bytes) Some instructions can use 64-bit (8 bytes) displacement and immediate values in 64-bit mode 24

25 Monster Instruction Assembly presentation: REP MOV [SS:EAX + EBX*8 + 0x ], 0x Data in hex: F3 36 C7 84 D F3 repetition prefix (REP) 36 segment override (SS) C7 opcode (MOV) 84, D8 MORDM, SIB (base register EAX, index register EBX, scale factor 8) displacement data (0x ) immediate data (0x ) 25

26 Example of RISC: ARM Fixed instruction size: 4 bytes (2 bytes in Thumb mode) Load/store architecture: operations only on registers But including fancy things like embedded shifting a += (j << 2) in one operation! Almost all instructions are conditional (mov, moveq, movne,..) Limited (but quite expressive) instruction set 26

27 Disassemblers x50 PUSH EAX Bits in memory Hexadecimal presentation Human-readable disassembly 8B 5C MOV EBX, [EBP+EAX*2+4] 27

28 Disassemblers: Linear Sweep Linear sweep disassembler: Take first byte of code, decode instruction, use length of instruction to skip to next instruction Example of problem with data embedded in code: 0042ABFE: EB 01 jmp 0042AC AC00: F3 BB rep mov ebx, h 0042AC06: 5F pop edi 28

29 Disassemblers: Recursive Traversal Recursive traversal disassembler: Starting from entry point and follow each branch (depth or breadth-first) to locate new starting points Still problems, e.g. with dynamically calculated branch targets: 29

30 Reversing C 30

31 Reversing C 31

32 Basic Data Types char short int long float double - 1 byte - 2 bytes - 4 bytes (platform word) - 4 bytes - 4 bytes floating point - 8 bytes floating point 32

33 Arrays One-dimensional arrays: char a[4] A[0] A[1] A[2] A[3] Multidimensional arrays char a[2][3]; A[0][0] A[0][1] A[0][2] A[1][0] A[1][1] A[1][2] 33

34 Example of multidimensional arrays char val[2][3][2] = {{{'0','1'},{'2','3'},{'4','5'}}, {{'6','7'},{'8','9'},{'a','b'}}}; 34

35 Structures and Unions struct { unsigned int id; unsigned short age; char name[16]; } record; Memory allocated for all members combined: sizeof(record) = 24 union foo { int one; char two; }; Memory allocated for largest member only: sizeof(record) = 4 35

36 Structure Alignment Data structures are aligned to word size #pragma pack() directive can change it unsigned int id (4 bytes) short age (2 bytes) 2 bytes pad name (16 bytes) #pragma pack(1) unsigned int id (4 bytes) short age (2 bytes) name (16 bytes) 36

37 Calling Conventions: stdcall int stdcall foobar(int x, int y, char *foo, char *bar); 37

38 Calling Conventions: cdecl int cdecl foobar(int x, int y, char *foo, char *bar); 38

39 Calling Conventions: fastcall int fastcall foobar(int x, int y, char *foo, char *bar); 39

40 Calling Conventions: thiscall class Foo { public: int Bar(int x, int y, char *foo, char *bar); }; 40

41 PE/COFF 41

42 Introduction to PE/COFF PE stands for Portable Executable Microsoft introduced PE in Windows NT 3.1 It originates from Unix COFF Features dynamic linking, symbol exporting/importing Can contain Intel, Alpha, MIPS and even.net MSIL binary code 64-bit version is called PE32+ 42

43 Complete Structure of PE 43

44 MZ Header MZ Header PE Header.text.data Imports Resources 44

45 PE Header MZ Header File Header Optional Header Section Header[] 45

46 PE Header (continued) MZ Header File Header Optional Header Section Header[] 46

47 PE Header (continued) MZ Header File Header Optional Header Section Header[] 47

48 PE Header (continued) MZ Header File Header Optional Header Section Header[] 48

49 PE Loading File on disk MZ/PE Header Section Table Image in memory MZ/PE Header Section Table.text.data.text RVA = Relative Virtual Address = Offset from image base in memory.data 49

50 Importing Symbols Symbols (functions/data) can be imported from external DLLs The loader will load external DLLs automatically All the dependencies are loaded as well DLLs will be loaded only once External addresses are written to the Import Address Table (IAT) 50

51 Importing Symbols Each DLL has one IMAGE_IMPORT_DESCRIPTOR The descriptor points to two parallel lists of symbols to import Import Address Table (IAT) Import Name Table (INT) The primary list is overwritten by the loader, the second one is not Executables can be pre-bound to DLLs to speed up loading Symbols can be imported by ASCII name or ordinal 51

52 Import Descriptors 52

53 Exports Symbols can be exported with ordinals, names or both Ordinals are simple index numbers of symbols Name is a full ASCII name of the exported symbol Exports can be forwarded to another DLL Forwarded symbol s address points to a name in the exports section 53

54 Resources Resources in PE are similar to an archive Resource files can be organised into directory trees The data structure is quite complex but there are tools to handle it Most common resources: Icons Version information GUI resources 54

55 Base Relocation Preferred image base MZ/PE Header Section Table.text.data Relocation offset Actual image base MZ/PE Header Section Table.text.data 55

56 Base Relocation Sometimes a DLL can not be loaded to its preferred address When rebasing, the loader has to adjust all hardcoded addresses Relocations are done in 4KiB blocks (page size on x86) Each relocation entry gives a type and points to a location The loader calculates the base address difference The offsets are adjusted according to the difference 56

57 ELF ELF = Executable and Linkable Format Used on many platforms Linux, Solaris, FreeBSD, NetBSD, OpenBSD Native executables for Android We ll talk more about Android applications on later lectures We won t go into ELF details during this lecture 57

58