File System Interpretation

Transcription

1 File System Interpretation Part III. Advanced Techniques and Tools for Digital Forensics CSF: Forensics Cyber-Security Fall 2018 Nuno Santos

2 Previously: Introduction to Android forensics! How does Android work?! What evidence can be found on Android devices?! How to retrieve evidence from Android devices?! How can Android apps be reverse engineered? 2

3 Remaining advanced topics! Advanced file system analysis (2 lectures)! Cloud forensics (1/2 lecture) + Exercises (1/2 lecture) 3

4 Class roadmap! Mobile forensics! Forensics analysis of Android apps! File system interpretation! Review: File system forensics! Layout of the FAT file system! Layout of the NTFS file system 4

5 Forensic analysis of Android apps 5

6 Android application repackaging! Tampering app made easy! Decompile and modify DEX bytecode! Recompile and distribute malicious APK 6

7 ! Get control flow, string information! Java decompiler Static code analysis! baksmali (used by apktool) 7

8 Packet capture! Use capture tools on Android side! Some tool like tcpdump required rooting! Build rogue AP and sniffing! ARP spoofing, MITM attack 8

9 Private information on debug log (Logcat)! Probably developers are too lazy! Google recommends screening of all logging API on Android before release! Example of PIN code on debug log PIN: syssec0! 9

10 Injecting debug code! Insert debug code around interested instructions on application! Print private key, private information, etc.! Alternatively compile Android code, compile and convert to smali, inject the resulting code! Native code is still a problem 10

11 ! Android app may use native code! Dynamic analysis of native code Native code debugging! No Dalvik VM is involved, native debugger like GDB, IDA could be used 11

12 Dynamic taint analysis! Does a mobile app misuse users private information?! Location, phone ids, microphone, camera, address book! Dynamic taint analysis is a technique that tracks information dependencies from an origin! Conceptual idea:! Taint source! Taint propagation! Taint sink 12

13 Taint tracking systems! Systems that implement dynamic taint tracking within a common Android smartphone architecture! E.g., TaintDroid, TaintART, etc. 13

14 Review: File system forensics 14

15 File system interpretation! File system interpretation examines data in a volume (i.e., a partition or disk) and interprets them as a file system 45#00#00#1d#7b#bd#00#00# c0#a8#01#37#23#82#23#83#...#! Outcome results: listing files in directories, recovering deleted content, viewing the contents of a sector, etc /18

16 Partitions have their own file systems! A disk that is organized using DOS partitions has a Master Boot Record (MBR) in the first 512-byte sector! The MBR has a partition table with four entries, one for each partition Master Boot Record A basic DOS disk with two partitions and the MBR 16

17 MBR layout! The MBR contains boot code, a partition table, and a signature value 17

18 ! File contents are stored inside blocks (e.g., 4KB) The ExtX file system! Each block has an address within the partition (0 up to max #blocks -1)! The blocks allocated to a file are kept by a record called inode! ExtX keeps track of all inodes inside a table! Directory entries associate the file name with the file s inode Table of Inodes 18

19 Inode block pointers! Each inode is the root of an unbalanced tree of blocks that belong to a given file 12 blocks * 4KB = 48KB 1024 blocks * 4KB = 4MB 1024 * 1024 blocks * 4KB = 4GB 2 30 blocks * 4KB = 4TB Triple indirect blocks 19

20 On-disk organization of a ExtX file system! Organized as sequence of logical blocks! The block size is defined upon disk formatting; 1, 2, 4 or 8KB are common! Blocks are grouped into larger units called block groups! All block groups have equal length possibly except the last one! The first data block aka boot block is not used by the FS! Has a fixed 1024 byte length and may contain bootstrap code 20

21 FS evidence can be grouped into categories Data evidence categories of the ExtX file system family Superblock, group descriptor Journal Directory entries Blocks, block bitmap Inodes, inode bitmap 21

22 Data categories of popular file systems Category FAT FS EXTx NTFS File system Boot sector Superblock, group descriptor $Boot, $Volume, $AttrDef Content Clusters, FAT Blocks, block bitmap Clusters, $Bitmap Metadata Directory entries, FAT Inodes, inode bitmap, $MFT, $MFTMirr, $STANDARD_INFORM ATION, $DATA, $ATTRIBUTE_LIST, $SECURITY_DESCRIPT OR File name Directory entries Directory entries $FILE_NAME, $IDX_ROOT, $IDX_ALLOCATION, $BITMAP Application N/A Journal Journal, disk quota 22

23 Layout of the FAT file system 23

24 The FAT FS: why is it relevant for forensics! It s a historical file system popularized by MS-DOS! First introduced in 1977, and evolved: FAT12, FAT16, FAT32! Today still in use, mostly the FAT32 spec from 1996! Still quite popular today! Common in USB sticks and memory cards! Used in some embedded systems! Very simple, it s good for learning FS forensics 24

25 Layout of a FAT file system! Layout of FAT16 on a volume (two more variants FAT12, FAT32)! FAT comes from the index table used to track directories and files: File Allocation Table 25

26 Partition boot sector! Occupies the first sector in the partition: FAT Partition Boot Sector 1. First 3 bytes form an 8086 jump instruction to the entry point of the bootstrap code at the end of the sector 2. Next, an 8 byte field which is used by the system manufacturer for an identification string 3. Next, the BIOS parameter block (BPB): contains all the values required by the OS to find the other system areas and calculate the mapping from file level accesses to logical sector addresses 4. Next, 448 bytes taken up with the bootstrap program 5. Ends with an end of sector marker, valued 0x55AA 00h# 03h# 0Bh# 24h# 3Eh# Jump (3B) OEM name (8B) BIOS Parameter Block (25B) Extended BIOS Parameter Block (26B) Bootstrap code (448B)! The boot sector is part of a reserved area that can be one or more sectors long (specified in the BPB) 1FEh# End of sector marker (1B) 26

27 ! In the FAT FS, data space is divided into an array of clusters The data area Array of clusters! Cluster consists of group of consecutive sectors! Cluster size is defined in the BPB (common cluster size 8 sectors = 4KB)! Clusters start with number 2 (first sector after the root directory) Clusters Sectors 8 Sectors! Clusters are the data units where file and folder contents are stored 27

28 The FAT data structure! In FAT16, each entry is 16 bits, fixed size! FAT12 and FAT32 have 12 and 32 bit entries, respectively! Possible values for FAT16 entries:! 0000h: entry is empty! 0001h: not used by the OS! FFF0-FFF6h: reserved by the OS! FFF7h: bad cluster! Remaining values: next block in a chain! >=FFF8h: represent end of file (EOF)! The size of the FAT is indicated in the BPB 28

29 The FAT is a table that has two functions! Manages the disk space by tracking free / allocated clusters! Keeps track of which clusters belong to each file FAT! Each FAT entry keeps track of the state of a cluster! The index of the FAT entry indicates the index of that cluster! The value of a FAT entry points to next cluster of same file 29

30 Relation directory, FAT, & clusters Directory entry structures (most relevant fields) FAT structure Clusters file1.dat# 4000#bytes# cluster#34# # 33# 32# 33# 34# EOF# 36# 34# file2.txt# 100#bytes# cluster#33# 35# 36# EOF# 37# 35# # 36# 30

31 Location of root directory Clusters Root directory FAT structure file1.dat# 4000#bytes# cluster#34# # 33# 32# 33# 34# EOF# 36# 34# file2.txt# 100#bytes# cluster#33# 35# 36# EOF# 37# 35# # 36# 31

32 Case study: Determine the layout of FAT FS! This is the partition boot sector Find the BPB 0x0B => 0x23 32

33 Byte offset BIOS parameter block (BPB)! Full specification can be found at: Some important fields (here, WORD = 2bytes, little-endian): Field length Sample value Field Name Meaning 0x0B WORD 0x0002 Bytes per Sector The size of a hardware sector. For most disks in use in the United States, the value of this field is x0D BYTE 0x08 Sectors per Cluster 0x0E WORD 0x0100 Reserved Sectors The number of sectors in a cluster. The default cluster size for a volume depends on the volume size and the file system. The number of sectors from the Partition Boot Sector to the start of the first file allocation table, including the Partition Boot Sector. The minimum value is 1. If the value is greater than 1, it means that the bootstrap code is too long to fit completely in the Partition Boot Sector. 0x10 BYTE 0x02 Number of FATs The number of copies of the file allocation table on the volume. Typically, the value of this field is 2. 0x11 WORD 0x0002 Root Entries The total number of file name entries that can be stored in the root folder of the volume. One entry is always used as a Volume Label. Files with long filenames use up multiple entries per file. Therefore, the largest number of files in the root folder is typically 511, but you will run out of entries sooner if you use long filenames. 0x16 WORD 0xC900 Sectors per FAT Number of sectors occupied by each of the file allocation tables on the volume. By using this information, together with the Number of FATs and Reserved Sectors, you can compute where the root folder begins. By using the number of entries in the root folder, you can also compute where the user data area of the volume begins. 33

34 1 st determine the size of the reserved area 34

35 Size of the reserved area! 1 partition boot sector + additional reserved sectors 0x0E WORD 0x0100 Reserved Sectors The number of sectors from the Partition Boot Sector to the start of the first file allocation table, including the Partition Boot Sector. The minimum value is 1. If the value is greater than 1, it means that the bootstrap code is too long to fit completely in the Partition Boot Sector.! 0E-0Fh: Reserved sectors (little endian)! 01 00! = 1! Thus, reserved area has the partition boot sector only 35

36 2 nd determine the size of FAT area Region for FAT data structures FAT marks data blocks free or in-use Linked-list structure to manage files 36

37 The FAT area! FAT = File Allocation Table! There may exist multiple FAT copies on the FAT area 0x10 BYTE 0x02 Number of FATs The number of copies of the file allocation table on the volume. Typically, the value of this field is 2.! 10h: 2 => the FAT area has two FAT copies 37

38 Determining the FAT area boundaries 0x16 WORD 0xC900 Sectors per FAT Number of sectors occupied by each of the file allocation tables on the volume. By using this information, together with the Number of FATs and Reserved Sectors, you can compute where the root folder begins. By using the number of entries in the root folder, you can also compute where the user data area of the volume begins.! 16-17h: Size of FAT is 00 7Bh sectors (123 sectors)! Thus, Root Directory starts at sector 1h+7Bh+7Bh (sector 247)

39 3 rd determine the root directory area Region for the directory entries of the root folder (fixed location) 39

40 Layout of the root directory! Root directory keeps files names, first cluster, size, and other metadata 32 bytes (fixed) FAT # root directory entries (defined in BPB) 00# 01# 02# 03# 04# 05# 06# # Root Directory Format of a root directory entry Offset Length Meaning 0x00 8B File Name 0x08 3B Extension 0x0b 1B File Attribute 0x0c 10B Reserved 0x16 2B Time of last change 0x18 2B Date of last change 0x1a 2B First cluster 0x1c 4B File size 40

41 Determine the root directory boundary 0x11 WORD 0x0002 Root Entries The total number of file name entries that can be stored in the root folder of the volume. One entry is always used as a Volume Label. Files with long filenames use up multiple entries per file. Therefore, the largest number of files in the root folder is typically 511, but you will run out of entries sooner if you use long filenames.! 11-12h: Root entries (little endian)! 00 02! = 512 decimal! Number of sectors of the root directory! = number of root entries x root entry size in bytes / bytes per sector! = 512 x 32 / 512 = 32 sectors

42 4 th determine the data area Store file and directory data Each cluster is a fixed size Files may span multiple clusters 42

43 Implications of the FAT to the data area layout! # bits per FAT entry sets the max # of clusters # bits per entry! FAT16 is limited to 2 16 = clusters! If both max # of clusters and their max size (32KB) are reached, the largest drive is limited to 4GB! In practice, subtract 12 reserved cluster entries # entries 00# 01# 02# 03# 04# 05#! To index bigger partitions, it s common to increase cluster size! 260MB 511MB: 8KB (16 x 512 sectors)! 512MB 1023MB: 16KB (32 x 512 sectors) 06# # FAT 43

44 Determine the data area boundary 0x0D BYTE 0x08 Sectors per Cluster! 0Dh: 4 sectors per cluster! Number of sectors of the data area! = number of clusters x sectors per cluster! = (number of FAT entries - 2) x sectors per cluster! 2 because the cluster numbering starts in 2; cluster 0 and 1 are not used! = ((# sectors per FAT x sector size / FAT entry size) - 2) x sectors per cluster! = ((123 x 512 / 2) - 2) x 4 = sectors The number of sectors in a cluster. The default cluster size for a volume depends on the volume size and the file system. 44

45 Tools to help interpret the BPB! In TSK, use the fsstat tool! From the output of the tool we can draw the layout of the analyzed partition! On the right: output of a tool for a FAT FS unrelated from the previous example 45

46 FATx layouts compared 46

47 The layout of the NTFS file system 47

48 NTFS layout! Central paradigm: Everything is a file! Each byte of an NTFS file system belongs to a file! File system data and meta data are located in files, too! Similarly to FAT FS, files are chunked into clusters! Cluster 0 starts at the beginning of the file system! At the beginning of cluster 0 is the boot sector 48

49 Regions of an NTFS volume 49

50 ! NTFS boot sector: NTFS boot sector layout! In reality, the NTFS format program first 16 sectors for the boot sector and the bootstrap code 50

51 Overview of the Master File Table (MFT)! Some files can fit entirely within the MFT entry! Bigger files require allocation of extents 51

52 File system metadata files! NTFS stores the metadata across several metadata files! The MFT itself is a file system metadata file! The first 16 MFT entries are reserved for these files! Names of metadata files begin with $ and uppercase letters Entry # Name Description 0 $MFT Entry for the MFT itself 1 $MFTMirr Backup of the first MFT entries 2 $LogFile Journal for metadata changes 3 $Volume Contains information on labels, identifier and version of volume 4 $AttrDef Definition of the attributes used in the file system 5 \ Root directory of the file system 6 $Bitmap Allocation status of all clusters in the file system 7 $Boot Boot sector (key information about file system); starts in cluster 0 9 $Secure Definition of security descriptors of the file system (e.g., access control) 52

53 Internals of the Master File Table! In NTFS, information about all files and directories is contained in the Master File Table (MFT)! Every file and directory has at least one entry in MFT! Entries are 1KB in size! First 42 bytes is a header, the remaining bytes store attributes! Attributes: small data structures with a specific purpose! E.g., attribute to store the file's name, attribute to store the file's content 53

54 The Master File Table and attributes MFT entry Clusters Master File Table (MFT) # 100# 101# 102# 103# 104# 105# # $STANDARD_INFORMATION# resident* Creation#time:#,# Modification#time:#,#...# $DATA# non#resident* Clusters:#34F36#...# Attributes 33# 34# 35# 36# 54

55 From block pointers to extents! Idea: keep track of contiguous clusters called extents! Results in less block pointers for contiguous clusters! Suits well OS allocation policies for reducing fragmentation! Since it results in many seeks, thus low performance! NTFS keeps file s extents (aka cluster runs) in run lists Run list Example runlist with three runs of allocated clusters Cluster s logical file system address 55

56 Compression! NTFS provides compression on the file system level! Optional units:! Single attributes! Files! Folders! Entire volumes! Automatic decompression, if compressed data is read 56

57 Compression example 1. Broke up content into equal sized chunks (compression units) 2. Compress the non-zeroed chunks 3. Remove sparse regions (i.e., filled with zeroes) 4. Update the runlist of the compressed attribute (e.g., $DATA) 57

58 Encrypted attributes! NTFS provides the capability for attribute contents to be encrypted! In theory, any attribute could be encrypted, but Windows allows only $DATA attributes to be encrypted! When an attribute is encrypted, only the content is encrypted and the attribute header is not! A $LOGGED_UTILITY_STREAM attribute is created for the file; it contains the (encrypted) keys needed to decrypt the data 58

59 Alternate data streams! In NTFS, every file has a $DATA attribute, which contains the file content! A file can have more than one $DATA attribute! Additional attributes are named Alternate Data Streams (ADS)! The default $DATA attribute does not have an associated name associated; additional attributes must have one! E.g., normal.txt (default $DATA attribute)! E.g., normal.txt:my_ads_data.txt ($DATA attribute of an ADS) 59

60 How an ADS looks like in an MTF entry The alternate data stream The default $DATA attribute 60

61 Command line interface to ADS streams! When you access a file using it s default name you access the file s default $DATA attribute! Examples:! Create ADS streams to hide a text file and a picture! Show the content of ADS streams 61

62 Detection of Alternate Data Streams! Windows 7: dir /r! Other: Special Tools like lads.! Using TSK (e.g. fls). 62

63 Conclusions! For file system interpretation, one needs to be familiar with the studied file system and its respective layout! FAT, ExtX, and NTFS are particularly important for beginner forensic investigators for their popularity! Normally, interpretation of low-level file system data structures is aided by special-purpose forensic tools 63

64 References! Primary bibliography! Bryan Carrier, File System Analysis, 2005, Chapters 9 & 11 64

65 Next class! III.8 Case Studies on File System Analysis 65