8/19/2010. Computer Forensics File system forensics. Data analysis. Process of analysing data. Different layers of information.

Transcription

1 Computer Forensics File system forensics Thomas Mundt Data analysis Different layers of information. Application data analysis Swap space analysis File system analysis Database analysis Volume analysis Memory analysis Physical storage media analysis Network analysis Computer analysis 2 Process of analysing data HDD Physical media analysis Sectors Volume analysis Volume File system analysis File Application analysis 3 1

2 General guidelines Copy important data, put the original data in a safe place. Problems: Bad sectors, internal structure not always accessable. Use write blocking device for copying. Analyse the copy. Calculate hash values (eg. SHA-1) to prove that the data has not changed. Minimize write access when performing a live analysis. Prevent change of access times. Prevent any write operation because they could overwrite evidence in unallocated space. 4 General guidelines Isolate the analysis environment from both the suspect data and the network environment (internet etc.). Minimal system or virtual system (which can be reset to an initial state very easily). Isolation is impossible when analysing a live system. Correlate data with other independent sources. Log and document all own activities. 5 Data organization Remember Binary storage of information / data formats for INT, DOUBLE etc. Encoding of characters (eg. ASCII, ISO , UTF-8 etc.) Unicode v. 4.0 supports over 96,000 different characters Internationalization and different conventions (see 96,000 above, where the comma delimits between every thousands). Big endian / little endian. 6 2

3 Example UTF-32 stores each character as a 4-byte value. UTF-16 stores most heavily used characters in a 2-byte value and the lesser-used in a 4-byte value. UTF-8 uses 1, 2, or 4 bytes. 7 Example thm$ hexdump -C L2\ -\ File\ System\ Forensics.pptx b e PK...!..n c0 2a a e b 43.*...n...[C f 6e e 74 5f d 2e 78 6d ontent_types].xm c 20 a a l...( ASCII interpretation of the dumped data. 8 Data structures Layout of data is sometimes unclear because data are not generally preceded with data type and meaning. Example thm$ hexdump C xxx.db d e e Main St bit integer for number. 14 char (ASCII) for name of street is little endian (in this particular case). Need to know the structure of all data. 9 3

4 Data structures Most forensic data is stored on hard disc drives. In commercial use since Capacity increasing. 10 IDE HDD 11 Magnetic storage 12 4

5 Cylinder heads sectors blocks 13 ATA / SATA 14 SCSI 15 5

6 Types of sector addresses Each block typically consists of 512bytes CHS Cylinder Heads Sectors. ATA specification defines16bit cylinder value, 4bit head value, and 8bit sector value. BIOS use 10bit cylinder value, 8bit head value. And 6bit sector value. 1024*16*63*512byte = 504MiB limit for HDDs. LBA Logical Block Addressing LBA = (((C* heads_per_cyl) + H) * sectors_per_track) + S 1 Both methods are still in use. 16 Challenges when examining HDDs Encryption. Unknown file systems. Hidden information. Internal configuration of HDDs, such as dead sector mapping. 17 Hard disk passwords ATA-3 spec introduced optional security features. Passwords can be set to lock the HDD against reading / writing. Data recovery is still possible by opening the disk. 18 6

7 Host protected area HPA was added in ATA-4 spec. Computer vendors can store data that would not be erased when a user formats the HDD. 0 GB 199 GB 200 GB User addressable sectors HPA 19 Device configuration overlay DCO was added in ATA-6. DCO hides additional sectors at the end of disk. 0 GB 199 GB 200 GB 202 GB User addressable sectors HPA DCO 20 Bad sector mapping When a hard disk is manufactured, there are areas on the platter that have bad sectors. During the final test phases of a hard disk, the platters are scanned at the factory and the bad sectors are mapped out - these are generally called 'Primary Defects'. All modern drive have a spare sector pool. 21 7

8 BIOS versus direct access. Direct access to controller enables the OS to operate in special modes (LBA). BIOS access to controller uses software interrupt command 0x13 (INT13h CHS or extended INT13h with 64-bit LBA). 22 SCSI versus ATA SCSI is a different interface standard, usually found in servers. SCSI uses a bus and spec does not know a controller. Instead a bus system is used. Driver addresses each device. 23 Data acquisition Copying to keep the original untouched throughout the entire process of analysis. Write blockers are frequently used to prevent changes. Steps Reading the source data. Raw copy vs. copy on file level. Copying HPA and DCO (and possible bad blocks). Error handling (errors on sector level for instance could be logged and copied as zeros). Writing the data. To an image file. To another medium. 24 8

9 Tools DriveID diskstat from The Sleuth Kit hpa 25 Write blockers Hardware write blockers ATA / SCSI / Firewire commands are checked and only accepted when on a list of no-write-commands. Software write blockers Modified INT13h handler. Modified direct access drivers. 26 Image files Stores the entire partition / HDD (including HPA). Some image file formats are able to handle relocated data (bad blocks). Wiping the disk before writing image file, so that investigator can more easily testify that there could not have been any contamination. Different image file formats Raw image contains only the data from the source device. Embedded image contains data from the source device and descriptive data such as hash values, dates, and times. 27 9

10 Volume analysis A volume is a collection of addressable sectors that an operating system can use for data storage. Sectors in a volume need not to be consecutive on a physical storage device. A volume may also be the result of assembling smaller volumes. A partition is a special volume. It is a collection of consecutive sectors in a volume. 28 Partitions Usage File systems smaller than HDD. Storage of RAM content for sleep. Swap partition for OS that use memory pages. Each OS typically uses a different partitioning method. A table stores starting sector, ending sector, and type of partition. 29 Partitions Start End Type NTFS FAT Linux Partition 1 Partition 2 Partition

11 Example Partition 1 Partition 2 HDD volume 1 Partition 3 HDD volume 2 Partition 4 Partition 5 Intermediate volume FS volume File system volume 31 Example RAID Investigating a system with a RAID volume can be very difficult. Need to collect all devices. Hardware RAID is not fully standardized. 32 RAID 33 11

12 RAID Analysis basics Basic theory is very simple. Locate the partion tables. Process them. Identify disk layout. Layout is used in a file system analysis. In some cases data outside partitions needs to be extracted. File system defines where data is located. In some cases data outside user accessable files needs to be extracted

13 Partion tables Consistency checks Valid Invalid Partition 1 Partition 2 Partition 3 Partition2 Partition1 37 Recovering deleted partitions A common technique that is used to thwart a forensic investigation is to repartition a disk. Similar problem is to recover a disk whose partition table has become corrupt. Some hints remain, for example FAT system has a field in each sector that identifies how many sectors are in a cluster, value is a power of 2. Any other value would indicate that the sector was not part of a FAT file system. Some file systems use specific magic signatures (FAT, for instance, has 0x55 and 0xAA in bytes 510 and 511 of the first sector). 38 Recovering deleted partitions Other tools use heuristics

14 PC based partitions DOS-style partitions (IA32). Apple partitions. Removable media. 40 DOS partitions No standard reference, no standard name. Many documents that discuss those partitions. Used by Windows and most LINUX and FreeBSD systems. Since Windows 2000 Microsoft differentiates between basic and dynamic disks. A basic disk refers to MBR or GPT disks, which are standalone. Dynamic disks can be combined and merged. 41 DOS partitions Microsoft calls disks using this type of partition Master Boot Record (MBR) disks. IA64 and Extensible Firmware Interface (substitute for BIOS) use GUID partition tables (GPT)

15 Basic MBR concepts A disk using DOS partitions has an MBR in the first 512- byte sector. The MBR contains boot code, a partition table, and a signature value. The boot code contains instructions that tell the computer how to process the partition table and locate the operating system on the partition marked as active. 43 Basic MBR concepts Partition table has four entries, each can describe a DOS partition. Each entry consists of Starting CHS address. Ending CHS address. Starting LBA address. Number of sectors in partition. Type of partition. Flags. 44 Type of DOS partitions 0 Empty 1e Hidden W95 FAT1 80 Old Minix bf Solaris 1 FAT12 24 NEC DOS 81 Minix / old c1 DRDOS/sec Lin (FAT- 2 XENIX root 39 Plan 9 82 Linux swap / So c4 DRDOS/sec (FAT- 3 XENIX usr 3c PartitionMagic 83 Linux c6 DRDOS/sec (FAT- 4 FAT16 <32M 40 Venix OS/2 hidden c7 Syrinx C: 5 Extended 41 PPC PReP Boot 85 Linux extended da Non-FS data 6 FAT16 42 SFS 86 NTFS volume set db CP/M / CTOS /. 7 HPFS/NTFS 4d QNX4.x 87 NTFS volume set de Dell Utility 8 AIX 4e QNX4.x 88 Linux plaintext df BootIt 2nd part 9 AIX 4f QNX4.x 3rd part 8e Linux LVM e1 DOS access bootable a OS/2 Boot Manag 50 OnTrack DM 93 Amoeba e3 DOS R/O b W95 FAT32 51 OnTrack DM6 94 Amoeba e4 SpeedStor Aux BBT c W95 FAT32 52 CP/M 9f BSD/OS eb BeOS fs (LBA) e W95 FAT16 (LBA) 53 OnTrack DM6 Aux a0 IBM Thinkpad hi ee EFI GPT f W95 Ext'd (LBA) 54 OnTrackDM6 a5 FreeBSD ef EFI (FAT-12/16/ 10 OPUS 55 EZ-Drive a6 OpenBSD f0 Linux/PA-RISC b 11 Hidden FAT12 56 Golden Bow a7 NeXTSTEP f1 SpeedStor 12 Compaq diagnost 5c Priam Edisk a8 Darwin UFS f4 SpeedStor 14 Hidden FAT16 <3 61 SpeedStor a9 NetBSD f2 DOS secondary 16 Hidden FAT16 63 GNU HURD or ab Darwin boot fb VMware VMFS Sys 17 Hidden HPFS/NTF 64 Novell Netware b7 BSDI fs fc VMware VMKCORE 18 AST SmartSleep 65 Novell Netware b8 BSDI swap fd Linux raid auto 1b Hidden W95 FAT3 70 DiskSecure Mult bb Boot Wizard hid fe LANstep 1c Hidden W95 FAT3 75 PC/IX be Solaris boot ff BBT 45 15

16 Extended partition concepts MBR is a simple method allowing up to four partitions. More partitions can be defined by creating an extended partition. Partition 1 Partition 2 Partition 3 Extended partition 46 Extended partitions Extended partition Partition 4 Partition 5 Partition 6 47 Example 48 16

17 Apple partitions Apple partitions are described in the partition map structure, which is located at the beginning of the disk. No boot code necessary, all in firmware. First entry defines maximum size of the partition map. 49 Apple partion map entry Byte Range Description Essential 0-1 Signature value (Ox504D) No 2-3 Reserved No 4-7 Total Number of partitions Yes 8-11 Starting sector of partition Yes Size of partition in sectors Yes Name of partition in ASCII No Type of partition in ASCII No Starting sector of data area in partition No Size of data area in sectors. No Status of partition (see table 5-8) No Starting sector of boot code No Size of boot code in sectors No Address of boot loader code No Reserved No Boot code entry point No Reserved No Boot code checksum No Processor type No Reserved No 50 Example Apple partition map Slot Start End Length Description 00: Unallocated 01: Apple_partition_map 02: Table 03: Unallocated 04: Apple_Driver43 05: Apple_Driver43 06: Apple_Driver_ATA 07: Apple_Driver_ATA 08: Apple_FWDriver 09: Apple_Driver_IOKit 10: Apple_Patches 11: Apple_HFS 12: Apple_Free 51 17

18 Removable media Most removable medai also have partitions. They use the same structures that HDDs use. Floppy disks are formatted for FAT12. They do not have a partition table, the entire disk is treated like a single partition. CDs use ISO 9660, Joliet, or even Apple HFS+ format. Recordable CDs have a notion of a session. Each session could be considered as a single partition. 52 BSD partitions FreeBSD uses DOS partitions and BSD partitions together. BSD partitions (UFS) are defined inside a BSD DOS partition. BSD uses partition tables. 53 BSD partitions NetBSD / OpenBSD partitions are also located inside a DOS partition (on IA32 systems). Partitions can be located anywhere on the disk

19 Logical Volume Manager LVM DOS partitions (type 0x8e) are devided into physical extents. A logical volume is made up of logical extents. Configuration files are stored in on both the local file system and the disks / partitions. 55 File system analysis After analysing the partition structure the file system inside the partition has to be analysed. A file system porovides a mechanism for users to store data in a hierarchy. A filesystem connects files with the sectors of a storage media. It adds meta-data to the file (dates, rights etc.) 56 Logical file system address File systems use logical volume addresses and assign logical file system addresses. Logical volume addresses are relative to the start of the volume. In some file systems every sector in the volume is assigned to a logical file system address

20 Logical file address Logical file address is relative to the start of a file. 58 Slack space Slack space occurs when the size of a file is not a multiple of a data unit. A file must allocate a full data unit. DOS and early Windows fill slack space with data from memory. Some Oses wipe empty sectors, others ignore them. 59 Allocation strategies Typically, an OS allocates consecutive data units. This is not always possible. Deleted files leave a hole. Results in fragmentation. Strategies are First available. Next available. Best fit or mimumum free space. Maximum free space

21 Sparse files Sparse files attemt to store data more efficiently. Empty blocks are not physically allocated. Most modern file systems (NTFS, ext3/4 support sparse files) 61 FAT Used by many (all) Microsoft Operating Systems. Also used by most flash memory devices (USB, Compact Flash). Every file and directory is allocated a directory entry. Content is stored in clusters. 62 FAT file system layout 63 21

22 FAT example 64 FAT cluster address of first cluster In FAT12/16, cluster 2 follows the root directory, in FAT32 cluster 2 is the first sector of the data area. 65 FAT allocation algorithms Most Oses seem to use next available algorithm for allocation of a cluster

23 FAT directory entry Byte Description 0-0 First character of file name in ASCII and allocation status (Oxe5 or OxOO if unallocated) 1-10 Characters 2 to 11 of file name in ASCII File Attributes Reserved Created time (tenths of second) Created time (hours, minutes, seconds) Created day Accessed day High 2 bytes of first cluster address (0 for FAT12 and FATl6) Written time (hours, minutes, seconds) Written day Low 2 bytes of first cluster address Size of file (0 for directories) 67 FAT dates 68 FAT times 69 23

24 Flag values Flag value Description Essential (0x01) Read only No (0x02) Hidden file No (0x04) System file No (0x08) Volume label Yes (0x0f) Long file name Yes (0x10) Directory Yes (0x20) Archive No 70 NTFS New Technology File System. Used by all modern versions of Windows. Badly documented by Microsoft. Open source teams re-engineered the file system now available for Unix / MacOS etc. 71 NTFS Every information about files and directories is stored in the Master File Table (and a backup copy of it). Each MFT entry is given an address. All entries have been 1024 bytes in size (exact size is defined in the boot sector). NTFS uses clusters

25 MFT 73 NFTS The first 16 MFT entries are reserved for file system metadata files (readable in the root directory, but usually hidden from the user). 74 NTFS file system metadata files Entry File Name Description $MFT The entry for the MFT itself. $MFTMirr Contains a backup of the first entries in the MFT. $LogFile Contains the journal that records the metadata transactions. $Volume Contains the volume information such as the label, identifier, and version. $AttrDef Contains the attribute information, such as the identifier values, name, and sizes.. Contains the root directory of the file system. $Bitmap Contains the allocation status of each cluster in the file system. $Boot Contains the boot sector and boot code for the file system. $Baddus Contains the clusters that have bad sectors. $Secure Contains information about the security and access control for the files (Windows 2000 and XP version only). $Upcase Contains the uppercase version of every Unicode character. $Extend A directory that contains files for optional extensions. Microsoft does not typically place the files in this 75 directory into the reserved MFT entries. 25

26 Typical file systems exist to read and write file content, but NTFS exists to read and write attributes, one of which contains file content. 76 Name $STANDARD_INFORMATION $ATTRIBUTE_LIST $FILE_NAME $VOLUME_VERSION $OBJECT_ID $SECURITY_DESCRIPTOR $VOLUME_NAME $VOLUME_INFORMATION $DATA $INDEX_ROOT $INDEX_ALLOCATION $BITMAP 77 Description General information, such as flags; the last accessed, written, and created times; and the owner and security ID. List where other attributes for ftle can be found. File name, in Unicode, and the last accessed, written, and created times. Volume information. A 16-byte unique identifier for the file or directory. The access control and security properties of the file. Volume name. File system version and other flags. File contents. Root node of an index tree. Nodes of an index tree rooted in $INDEX_ROOT attribute. A bitmap for the $MFT file and for indexes

27 NTFS and sparse files 79 NTFS and sparse files 80 B-Tree NTFS uses B-Trees for indexing

28 Allocation NTFS implementations seem to use the best-fit algorithm on most implementations. 82 NTFS example 83 NTFS example 84 28

29 ext2 / ext3 / extx Derived from UFS. Used by a variety of Linux distributions as default file system. Many optional features. Journaling. Sparse files. Large files (>4GB). Implementation depends on Linux distribution (or even the currently installed kernel features). 85 extx concepts / example Super block (equivalent to boot sector) contains information about the file system. 86 extx concepts / example File system is devided into groups, each group has the same layout. Files are supposed to be stored in the same block group, when they belong together (avoids the HDDs heads to be moved too much)

30 extx concepts / example Every block group contains a backup copy of the super block and the group description table. 88 extx concepts / example Meta information is stored in inodes. Each inode represents one file. 89 extx concepts / example Bigger files are adressed indirectly by using blocks to link to further data content

31 extx concepts / example Directory entries contain the file name and inode. They also contain a pointer to the next entry. Unused entries are skipped over by increasing the pointer of the previous entry. 91 Countermeasures and problems Wiping. Encryption. Compression. 92 Plausible deniability Deniability is the avoidance of being blamed to know something / to possess somthing / to have something done. Plausible Deniability means that there is no way to prove that someone has knowledge / did something. Plausibility is the key concept of plausible deniability Often used in politics / espionage / business 93 31

32 Reasons Prosecutor (or similar entity such as journalist) cannot prove that information is present (at least beyond reasonable doubt) Some countries changed rules regarding encrypted material UK law enformecment officers can require the disclosure of keys Regulation of Investigatory Powers Act Part III - Investigation of electronic data protected by encryption etc. - Power to require disclosure 94 Methods No written documents Verbal agreements beside written documents 95 Presence of information What is information? Shannon: Frequently the messages have meaning; that is they refer to or are correlated according to some system with certain physical or conceptual entities. These semantic aspects of communication are irrelevant to the engineering problem. The significant aspect is that the actual message is one selected from a set of possible messages. The choice of a logarithmic base corresponds to the choice of a unit for measuring information. If the base 2 is used the resulting units may be called binary digits, or more briefly bits, a word suggested by J. W. Tukey. A device with two stable positions, such as a relay or a flip-flop circuit, can store one bit of information. N such devices can store N bits The Bell System Technical Journal, Vol. 27, p. 379, (July 1948)

33 Data, Information, Knowledge Data is the lowest level of abstraction, information is the next level, and finally, knowledge is the highest level among all three. Is this data? Is this information? 69ba 90eb 44a e0e4 1d8c 9fb ed5a 967f c4e2 78ac 2d56 7e05 152f 1ba0 bf37 61d1 b564 cc7b 22dd 1828 c9cd ba14 d130 2f9c 6fe b1ad 3afd 9ee8 4fb7 6fd2 f835 f656 76e6 1f12 a38c 76cc d4cf 336b 1afe f018 9e44 ef00 26bb 97 Information Is this information? -bash-3.2$ hexdump /dev/urandom 69ba 90eb 44a e0e4 1d8c 9fb ed5a 967f c4e2 78ac 2d56 7e05 152f 1ba0 bf37 61d1 b564 cc7b 22dd 1828 c9cd ba14 d130 2f9c 6fe b1ad 3afd 9ee8 4fb7 6fd2 f835 f656 76e6 1f12 a38c 76cc d4cf 336b 1afe f018 9e44 ef00 26bb 98 Implementation 1st step encrypted volumes 2nd step hidden volumes 3rd step hidden volume inside encrypted cover volume No part of the hidden volume can be distinguished from random data (they pass the chi-square-randomness test) Tools FreeOTFE ToolCrypt 99 33

34 Implementation From TrueCrypt Documentation 100 Cryptography Volumes are encrypted with a key depending on the user s password. 101 Problems Random Number Generator needed to deliver background noise Header structures Traces in RAM / Swap files / Paging files / Hibernation files Malware Relocated sectors Hard disks Special wear leveling file systems Journaling file systems

35 Random Number Generator Needs to be non-deterministic Has to pass chi-square-randomness test 103 Header structures All information needs to be encrypted The entire volume needs to look chaotic 104 Page files Encryption tools need to lock their memory from paging Tools cannot guarantee that user process will not store unencrypted data in a paging file

36 Hibernation files Tools cannot control creation of hibernation files (suspend to disk) Entire content of RAM is written to file Tools could put hibernation files in an encrypted volume Secure operating systems needed 106 Dump files Operating system might create a dump of memory onto hard disk Preventing OS to do so needs to be configured 107 RAM Tools cannot guarantee that RAM is being copied during operation Physical security is needed

37 Wear leveling / sector relocation / Journaling file systems Tools cannot guarantee that data is really overwritten Avoid wear those file systems Journalling Flash File System jffs 109 Further problems Changing passwords Backup Copied volumes (brute force against passwords)