HTCIA International Conference. Atlanta, GA. Robert Shullich CPP, CISSP, CISM, CISA, CGEIT, CRISC, GSEC, GCFA. September 20th,

Transcription

1 HTCIA International Conference September 20-22, Atlanta, GA Demystifying the Microsoft Extended File System (exfat) Robert Shullich CPP, CISSP, CISM, CISA, CGEIT, CRISC, GSEC, GCFA September 20th,

2 Agenda About Me Why a new file system Forensics Relevance Features Advantages Timelines Support Limits Internals September 20th,

3 About Me I have been in the IT field for 35+ Years, and in InfoSec for over 15 Years I carry many IT and InfoSec certifications This research was part of a term project for a forensics class for my masters in Forensic Computing I then expanded the term paper into a practical paper for my SANS GCFA certification A link to the SANS paper and my blog is at the end of this presentation September 20th,

4 Why do we need a new file system? Current Limits Exhausted Larger volumes (>2TB) Larger files sizes (>4GB) Faster I/O (UHS-1: 104 MB/2 - UHS-2: 300MB/s) Removable Media Flexibility Extensibility NTFS Features without the overhead September 20th,

5 Relevance to Forensics Study Digital Evidence Extraction Finding the evidence Including the hiding places Validation Daubert Expert Testimony Need to know and understand file org New Media (SD Cards) will drive exfat adoption, and the potential for CP investigations. September 20th,

6 What happens when you have exfat formatted media and no exfat support? September 20th,

7 Forensics Challenges Linux OS Support Tuxera drivers may help Mac OS Support Open Source Tools Commercial Tools Encase FTK Documentation September 20th,

8 Disclaimer The released specification and implementation is Release 1.00 of exfat The specification mentions additional features that were not implemented yet, but may at a future time/ Some of these are Windows CE holdovers Both may be presented today Some directory entries will be skipped September 20th,

9 Exponents 10 2 = 10 times 10 = = 10 times 10 times 10 = 1000 (1K) 2 2 = 2 times 2 = = 2*2*2*2*2*2*2*2*2 = =2*2*2*2*2*2*2*2*2* = 1024 (1K) 2 12 = 2*2*2*2*2*2*2*2*2*2*2*2 = 4096 September 20th,

10 International System of Units (SI) Table File System in powers of 2 Shorthand Longhand Nth Bytes KiB Kibibyte Device characteristics in power of 10 GiB Gibibyte MiB MiB Mebibyte KiB TiB Tebibyte GiB PiB Pebibyte TiB EiB Exbibyte PiB ZiB Zebibyte EiB YiB Yobibyte ZiB September 20th,

11 Features of exfat 1.00 Sector sizes from 512 to 4096 bytes Clusters sizes to 32MiB Root Directory Unlimited Subdirectories to 256MiB Built for speed, less overhead than NTFS but has some of the NTFS features UTC Timestamp Support Vista/Server 2008 SP2+, XP with KB September 20th,

12 Features of exfat 1.00 (cont d) OEM Parameters Sector for device dependent parameters 12 sector VBR, support of larger boot program Potential capacity to 64ZiB Current support 128 PiB Up to 2,796,202, files per subdirectory File Names max to 255 Characters Unicode File Names and Volume Labels September 20th,

13 Future Features of exfat TexFAT (To be released later) Exists in Windows CE Transaction Safe exfat ACL (To be released later) Exists in Windows CE Encryption Support? Not announced, but mentioned how easy to add September 20th,

14 MBR Partition Limitations Microsoft File Systems are limited when stored in a MBR partition A partition is defined by a Master Boot Record A MBR uses a 4 byte value for number of sectors To get the maximum volume size, exfat cannot be created within a partition September 20th,

15 Advantages of exfat Handle growing capacities in media, increasing capacity to >32 GB. > 1000 files in a single directory. Speeds up storage allocation processes. Breaks file size 4 GB barrier. Supports interoperability with future desktop OSs. Provides an extensible format. Large cluster sizes September 20th,

16 Disadvantages of exfat Not all Windows CE features implemented No direct conversion to or from other FS Cannot use CONVERT command to NTFS No Floppy Support Mostly a Microsoft Desktop and Server World No Support for Older MS systems No Support for Non-MS systems No XBOX, PS3 or other special devices September 20th,

17 Key Dates for exfat September 2006 Windows CE 6.0 March 2008 Windows Vista Service Pack 1 January 2009 Announcement at CES of SDXC specification January 2009 Windows XP Drivers Available May 2009 Windows Vista Service Pack 2 August 2009 Tuxera Signs File System IP Agreement with Microsoft March 2009 Pretec Releases first SDXC Cards December 2009 Microsoft (re)announces exfat license program for third-parties December 2009 SDXC laptops due soon December 2009 Diskinternals releases exfat recovery utility December 2009 Encase support September 20th,

18 More Key Dates for exfat December 2009 Sony, Canon & Sanyo License January 2010 Funai License (LCD TV) February 2010 Panasonic License February 2010 Panasonic 64/48GB SDXC February 2010 Sony Memory Stick XC February 2010 Sandisk Ultra XC 64GB Card 3.0 Spec $350 September 20th,

19 More Key Dates June 1 st 2010 Tuxera Releases Linux & Android exfat drivers June 3 rd 2010 Kingston Releases Class 10 SDXC 64GB Card 60 MB/s read, 35 MB/s write. September 20th,

20 SD Card Association New Memory Card Consumer Appliances Follows SDHC Specification for 2TB Capacity September 20th,

21 September 20th,

22 SDXC Storage Capabilities From 32GB to 2TB on a card Exclusively exfat File System 300 MB/s I/O Transfer Storage 4,000 RAW images 100 HD movies or 60 hours of HD recording 17,000 fine-grade photos in a single directory September 20th,

23 Support for exfat Windows XP & Server 2003 KB (requires SP2 or SP3) Vista & Server 2008 SP1 Vista & Server 2008 SP2 (Adds UTC timestamp support) Windows 7 September 20th,

24 Reference Standards Bits are numbered right to left Decimal Offsets (zero based) Little-Endian numbers Unsigned numbers Sectors vs. Clusters Strings are 16 bit Unicode Strings not Terminated September 20th,

25 Endian Numbering order may vary based on processor type, is determined by the order the data bytes are read from the register. A 32 bit number is read as 4 8 bit bytes If I have the number 0x Big-Endian will store it as: 0x Little-Endian will store it as: 0x September 20th,

26 File System Integrity Version Verified 3 Checksums VBR UP-Case Table File Set Critical Directory Entries Other Checks and Balances File System should NOT mount if failures September 20th,

27 exfat Limits Volume size 128PiB MS said 64ZiB MS now says 256TiB File Size 16 EiB (64 bit number) Bigger than volume size Subdirectory 256MiB Sector bytes ( ) Cluster 32MiB (2 25 ) No floppy support No FAT32 minimum cluster (65,525) restriction No 8.3 file name support September 20th,

28 Data Hide Alert! FAT32 max cluster 32KiB exfat max cluster 32MiB This is an increase of 1024 fold Potential for massive slack space September 20th,

29 Volume Space Layout The Main Boot Region Contains main VBR The Backup Boot Region Contains backup VBR The FAT Region Contains FAT Table(s) The Data Region (Cluster Heap) This is where data resides September 20th,

30 September 20th,

31 VBR Volume Boot Record Contains 12 sectors 1 sector main boot sector Jump Code (3 bytes) BPB (BIOS Parameter Block) Boot Strap Code 8 sectors main extended boot sectors 1 sector OEM parms 1 sector reserved 1 sector VBR Checksum September 20th,

32 Boot Parameter Block (BPB) OEM Label EXFAT Volume Length (64-bit) [sector] FAT Location & Size [sector] Heap Location & Size [sector, cluster] Volume Serial Number Location of Root Directory [cluster] Volume Flags Sector and Cluster Sizes [2-shift] Percent in use File System Revision (0x0010=1.00) 1.00) September 20th,

33 Sectors & Clusters A2-Shift is a power of 2 Another name for exponent Sector size and sectors per cluster Each stored in 1 byte Theoretical maximum is Sector Size Maximum 2 12 Sectors per cluster is derived Cluster Size Maximum is 2 25 September 20th,

34 Executable Boot Code First 3 bytes of Main Boot Sector Jump Code 0xEB7690 Offset 120 size 390 Remainder of boot code Offset 510 End signature marker 0xAA55 = 55AA Offset 512 Unused if defined September 20th,

35 More Bootable Code Up to 8 Main Extended Boot Sectors FAT32 had 3 sector VBR with 1 MEBS Entire sector can be used for boot code Last 8 bytes of sector is marker 0xAA = AA Larger capacity for boot virus! September 20th,

36 VBR Checksum Sector The 12 th sector of the VBR Repeating 4 byte checksum Checksum of previous 11 sectors Flags and Percent excluded These are volatile and change often Boot Sector Virus & Checksum September 20th,

37 VBR Checksum Sector Offset A B C D E F C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ. Lines through 01BF repeated C0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ D0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ E0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ F0 C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B C9 D0 18 8B ÉÐ. ÉÐ. ÉÐ. ÉÐ. September 20th,

38 FAT File Allocation Table When it is used, same as legacy FAT Not used when file contiguous Never used for cluster allocation FAT 32 has 32 bit cells, uses 28 bits exfat has 32 bit cells, uses 32 bits There is no 64 bit FAT Maximum clusters is With TexFAT 2 FAT Tables (2 Bitmaps) Addressed by pointer in VBR Size stored in VBR September 20th,

39 Cell Values in FAT Table 0x No significant meaning 0x Not a valid cell value 0xFFFFFFF6 Largest Value 0xFFFFFFF7 Bad Block 0xFFFFFFF8 Media Descriptor Fixed Disk 0xFFFFFFF9-0xFFFFFFFE 0xFFFFFFFE Not Defined 0xFFFFFFFF End of File (EOF) September 20th,

40 September 20th,

41 FAT Table Example Media Reserved Allocation Bit Map UP-Case Table Root Directory Offset F8 FF FF FF FF FF FF FF FF FF FF FF FF FF FF FF 0010 FF FF FF FF A C E September 20th,

42 Allocation Bitmap Keeps track of cluster allocation status Zero Free Cluster One Allocated Cluster 1 Byte = Tracking of 8 Clusters Bit Zero Byte Zero = Cluster 2 Cluster 0 & Cluster 1 are not defined Addressed by Directory Entry With TexFAT 2 of these (FAT Pairing) September 20th,

43 Data Hide Alert! The Allocation Bitmap and the UP-Case Table are stored as files, and provide hiding space in the metadata These files are static, typically won t move, and have slack space. Nothing prevents someone from moving these files elsewhere in the cluster heap, and actually making them larger September 20th,

44 September 20th,

45 Directories in exfat Root (VBR Pointer) Contains certain critical entries Almost unlimited in size Subdirectory (by File Entry) Contains file sets 256MiB Max size No physical. or.. entries Uses 16 Bit Unicode for strings Every Entry 32 bytes in size Entry 0x00 is end of directory Has capabilities for user entries September 20th,

46 Data Hide Alert! Manipulation of the Allocation Bitmap, and creation of user directory entries provides the capability of hiding a file system within the file system It may also be possible to hide data within the directory metadata itself September 20th,

47 Entry Type Type Field Offset (Bits) Size (Bits) In Use 7 1 Category 6 1 Importance 5 1 Code 0 5 September 20th,

48 Entry Type In Use: 0 Not in Use, 1- In Use Category: 0 Primary, 1 Secondary Importance: 0 Critical, 1 Benign Code: Identifies the entry September 20th,

49 Volume Label Directory Entry 0x83 or 0x03 Entry Primary Entry Only resident in Root Directory Contains the Volume Label 16 bit Unicode 0x03 means no volume label September 20th,

50 Volume Label Directory Entry Offset A B C D E F A D ƒ.e.x.f.a.t B K... Type Volume Name Length (10) Volume Label (exfat-128k) September 20th,

51 Allocation Bitmap Directory Entry 0x81 Entry Primary Entry Only resident in Root Directory Points to the Allocation Bitmap If TexFAT, then 2 of these Flag bits says which FAT/Bitmap Cluster Address of Bitmap Size of Bitmap September 20th,

52 Allocation Bitmap Directory Entry Offset A B C D E F F Type Cluster Address (Cluster 2) Size (63 bytes) September 20th,

53 UP-Case Table Directory Entry 0x82 Entry Primary Entry Only resident in Root Directory File names are case insensitive Used to fold file name Table has a checksum (32 bits) September 20th,

54 UP-Case Table Directory Entry Offset A B C D E F D D3 19 E CC Type Cluster Address (3) Table Checksum Length (0x16CC = 5,836) September 20th,

55 File Directory Entry Set Used to define a file May have 3 to 19 entries, or more 1 Primary, many Secondary Is considered an array Must be in order Must be contiguous (no gaps) Entire Set has Checksum September 20th,

56 File Directory Entry 0x85 or 0x05 Entry Primary Entry Set Checksum (16 bits) Not modified on file delete Secondary Count # Secondary entries that follow File Attributes Timestamps September 20th,

57 Timestamps & Time Zones 3 Timestamps (MAC) 32 bit DOS Date/Time Local Machine Time 10ms Offset (MC) TZ Offset (MAC) 15 minute increments 7 bit signed number ±16 hours Present with UTC support September 20th,

58 Timestamp Accuracy FAT32 Last Access Date only exfat Last Access Date/Time All DOS DATE/TIME Double Seconds 10ms adds ms to time 10ms only for Create/Modify September 20th,

59 Timestamp Reliability Timestamps appear to be updated when the file is created or modified. Last Accessed Timestamp appear to be updated when file is created or modified. Last Accessed Timestamp appear NOT modified on file read. Forensics Implication on MAC time analysis September 20th,

60 File Attributes Attribute Offset Size Mask Reserved Archive 5 1 0x20 Directory 4 1 0x10 Reserved1 3 1 System 2 1 0x04 Hidden 1 1 0x02 Read-Only 0 1 0x01 September 20th,

61 File Directory Entry Type # Secondary Entries Set Checksum (0x92D4) Attributes (0x0020 = Archive) Create Offset A B C D E F D B F1 62 BA 3A B A8 00 EC EC EC Accessed Modified 10ms Create 10ms Modified TZ Offset CMA EC = GMT-5 September 20th,

62 Formatted File Directory Entry Root Entry Type Read is: 85 Directory Entry Record Checksum: 92D4 Calculated Checksum is: 92D4 Size Directory Set (bytes): 160 Secondary Count 004 File Attributes: 0020 Archive Create Timestamp: 3B /06/ :18:08 18 Last Modified Timestamp: 3ABA62F1 05/26/ :23:34 Last Accessed Timestamp: 3B /06/ :18:08 10 ms Offset Create A ms Offset Modified 00 0 Time Zone Create EC 236 Value of tz is: GMT -05:00 Time Zone Modified EC 236 Value of tz is: GMT -05:00 Time Zone Last Accessed EC 236 Value of tz is: GMT -05:00 September 20th,

63 Stream Extension Directory Entry 0xC0 or 0x40 Entry Secondary Entry Length of Name Length of File (2 of them) Cluster address of first data block Name Search Hash value Secondary Flag FAT Invalid Allocation Possible September 20th,

64 Stream Extension Directory Entry Entry Flags (Alloc Possible/Fat Invalid) Length of File Name (0x28= 40) Name Hash (0x3CAD) Offset A B C D E F 0000 C AD 3C F 46 1D F 46 1D Cluster (5) Data Length 0x011d461f = 18,695,711 September 20th,

65 Parameters for Samples Bytes Per Sector: 2 to the 09 power is: 512 Sectors Per Cluster: 2 to the 08 power is: 256 Bytes per Cluster: (128K) September 20th,

66 Formatted Stream Extension Root Entry Type Read is: C0 Directory Entry Record, Stream Extension Secondary Flags: 03 Flag Bit 0: Allocation Possible Flag Bit 1: FAT Chain Invalid Length of UniCode Filename is: 40 Name Hash Value is: AD3C Stream Extension First Cluster 5 Cluster 5 is Allocated Stream Extension Data Length Bytes Slack: Clusters Used: 143 Stream Extension Valid Data Length Bytes Slack: Clusters Used: 143 September 20th,

67 File Name Extension Directory Entry 0xC1 or 0x41 Entry Secondary Entry Secondary Flags Allocation not possible FAT Invalid 15 Characters (30 bytes) of Name Name in 16 Bit Unicode In order (FAT32 LFN was reversed) Up to 17 max, total 255 character September 20th,

68 File Name Extension Directory Entry Offset A B C D E F 0000 C E Á.b.u.s.i.n.e.s F 00 6F F s._.o.f._.s.e.c C F 00 5F 00 Á.u.r.i.t.y._._ D D 00 b.u.s C B E 00 Á.3.2.k.b.p.s D m.p.3... File Name = business_of_security bus kbps.mp3 it b 3 September 20th,

69 Significance of not in use flag 0x05, 0x40 & 0x41 Entries Not in use may mean deleted files May also be reallocated rename Set Checksum not changed when entries marked not in use September 20th,

70 Summary exfat is a new generation of the FAT family of Microsoft File Systems The need for forensics tools will heat up in 2010 We don t have the right tools yet Documentation and support for exfat is scarce September 20th,

71 Q&A September 20th,

72 Contact Information Blog: rshullic.wordpress.com Blog: shullich.blogspot.com com September 20th,

73 References Sans Reading Room: s/rss/reverse_engineering_the_microsoft_exfat_file_s ystem_33274 Microsoft Patent: Microsoft Patent (June 25, 2009). Quick Filename Lookup Using Name Hash. Pub No. US 2009/ A1 Retrieved December 10, 2009 from September 20th,