Understanding APFS Putting Theory into Practice

Transcription

1 Understanding APFS Putting Theory into Practice Tim Standing Vice President Software Engineering - Mac OWC, Inc.

2 APFS and the T2 Chip APFS: How much does copy on write affect performance on HDDs? How does APFS allocate space for files? How does APFS organize files in a volume? T2 Chip: Flash Memory Controller Contains Secure Enclave Coprocessor Encryption Engine

3 Follow up from MacSysAdmin 2017

4 COPYING FILES WITH HFS+ Original File Extents Table Offset Length Nina s Birthday.mp4 Available space on disk: 90 GB 120 GB

5 COPYING FILES WITH HFS+ Original file with copy Original File Copy Extents Table Offset Length Nina s Birthday.mp4 Available space on disk: 80 GB 120 GB

6 COPYING FILES WITH HFS+ Editing 4 frames Original File Copy Extents Table Offset Length Edited frames Nina s Birthday.mp4 Available space on disk: 80 GB 120 GB

7 COPYING FILES WITH APFS Original File Extents Table Offset Length Nina s Birthday.mp4 Available space on disk: 90 GB 120 GB

8 COPYING FILES WITH APFS Original file with copy Original File Copy Extents Table Offset Length Nina s Birthday.mp4 Available space on disk: 90 GB 120 GB

9 COPYING FILES WITH APFS Original file after editing 4 frames Extents Table Offset Length shared with copy changed frame Old frames (removed from extents table) shared with copy changed frame shared with copy shared with copy changed frame shared with copy Edits saved separately, requiring extra extents changed frame shared with copy Nina s Birthday.mp4 120 GB

10 How to test the effect of copy on write with HDDs? Write to discontinuous parts of file 1) Create 10 GB File 2) Duplicate File 3) Write to File 4) Determine Time to Read Entire File

11 Time to read 10 GB file from HFS+ volume 160 Seconds to Read Entire File HFS Number of Discontiguous Writes

12 Time to read 10 GB file from HFS+ vs. APFS 160 Seconds to Read Entire File HFS+ APFS Number of Discontiguous Writes

13 Time to read 10 GB file from HDDs and SSDs (APFS Volumes) 160 Seconds to Read Entire File HDDs SSDs Number of Discontiguous Writes

14 Time to read 10 GB file from different HDDs (APFS Volumes) 160 Seconds to Read Entire File TB Consumer 12 TB Enterprise Number of Discontiguous Writes

15 Time to read different size files from HDDs (APFS Volumes) 160 Seconds to Read Entire File MB File 100 MB File 1 GB File 10 GB File Number of Discontiguous Writes

16 Automatic defragmention of APFS Volumes Status of automatic defragmentation Automatic defragmentation built into APFS Enabled via diskutil command in Terminal Enabling automatic defragmentation Disabled by default

17 How does APFS solve file system problems How does APFS allocate space How is it different than HFS+? Why is it faster? How does APFS organize files How does this affect the time taken to copy large numbers of files? How does this make snapshots possible?

18 Blocks in a file system Blocks in the volume (normally 4 KB) Blocks are numbered starting with zero. This is called the block address or block number

19 Allocating space for files on HFS+ Allocation bitmap - stored on disk Free Data Free Data Data Free Data Data Blocks in the volume

20 Allocating space for files on HFS+ How large is the bitmap for a 14 TB HFS+ volume? 14 TB = 14,000,000,000,000 Bytes = Blocks (4 KB) = Bytes for the allocation bitmap = 427 MB for the allocation bitmap Finding space for a new block involves a linear search through the bitmap On a full volume, this can take a lot of time Volumes start slowing down dramatically when they are 90% full

21 Allocating space for files on APFS Chunk Info Record Number Blocks Number Free Blocks Pointer to Bitmap 4 KB Bitmap Allocation bitmap - stored on disk Free Data Free Data Data Free Data Data Blocks on disk the volume

22 Allocating space for files on APFS Chunk Info Record Number Blocks Number Free Blocks Pointer to Bitmap 4 KB Bitmap Over 500 times faster at searching over filled blocks than HFS+ Checking one value determines if there is free space in the next 128 MB of the volume Bitmaps not allocated until first block is filled Allocation bitmap - stored on disk Free Data Free Data Data Free Data Data Blocks on disk the volume

23

24

25

26 Kungstorget, Göteborg

27 How are files organized in a file system? File systems are databases for file info and file data Like databases, usually implemented using B-Trees Entire file system tree cannot be kept in memory Must be fast to read sections of tree from disk File system B-Trees structures are usually = size of file system block

28 B-Tree records Each level in the tree contains records with keys and pointers Mark Mark is >= record we are searching for Tycho Tycho is >= record we are searching for Key Pointer

29 A simple B-Trees with 8 names Mark Tycho Justine Mark Patrick Tycho Information About Person Barry Justine Lauren Mark Nina Patrick Renee Tycho Barry Justine Lauren Mark Nina Patrick Renee Tycho

30 Finding Nina in the B-Tree Mark Tycho Tycho >= Nina Justine Mark Patrick Tycho Barry Justine Lauren Mark Nina Patrick Renee Tycho Barry Justine Lauren Mark Nina Patrick Renee Tycho

31 Finding Nina in the B-Tree Mark Tycho Justine Mark Patrick Tycho Patrick >= Nina Barry Justine Lauren Mark Nina Patrick Renee Tycho Barry Justine Lauren Mark Nina Patrick Renee Tycho

32 Finding Nina in the B-Tree Mark Tycho Justine Mark Patrick Tycho Barry Justine Lauren Mark Nina Patrick Renee Tycho Read Nina Info Barry Justine Lauren Mark Nina Patrick Renee Tycho

33 How does HFS+ organize files? Record in B-Tree 1 File system Block (4 KB) = records Directory ID & Filename Block Number To find info about a file or directory: 1. Search for record >= one you are looking for 2. Note the block number 3. Read file system block with that number 4. Repeat until you get the info record

34 Finding a file in the HFS+ B-Tree Find record >= one we are searching for 2. Read block #24

35 Finding a file in the HFS+ B-Tree 24 Block #24 60

36 Finding a file in the HFS+ B-Tree 24 Block #24 60 Block # Find record >= one we are searching for 2. Read block #60

37 Finding a file in the HFS+ B-Tree 24 Block #24 60 Block #60 98 Block #98 Info for file we are searching for 1. Find record >= one we are searching for 2. Read block #98

38 Finding a file in the HFS+ B-Tree 24 Block #24 60 Block #60 98 Block #98 Info for file we are searching for HFS+ B-Trees are usually no more than 6 levels deep Read less than 6 blocks to get info for a file in a directory Read less than 6 blocks for each directory level traversed

39 How does APFS organize files? Record in B-Tree 1 File system Block (4 KB) = records Directory ID & Filename Object ID To find info about a file or directory: 1. Search for record >= one you are looking for 2. Note the Object ID 3. Search through the Object ID B-Tree for the block number 4. Read file system block with that number 5. Repeat until you get the info record

40 Finding a file in the APFS B-Tree Find record >= one we are searching for 2. Search for Object # 18 in Object ID B-Tree

41 Finding a file in the APFS B-Tree 18 Block #24 Read Block #24 42

42 Finding a file in the APFS B-Tree 18 Block # Find record >= one we are searching for 2. Search for Object # 42 in Object ID B-Tree

43 Finding a file in the APFS B-Tree 18 Block #24 42 Block #98 Read Block #98 54

44 Finding a file in the APFS B-Tree 18 Block #24 42 Block # Find record >= one we are searching for 2. Search for Object # 54 in Object ID B-Tree

45 Finding a file in the APFS B-Tree 18 Block #24 42 Block #98 54 Block #121 Read Block #121 Info for file we are searching for

46 Finding a file in the APFS B-Tree 18 Block #24 42 Block #98 54 Traversing one level in the file B-Tree requires traversing Block #121 Read Block #121 Info for file we are searching for all levels of the Object ID B-Tree Read blocks to get info for 1 file in a directory

47 Finder Copy Speed - Duplicating ~130,000 files 800 HFS+ APFS 600 seconds ThunderBlade SSD HDD

48 Transaction IDs are stored in the Object ID B-Tree Looking up file with object ID 21 Obj 20 Tx Obj 21 Tx Obj 22 Tx 412 Obj 23 Tx 482 Obj 24 Tx 601 Block #62 Info for file before snapshot Volume Info: Transaction ID (Tx) 601

49 Taking a Snapshot Looking up file with object ID 21 Obj 20 Tx Obj 21 Tx Obj 22 Tx 412 Obj 23 Tx 482 Obj 24 Tx 601 Block #62 Info for file before snapshot Volume Info: Transaction ID 743 Snapshot Info: Transaction ID 601

50 Writing to a file after a snapshot Looking up file with object ID 21 Obj 19 Tx Obj 21 Tx Obj 21 Tx 654 Obj 22 Obj Tx 412 Tx 601 Block #62 Info for file before snapshot Volume Info: Transaction ID 743 Block #98 Info for file after snapshot Snapshot Info: Transaction ID 601

51 Accessing the latest version of a file Looking up file with object ID 21 Obj 19 Tx Obj 21 Tx Obj 21 Tx 654 Obj 22 Obj Tx 412 Tx 601 Volume Info: Transaction ID 743 Block #98 Info for file after snapshot Snapshot Info: Transaction ID 601

52 Accessing the file from the snapshot Looking up file with object ID 21 Obj 19 Tx Obj 21 Tx Obj 21 Tx 654 Obj 22 Obj Tx 412 Tx 601 Block #62 Info for file before snapshot Volume Info: Transaction ID 743 Snapshot Info: Transaction ID 601

53

54

55 How is the T2 chip used as a storage processor? Flash Memory Controller Contains Secure Enclave Coprocessor Encryption Engine

56 Flash memory controller Read/Write Channel Flash Memory Controller Flash Memory Module Cmd Data Idle Cmd Data Idle Cmd Data Idle Time

57 Faster flash memory controller Read/Write Channel Flash Memory Controller Flash Memory Flash Memory Flash Memory Module Module Module Cmd Data Cmd Data Cmd Data Cmd Data Cmd Data Cmd Data Cmd Data Time

58 T2 flash memory controller Read/WriteChannel Read/WriteChannel Flash Memory Modules

59 Secure Enclave coprocessor in the T2 chip At power on, Secure Enclave coprocessor: Loads Apple Root CA public key and L4 microkernel from ROM Verifies code signature of Bridge OS before loading it on the T2 chip

60 Secure Enclave coprocessor in the T2 chip At OS install time, If Secure Boot is enabled: A digest of the versions of installed system software, the machine ID and the startup volume ID are sent to Apple servers Apple servers sign the digest with the Apple private key and send the digest and signature back to the Mac. The digest and signature are stored in Secure Enclave coprocessor.

61 Secure Enclave coprocessor in the T2 chip At startup, if Secure Boot is enabled, Secure Enclave coprocessor: Checks that signature of digest is correct Checks that machine ID, system software version info and startup volume ID all match those found in the digest If the signature is invalid or the digest doesn t match the existing what is present on Mac, new copies of system software are downloaded, the new software is installed, a new digest is created and a new digest is sent to Apple for signing. The new signature and digest are stored.

62 More reading on Secure Enclave Chip and Secure Boot Documentation on Secure Boot on ios: Apple s Patent on the Secure Enclave Chip:

63 Review of FDE - Full Disk Encryption Early implementation 16BCD043 Encryption Software A FEB C9AB1 Unencrypted Data password1234 Encrypted Data User password is used as the key Mac Main Memory

64 Using MEK - Media Encryption Keys Setting up a password Generate Encrypted Key password1234 User s Password Encryption Software encrypted 256 bit key Key Encrypted encrypted 256 bit key Key Encrypted Key unencrypted 256 bit key Key written to disk Media Encryption Key Mac Main Memory

65 Using MEK - Media Encryption Keys Unlocking a volume Generate Unencrypted Key password1234 User s Password Encryption Software encrypted 256 bit key Key Encrypted encrypted 256 bit key Key Encrypted key unencrypted 256 bit key 256 bit key Key read from disk Media Encryption Key Mac Main Memory

66 Using MEK - Media Encryption Keys Reading encrypted data from disk 16BCD043 Encryption Software A FEB C9AB1 Unencrypted Encrypted Encrypted user data Data unencrypted 256 bit key Data read from disk Media Encryption Key Mac Main Memory

67 Using MEK - Media Encryption Keys Changing a password Encrypt key with new password A Better Password User s Password Encryption Software encrypted 256 bit key Key Encrypted encrypted 256 bit key Key Encrypted Key unencrypted 256 bit key Key written to disk Media Encryption Key Mac Main Memory

68 Using MEK - Media Encryption Keys Secure erasing volume Encryption Software Random number Random Number Random number Written to disk Mac Main Memory

69 Using MEK - Media Encryption Keys Unlocking a volume after secure erase Unencrypt Random Number A Better Password User s Encryption Software Random number Random number Password Random number Invalid key read from disk Mac Main Memory

70 Using MEK - Media Encryption Keys Reading data from disk after secure erase A346CD89 4AD98F32 2CB BFED91 Encryption Software 16BCD043 A FEB C9AB1 Random Data Invalid key Encrypted Data Mac Main Memory

71 Using MEK - Media Encryption Keys Advantages: Can change password simply Can secure erase data Securely erased data is protected by full strength random key Can have different passwords on separate volumes on the same disk Disadvantages: Encryption key is resident in memory whenever volume is unlocked

72 SED - Self Encrypting Drives Initializing disk with no password Encryption Hardware Random Number Generator unencrypted 256 bit key Media Encryption Key written to media Self Encrypting Drive

73 SED - Self Encrypting Drives Powering on a disk no password Encryption Hardware unencrypted 256 bit key Media Encryption Key unencrypted 256 bit key Media Encryption Key read from media Self Encrypting Drive

74 SED - Self Encrypting Drives Reading data without a password 16BCD043 Encryption Hardware A FEB C9AB1 Unencrypted Data unencrypted 256 bit key Media Encryption Key Encrypted user data read from disk Mac Main Memory Self Encrypting Drive

75 SED - Self Encrypting Drives Adding or changing a password password1234 User enters Encryption Hardware encrypted 256 bit key Key encrypted 256 bit key password unencrypted 256 bit key Encrypted Key Key Encrypted Key written to media Media Encryption Key Mac Main Memory Self Encrypting Drive

76 SED - Self Encrypting Drives Entering a password to unlock a drive Unencrypt Key password1234 User enters Encryption Hardware encrypted 256 bit key Key encrypted 256 bit key password unencrypted 256 bit key Encrypted Key Key Encrypted Key read from media Media Encryption Key Mac Main Memory Self Encrypting Drive

77 SED - Self Encrypting Drives Reading data 16BCD043 Encryption Hardware A FEB C9AB1 Unencrypted Data unencrypted 256 bit key Media Encryption Key Encrypted user data read from disk Mac Main Memory Self Encrypting Drive

78 Using SED - Self Encrypting Drives Advantages: Can change password simply Can secure erase data Securely erased data is protected by full strength random key No unencrypted key present in main memory Disadvantages: Only one password per disk

79 How does the T2 chip perform encryption??

80 What we know about encryption with the T2 chip Performs AES 256 bit encryption in hardware You can encrypt a volume with 500 GB of data in less than a second The Media Encryption Keys never leave the T2 chip APFS containers don t require a password Moving the SSD modules from one imac Pro to another makes them unreadable If SSD modules are removed from an imac Pro and replaced, they are no longer readable if the imac Pro has had other SSD modules installed in the interim

81 How T2 chip could implement encryption AFP Volume Info Object Object ID Container #1 401 Object ID 401 T2 Chip - power off Stored Key Unencrypted key Media Encyption Key Invalid Key Volume #1 (encrypted) Encrypted key Invalid Key Encryption Hardware Volume # Unencrypted key Unencrypted key Invalid Key Invalid Key Volume #3 528 Secure Enclave Coprocessor

82 How T2 chip could implement encryption AFP Volume Info Object Object ID Container #1 401 Object ID 401 T2 Chip - power on Stored Key Unencrypted key Media Encyption Key Unencrypted key Volume #1 (encrypted) Encrypted key Invalid Key Encryption Hardware Volume # Unencrypted key Unencrypted key Unencrypted key Unencrypted key Volume #3 528 Secure Enclave Coprocessor

83 How T2 chip could implement encryption Read object 401 data 16BCD043 A T2 Chip FEB C9AB1 Read Command Object ID Stored Key Media Encyption Key Encrypted data Offset: 0x Unencrypted key Unencrypted key read from media Length: 0x Encrypted key Invalid Key Encryption Hardware Buffer: 0xA Unencrypted key Unencrypted key Object ID: Unencrypted key Unencrypted key Secure Enclave Coprocessor Unencrypted data

84 How T2 chip could implement encryption Read object 526 data before password entered 16BCD043 A T2 Chip FEB C9AB1 Read Command Object ID Stored Key Media Encyption Key Encrypted data Offset: 0x Unencrypted key Unencrypted key read from media Length: 0x1000 Buffer: 0xA Encrypted key Unencrypted key Invalid Key Unencrypted key Encryption Hardware 16BCD043 A Object ID: Unencrypted key Unencrypted key FEB79136 Secure Enclave 854C9AB1 Coprocessor Invalid data

85 How T2 chip could implement encryption Enter password for object 526 T2 Chip Object ID Stored Key Media Encyption Key Password Command 401 Unencrypted key Unencrypted key Password: Encrypted key Unencrypted key Encryption Hardware Object ID: Unencrypted key Unencrypted key 528 Unencrypted key Unencrypted key Secure Enclave Coprocessor

86 How T2 chip could implement encryption Read object 526 data after password entered 16BCD043 A T2 Chip FEB C9AB1 Read Command Object ID Stored Key Media Encyption Key Encrypted data Offset: 0x Unencrypted key Unencrypted key read from media Length: 0x Encrypted key Unencrypted key Encryption Hardware Buffer: 0xA Unencrypted key Unencrypted key Object ID: Unencrypted key Unencrypted key Secure Enclave Coprocessor Unencrypted data

87 How T2 chip could implement encryption Encrypt object 528 T2 Chip Object ID Stored Key Media Encyption Key Encrypt Command 401 Unencrypted key Unencrypted key Password: Encrypted key Unencrypted key Encryption Hardware Object ID: Unencrypted key Unencrypted key 528 Encrypted key Unencrypted key Secure Enclave Coprocessor

88 How T2 chip could implement encryption Are all Media Encryption Keys the same? T2 Chip Object ID Stored Key Media Encyption Key 401 Unencrypted key 0x Encrypted key 0x Encryption Hardware 527 Unencrypted key 0x Encrypted key 0x Secure Enclave Coprocessor

89 How T2 chip could implement encryption Read object 526 data before password entered 16BCD043 A T2 Chip FEB C9AB1 Read Command Object ID Stored Key Media Encyption Key Encrypted data Offset: 0x Unencrypted key 0x read from media Length: 0x1000 Buffer: 0xA Encrypted key Unencrypted key Invalid Key 0x Encryption Hardware 16BCD043 A Object ID: Unencrypted key 0x FEB79136 Secure Enclave 854C9AB1 Coprocessor Invalid data

90 How T2 chip could implement encryption Reading password protected data without a password 16BCD043 A T2 Chip FEB C9AB1 Read Command Object ID Stored Key Media Encyption Key Encrypted data Offset: 0x Unencrypted key 0x read from media Length: 0x Encrypted key Invalid Key Encryption Hardware Buffer: 0xA Unencrypted key 0x Object ID: Unencrypted key 0x Secure Enclave Coprocessor Unencrypted data

91 Rick Rockhold VP Software Engineering, Windows OWC, Inc.

92 Mike Cobb Director of Engineering DriveSavers, Inc.

93 Q & A