BTREE FILE SYSTEM (BTRFS)

BTREE FILE SYSTEM (BTRFS)

What is a file system? It can be defined in different ways A method of organizing blocks on a storage device into files and directories. A data structure that translates the physical view of a disc into a logical structure. The way in which file s are named and where they are placed logically for storage and retrieval.

NEED FOR DIFFERENT FILESYSTEMS This depends on Requirements of OS Required level of security and efficiency Minimize access time (eg.: cdfs) Exploit device features ( eg.: flash filesystem) No seek time Avoid continuous writes Block wise erasing

FILE SYSTEM TERMINOLOGY Inode : A data structure holding information about files in a Unix file system. Blocks : Uniformly sized unit of data storage for filesystem. Extents : bunch of contiguous physical blocks.

FILE SYSTEM TERMINOLOGY cont d Super block : Block that describes the entire file system Directory : A file that lists the files and directories contained within, and their associated inode. Journaling : Logging of filesystem transactions

WHAT IS BTRFS? Next Gen. File system for Linux Also called BUTTER FS or BETTER FS Motivated from zfs of Solaris Support for copy on write Unlike zfs which is licensed under CDDL(Common Development and Distribution License), Btrfs is GPL licensed Only test versions are released(accepted in 2.6.29) It have lots and lots of features compared to extended series

DATA AND METADATA STRUCTURE Uses COW (copy on write) tree or Rodeh's btrees Advantages of COW btrees Increase in the overall depth is infrequent Minimum number of rearrangement Search could be done in O(log2N) Entire tree need not be in memory COW facility speeds up operations

BTRFS DATA STRUCTURES Btrfs internally only knows about three data structures Block header - btrfs_header Key - btrfs_key Item - btrfs_item

btrfs_header The block header contains checksum for the block contents uuid of the filesystem level of the block in the tree block number where this block is supposed to live struct btrfs_header { u8 csum[32 bytes]; u8 fsid[16]; le64 blocknr; le64 generation; le64 owner; le16 nritems; le16 flags; u8 level; }

btrfs_key Key contains unique object id analogous to inode number in ext series Object id is Most Significant Bits of key results in grouping together all info associated with particular object id struct btrfs_key { u64 objectid; u32 type; u64 offset ; }

btrfs_key cont d Offset field of the key indicates the byte offset for a particular item. Inode has offset value 0 Type field gives type of the item can be inode, file data etc.

btrfs_item Key key describing the item Offset it is the offset of item Size size of item * Design has under gone massive changes. Fields may have been changed. struct btrfs_item { Struct btrfs_key key; le32 offset; le16 size; }

Example of Initialized data structures inside the filesystem tree.

BTRFS NONLEAF NODES Contains [ key, block headers ] pairs Key tells you where to look for the item you want Header tell where next node or leaf in the btree is located on disk - given by blocknr field of header

BTRFS LEAF NODES Broken up into two sections,item header and data, that grow towards each other Data is variably sized Offset and size filed gives the location of data associated with an item

64 MB 1 MB Rest FILES Inline extents are used for small files Large files are stored in extents and are [ disk block, disk num blocks ] pair record the area of disk for the file Extnets store logical offset allows write in the middle of extent Old data New data Old data

DIRECTORY Directories are indexed in two ways using hash keys for filename look up and directory item s key is composed of as follows Second indexing is used by readdir function to retrieve information in inode

Extent Block Groups The disk up into chunks of 256MB or more For each chunk, information about the number of blocks available is recorded Flags indicate whether the extent is for data or metadata At creation time disk id broken up into 33% for metadata and 66% for data alternatively

BTRFS organization of files, inodes, ditectories, pointers, etc Ext file systems Btrfs file systems

Metadata Structure Of Btrfs Btrfs uses 6 types of COW trees to keep track of data and metadata They are sorted using 132 bit key value Key is formed from three components Most significant 64 bits is the object id Next 8bits is from the type fields Remaining 64 bits are for objects own use

FILE SYSTEM TREE User-visible files and directories live in file system tree Files and directories have a back-reference to parent There is one file system tree per sub volume Small files are kept in inline extent data item Otherwise data is kept outside the tree in extent and extent data item in the tree is used to track them

EXTENT ALLOCATION TREE Used to track space usage by extents In-memory tree of page-sized bitmaps to speed up allocations Items in the extent allocation tree do not have object ids Extent items contain a back-reference to the tree node or file occupying that extent Extent tree is COW writes may result in change of extent tree

CHECKSUM TREE Checksums for data and metadata are calculated and stored in checksum tree as checksum item There is one checksum item per contiguous run of allocated blocks CRC-32C check summing is used [crc32c -x32 + x28 + x27 + x26 + x25 + x23 + x22 + x20 + x19 + x18 + x14 + x13 + x11 + x10 + x9 + x8 + x6 + 1]

LOG TREE Log tree is created for each subvolume Used to prevent the heavy workloads redundant i/o operations result from fsync() Log tree journals fsync initiated copy on writes Log tree items are replayed and deleted at the next full tree commit or at remount if file system crash occurs

CHUNKS AND DEVICE TREES Chunks may be mirrored or striped across multiple devices But file system sees only the logical address space that chunk are mapped into The mapping is stored as device and chunk map items it the chunk tree Device tree is the inverse of the chunk tree Byte ranges of block devices back to individual chunks The chunks containing the chunk tree, the root tree, device tree and extent tree are always mirrored

ROOT TREE Records the root block for the extent tree and the root blocks and names for each sub volume and snapshot tree At transaction commit root block pointers are updated Root tree assigns them object id This object id can be used for back referencing

KEY FEATURES Key features of btrfs are Dynamic Inode Allocation Writable Snapshots And Subvolumes Copy On Write Compression Online Resizing Online File System Check And Defragmentation Inplace Ext3 Conversion Multiple Device Support Space Efficient Packing Of Small Files

DYNAMIC INODE ALLOCATION Inodes are created only when they are needed Ext series uses static inode tables only advantage is corruption detection Static inode table wastes space, slows down fsck, it needs inode blocks to be at fixed location In dynamic allocation inode can be kept close to the file Corruption detection can be implemented using check summing inode blocks

WRITABLE SNAPSHOT AND SUBVOLUME Allows creation of writable snapshots They can be used for back ups and testing transactions Occupied space actually increases only when data is written to snapshot Sub volumes are named btrees They have inodes inside the tree of tree roots Snapshots are subvolumes whose root is shared with another They can be mounted as unique root to prevent access

COPY ON WRITE Optimization strategy Each process requesting a resource is given pointers to same resource Implemented as marking resource as read only and any modification generates a kernel interrupt When performing COW operation to btree nodes reference count of all nodes it point is incremented When transaction commits new tree pointer is inserted in root tree keeps progress records to keep track of transaction

COMPRESSION Uses zlib compression to improve perfomance Few bits of type field could be used to indicate compression In case of compression uncompressed size is also stored If compression failed due to some reason it is flagged to avoid further compression requests Back ground compression can be done when fs is idle in case of slow compression

ONLINE RESIZING Allows file system to be resized while it is mounted It is implemented in btrfsprogs, a tool similar to e2fsprogs Online resizing is still under development

Online FSCK & Defragmentation Btrfsck can be done while it is mounted only ext4 in ext series allows that In ext it is difficult to get fs back online after something has gone wrong Btrfs aims to be tolerant of invalid metadata Offline btrfsck is made to ensure safe mounting Btrfsck verifies extent allocation maps and reference counts The cost paid for this efficiency is more ram. It requires 3x ram than ext2fsck

INPLACE EXT3 CONVERSION Features that make conversion possible are Btrfs metadata need not be in fixed locations COW allows unmodified copy to me maintained allows undo the conversion Libe2fs is used to read ext3 metadata Uses the free blocks in the Ext3 filesystem to hold the new Btrfs Btrfs files points to the same blocks used by the Ext3 files

INPLACE EXT3 CONVERSION cont d

MULTIPLE DEVICE SUPPORT Currently you can have metadata in following manner In current filesystems, to create a RAID or add RAID0 - metadata is appended across all new RAID level it needs to use lvm to create devices present new volume and then use H/W RAID card of RAID-1 - metadata is mirrored across all S/W RAID to combine volumes devices present RAID-10 - metadata is appended and mirrored across all devices present SINGLE - metadata is mirrored on a single device Metadata is mirrored across all devices But btrfs can do RAID as part of filesystem By default Data is stripped across all availale devices

SPACE EFFICIENT PACKING OF SMALL FILES Small files are packed inside the inline extends If compression is used for data that are larger to keep in inline extends and smaller to keep in external extends can be placed in inline extends