Lecture Notes. UNIX Fast File System Log-Structured File System Analysis and Evolution of Journaling File Systems

Transcription

1 Advanced Tpics in Cmputer Systems, CS262A Prf. Eric Brewer FFS/LFS Lecture Ntes UNIX Fast File System Lg-Structured File System Analysis and Evlutin f Jurnaling File Systems I. Backgrund i-nde: structure fr per-file metadata (unique per file) cntains: wnership, permissins, timestamps, abut 10 data-blck pinters They frm an array, indexed by i-number. S each i-nde has a unique i-number. Array is explicit fr FFS, implicit fr LFS (its i-nde map is cache f i-ndes indexed by i-number) indirect blcks: i-nde nly hlds a small number f data blck pinters fr larger files, i-nde pints t an indirect blck (hlds 1024 entries fr 4-byte entries in a 4K blck), which in turn pints t the data blcks. Can have multiple levels f indirect blcks fr even larger files II. A Fast File System fr UNIX Original UNIX file system was simple and elegant, but slw. Culd nly achieve abut 20 KB/sec/arm; ~2% f 1982 disk bandwidth Prblems: blcks t small cnsecutive blcks f files nt clse tgether (randm placement fr mature file system) i-ndes far frm data (all i-ndes at the beginning f the disk, all data after that) i-ndes f directry nt clse tgether n read-ahead Aspects f new file system: 4096 r 8192 byte blck size (why nt larger?) large blcks and small fragments disk divided int cylinder grups each cntains superblck, i-ndes, bitmap f free blcks, usage summary inf 1 9/2/09

2 Nte that i-ndes are nw spread acrss the disk: keeps i-nde near file, i-ndes f a directry tgether cylinder grups ~ 16 cylinders, r 7.5 MB cylinder headers spread arund s nt all n ne platter Tw techniques fr lcality: dn t let disk fill up in any ne area paradx: t achieve lcality, must spread unrelated things far apart nte: new file system gt 175KB/sec because free list cntained sequential blcks (it did generate lcality), but an ld system has randmly rdered blcks and nly gt 30 KB/sec Specific applicatin f these techniques: gal: keep directry within a cylinder grup, spread ut different directries gal: allcate runs f blcks within a cylinder grup, every nce in a while switch t a new cylinder grup (jump at 1MB). layut plicy: glbal and lcal glbal plicy allcates files & directries t cylinder grups. Picks ptimal next blck fr blck allcatin. lcal allcatin rutines handle specific blck requests. Select frm a sequence f alternative if need t. Results: 20-40% f disk bandwidth fr large reads/writes x riginal UNIX speeds. Size: 3800 lines f cde vs in ld system. 10% f ttal disk space unusable (except at 50% perf. price) Culd have dne mre; later versins d. Enhancements made t system interface: (really a secnd mini-paper) lng file names (14 -> 255) advisry file lcks (shared r exclusive); prcess id f hlder stred with lck => can reclaim the lck if prcess is n lnger arund symblic links (cntrast t hard links) atmic rename capability (the nly atmic read-mdify-write peratin, befre this there was nne) 2

3 disk qutas Culd prbably have gtten cpy-n-write t wrk t avid cpying data frm user- >kernel. (wuld need t cpies nly fr parts that are nt page aligned) Overallcatin wuld save time; return unused allcatin later. Advantages: 1) less verhead fr allcatin, 2) mre likely t get sequential blcks 3 key features f paper: parameterize FS implementatin fr the hardware it s running n. measurement-driven design decisins lcality wins A majr flaws: measurements derived frm a single installatin. ignred technlgy trends A lessn fr the future: dn t ignre underlying hardware characteristics. Cntrasting research appraches: imprve what yu ve gt vs. design smething new. III. Lg-Structured File System Radically different file system design. Technlgy mtivatins: CPUs utpacing disks: I/O becming mre-and-mre f a bttleneck. Big memries: file caches wrk well, making mst disk traffic writes. Prblems with current file systems: Lts f little writes. Synchrnus: wait fr disk in t many places. (This makes it hard t win much frm RAIDs, t little cncurrency.) 5 seeks t create a new file: (rugh rder) file i-nde (create), file data, directry entry, file i-nde (finalize), directry i-nde (mdificatin time). Basic idea f LFS: Lg all data and meta-data with efficient, large, sequential writes. Treat the lg as the truth (but keep an index n its cntents). Rely n a large memry t prvide fast access thrugh caching. 3

4 Data layut n disk has tempral lcality (gd fr writing), rather than lgical lcality (gd fr reading). Why is this a better? Because caching helps reads but nt writes! Tw ptential prblems: Lg retrieval n cache misses. Wrap-arund: what happens when end f disk is reached? N lnger any big, empty runs available. Hw t prevent fragmentatin? Lg retrieval: Keep same basic file structure as UNIX (inde, indirect blcks, data). Retrieval is just a questin f finding a file s inde. UNIX indes kept in ne r a few big arrays, LFS indes must flat t avid update-inplace. Slutin: an inde map that tells where each inde is. (Als keeps ther stuff: versin number, last access time, free/allcated.) Inde map gets written t lg like everything else. Map f inde map gets written in special checkpint lcatin n disk; used in crash recvery. Disk wrap-arund: Cmpact live infrmatin t pen up large runs f free space. Prblem: lng-lived infrmatin gets cpied ver-and-ver. Thread lg thrugh free spaces. Prblem: disk will get fragmented, s that I/O becmes inefficient again. Slutin: segmented lg. Divide disk int large, fixed-size segments. D cmpactin within a segment; thread between segments. When writing, use nly clean segments (i.e. n live data). Occasinally clean segments: read in several, write ut live data in cmpacted frm, leaving sme fragments free. Try t cllect lng-lived infrmatin int segments that never need t be cleaned. Nte there is nt free list r bit map (as in FFS), nly a list f clean segments. Which segments t clean? Keep estimate f free space in each segment t help find segments with lwest 4

5 utilizatin. Always start by lking fr segment with utilizatin=0, since thse are trivial t clean... If utilizatin f segments being cleaned is U: write cst = (ttal bytes read & written)/(new data written) = 2/(1-U). (unless U is 0). write cst increases as U increases: U =.9 => cst = 20! need a cst f less than 4 t 10; => U f less than.75 t.45. Hw t clean a segment? Segment summary blck cntains map f the segment. Must list every i-nde and file blck. Fr file blcks yu need {i-number, blck #} T clean an i-nde: just check t see if it is the current versin (frm i-nde map). If nt, skip it; if s, write t head f lg and update i-nde map. T clean a file blck, must figure ut it if is still live. First check the UID, which nly tells yu if this file is current (UID nly changes when is deleted r has length zer). Nte that UID des nt change every time the file is mdified (since yu wuld have t update the UIDs f all f its blcks). Next yu have t walk thrugh the i-nde and any indirect blcks t get t the data blck pinter fr this blck number. If it pints t this blck, then mve the blck t the head f the lg. Simulatin f LFS cleaning: Initial mdel: unifrm randm distributin f references; greedy algrithm fr segmentt-clean selectin. Why des the simulatin d better than the frmula? Because f variance in segment utilizatins. Added lcality (i.e. 90% f references g t 10% f data) and things gt wrse! First slutin: write ut cleaned data rdered by age t btain ht and cld segments. What prg. language feature des this remind yu f? Generatinal GC. Only helped a little. Prblem: even cld segments eventually have t reach the cleaning pint, but they drift dwn slwly. tying up lts f free space. D yu believe that s true? Slutin: it s wrth paying mre t clean cld segments because yu get t keep the free space lnger. Better way t think abut this: dn t clean segments that have a high d-free/dt (first derivative f utilizatin). If yu ignre them, they clean themselves! LFS uses age as an apprximatin f d-free/dt, because the latter is hard t track directly. New selectin functin: MAX(T*(1-U)/(1+U)). Resulted in the desired bi-mdal utilizatin functin. LFS stays belw write cst f 4 up t a disk utilizatin f 80%. 5

6 Checkpints: Just an ptimizatin t rll frward. Reduces recvery time. Checkpint cntains: pinters t i-nde map and segment usage table, current segment, timestamp, checksum (?) Befre writing a checkpint make sure t flush i-nde map and segment usage table. Uses versin vectr apprach: write checkpints t alternating lcatins with timestamps and checksums. On recvery, use the latest (valid) ne. Crash recvery: Unix must read entire disk t recnstruct meta data. LFS reads checkpint and rlls frward thrugh lg frm checkpint state. Result: recvery time measured in secnds instead f minutes t hurs. Directry peratin lg == lg intent t acheive atmicity, then red during recvery, (und fr new files with n data, since yu can t red it) Directry peratin lg: Example f intent + actin : write the intent as a directry peratin lg, then write the actual peratins (create, link, unlink, rename) This makes them atmic On recvery, if yu see the peratin lg entry, then yu can REDO the peratin t cmplete it. (Fr new file create with n data, yu UNDO it instead.) => lgical REDO lgging An interesting pint: LFS efficiency isn t derived frm knwing the details f disk gemetry; implies it can survive changing disk technlgies (such variable number f sectrs/track) better. Key features f paper: CPUs utpacing disk speeds; implies that I/O is becming mre-and-mre f a bttleneck. Write FS infrmatin t a lg and treat the lg as the truth; rely n in-memry caching t btain speed. Hard prblem: finding/creating lng runs f disk space t (sequentially) write lg recrds t. Slutin: clean live data frm segments, picking segments t clean based n a cst/benefit functin. Sme flaws: Assumes that files get written in their entirety; else wuld get intra-file fragmentatin in LFS. If small files get bigger then hw wuld LFS cmpare t UNIX? 6

7 A Lessn: Rethink yur basic assumptins abut what s primary and what s secndary in a design. In this case, they made the lg becme the truth instead f just a recvery aid. IV. Analysis and Evlutin f Jurnaling File Systems Jurnaling file systems: Write-ahead lgging: cmmit data by writing it t lg, synchrnusly and sequentially Unlike LFS, then later mved data t its nrmal (FFS-like) lcatin; this write is called checkpinting and like segment cleaning, it makes rm in the (circular) jurnal Better fr randm writes, slightly wrse fr big sequential writes All reads g the the fixed lcatin blcks, nt the jurnal, which is nly read fr crash recvery and checkpinting Much better than FFS (fsck) fr crash recvery (cvered belw) becuase it is much faster Ext3 filesystem is the main ne in Linux; RieserFS was becming ppular Three mdes: writeback mde: jurnal nly metadata, write back data and metadata independently metadata may thus have dangling references after a crash (if metadata written befre the data with a crash in between) rdered mde: jurnal nly metadata, but always write data blcks befre their referring metadata is jurnaled. This mde generally makes the mst sense and is used by Windws NTFS and IBM s JFS. data jurnaling mde: write bth data and metadata t the jurnal Huge increase in jurnal traffic; plus have t write mst blcks twice, nce t the jurnal and nce fr checkpinting (why nt all?) Crash recvery: Lad superblck t find the tail/head f the lg Scan lg t detect whle cmmited transactins (they have a cmmit recrd) Replay lg entries t bring in-memry data structures up t date This is called red lgging and entries must be idemptent Playback is ldest t newest; tail f the lg is the place where checkpinting stpped Hw t find the head f the lg? Sme fine pints: Can grup transactins tgether: fewer syncs and fewer writes, since ht metadata may changes several times within ne transactin Need t write a cmmit recrd, s that yu can tell that all f the cmpund transactin made it t disk ext3 lgs whle metadata blcks (physical lgging); JFS and NTFS lg lgical recrds instead, which means less jurnal traffic head f line blcking: cmpund transactins can link tgether cncurrent streams (e.g. 7

8 frm different apps) and hinder asynchrnus apps perfrmance (Figure 6). This is like having n left turn lane and waiting n the car in frnt f yu t turn left, when yu just want t g straight. Distinguish between rdering f writes and durability/persistence: careful rdering means that after a crash the file system can be recvered t a cnsistent past state. But that state culd be far in the past in the case f JFS. 30 secnds behind is mre typical fr ext3. If yu really want smething t be durable yu must flush the lg synchrnusly. Semantic Blck-level Analysis (SBA): Nice idea: interpse special disk driver between the file system and the real disk driver Prs: simple, captures ALL disk traffic, can use with a black-bx filesystem (n surce cde needed and can even use via VMWare fr anther OS), can be mre insightful than just a perfrmance benchmark Cns: must have sme understanding f the disk layut, which differs fr each filesystem, requires a great deal f inference; really nly useful fr writes T use well, drive filesystem with smart applicatins that test certain features f the filesystem (t make the inference easier) Semantic trace playback (STP): Uses tw kinds f interpsitinl 1) SBA driver that prduces a trace, and 2) user-level library that fits between the app and the real filesystem User-level library traces dirty blcks and app calls t fsync Playback: given the tw traces, STP generates a timed set f cmmands t the raw disk device. This sequence can be timed t understand perfrmance implicatins. Claim: faster t mdify the trace than t mdify the filesystem and simpler and less errr-prne than building a simulatr Limited t simple FS changes Best example usage: shwing that dynamically switching between rdered mde and data jurnaling mde actually gets the best verall perfrmance. (Use data jurnaling fr randm writes.) 8