Looking up objects in Pastry

Transcription

1 Review: Pstry routing tbles b c d e f b c d e f b c d e f b c d e f Row0 Row 1 Row 2 Row 3 Routing tble of node with ID i =1fc s - For ech prefi p of i, nd ech digit d [0 f], hs contct with ID prefi pd

2 Looking up objects in Pstry d471f1 d41c d47c4 d42b d4213f d13d3 1fc locte(d41c) Corrects one digit t time - Queries nodes with IDs: d13d3, d4213f, d42b - Then use lef set to find node with nerest ID to trget

3 Joining the system Must know of one eisting node in system - Query it nd other nodes to find node closest to your ID d471f1 d41c d47c4 d42b ddnode(d41c) d4213f d13d3 1fc Initilize lef tble from node closes to ID - Will know lmost complete lef set for new node

4 Initilizing routing tble Cn t initilize routing tble from closest node - Eg, 1fffff s closest node might be y But cn fill up routing tble from intermedite nodes - Cn use entire first row of first node contcted - Use second row of second node contcted, since sme first digit s joining node Once join procedure complete, cn issue queries - New node knows enough to route to nd ID But wht bout queries to IDs ner new node?

5 Fiing up stte for join New node must fi other nodes routing tbles s well s initilize its own For correctness, need to fi up neighbor s lef sets - Esy, node cn contct them fter initilizing its own lef set - If lef sets correct, routing works, but could tke O(N) hops Updting other nodes routing tbles: - Old routing tbles either correct, or missing entry new node could fill - Automticlly fill holes s side ffect of lookups - New node sends its stte to ech node in its routing tble - Nodes periodiclly query to try to fill holes in their tbles

6 Node filure Nodes cn fil without wrning - Other node s routing tbles & lefs sets point to ded node Routing tble: Detect timeout, tret s empty slot - Route to numericlly closest vilble - Repir: Ask ny node on sme row for its contct Or sk ny node below, since ll will hve correct prefi Lef sets: Node closest to trget could be ded - Need to find net closest - Tht s why lef sets not just one neighbor (O(log N)) - Esy to updte lef sets by contcting other nerby nodes

7 How relible is Pstry? For correctness, only need lef sets Assume independent node filures - Ech node fils with probbility p in mintennce intervl - Sy lef set contins L vlues - Probbility of being cut off is p L - So for lrge N, if L log N, pretty good Is independent filure resonble ssumption? - Good tht nodeid = MD(IP Address) - Proimity in ID spce not correlted with physicl proimity - But big network outges, synchronized renumbering correlted

8 Loclity Lookup tkes O(log n) hops - But hops could be long (NYC Germny LA) Note: Mny options for top levels of routing tble - Cn chose ny node with correct prefi - So pick nodes tht re close to you to speed lookup - But mkes it hrder to ssume independent filures Continuously djust routing tble for loclity - Asks current entry for tht entry s complete tbles - Ping suitble nodes from other node s tbles - Use them insted of current entry if ping sys closer No choice for lef sets, though

9 Short routes property Loclity optimiztion helps recursive lookups - New node will know of nodes close to it - Very good if tringle property holds (X close to Y nd Y close to Z = X close to Z) - Often does hold, but not lwys This is known s short routes property - Individul hops re lower ltency - But less nd less choice (lower node density) s you get close in ID spce - So lst few hops likely to be very long - You don t end up close to the inititing node, just get there more quickly

10 Scribe Pstry cn be used to form multicst trees - Hsh nme of multicst group to get ID - Node closest to ID is rendez-vous point or root - To multicst messge, deliver it do RP, which sends it down the tree Form multicst tree by routing JOIN msgs to ID - Ech node keeps trck of groups + children for ech group - On receipt of JOIN messge, dd sender to children - If child joins new group, send join to prent (prent is just net hop towrds ID) Send just proceeds from RP to leves - Senders cche IP ddress of RP to sve upwrds routing Leve protocol similr to join

11 Scribe loclity Short routes property helps multicst trees - Towrds leves, prents re in high-levels of routing tble - These re precisely the contcts with best loclity - So often delivering messges to nerby nodes - Which my well reduce link stress (eg, node 1bc t NYU will chose node 2defy t NYU over frther nodes) Bottleneck remover lgorithm for overlod - Node my decide it is forwrding too mny copies - Mesures children & boots furthest wy - Booted node effectively gets pushed down the tree

12 Relibility & filure Scribe sends messges over TCP - But doesn t gurntee relibility - Nodes cn crsh nd leve system bruptly - In fct, Sbribe itself doesn t gurntee relible delivery Detect filures using hertbet messges - Ech non-lef node periodiclly sends hertbet to children - Any multicst messge serves s hertbet - So only need etr trffic when group quiescent - Upon timeout, route round filed node in Pstry Must replicte root stte in cse root fils - Typiclly replicted on nerest nodes to ID

13 Relible/ordered multicst Cn build relible/ordered multicst on Scribe Source ssigns sequence number to ech messge - Nodes do not send messges out of order - Wit for ll previous messges before sending net - After fult+repir, you know wht you re missing Nodes buffer old messges - Keep round for longer thn detect+repir time - So when you repir, cn request messges you missed

14 Splitstrem Problem: Scribe mkes uneven use of resources In fully-blnced tree w height h, fnout f - f h lef nodes consume no upstrem b/w - (f h 1)/(f 1) internl nodes consume f strem b/w - Eg, with f = 1, < 10% of nodes crry forwrding lod! Better pproch: Stripe dt over forest of trees - Ech node is lef in some, internl in others - Could round-robin pckets down multicst trees - Or could stripe t the bit level - One tree could be prity bit, to survive filure

15 Interior-node-disjoint trees Wnt to void filure ffecting multiple strems - Eg, sy node n is your ncestor in multiple trees - If n fils, you lose multiple strems (so prity won t help) Solution: Ech node is interior in only one tree - Sy digits re in bse 1 - Cn chieve by hving 1 trees, ech with group ID tht strts with different digit - Cn only be interior node if group ID nd you hve t lest one-digit prefi in common

16 CFS Another ppliction of P2P systems Ide: Replicte widely stored dt in DHT - Eg, Linu distribution - Cre lot bout dt integrity no tmpering! CFS coopertive file system is P2P file system - Red-mostly file system - Publish opertion breks into blocks - Spreds chunks ll round DHT - Digitlly sign entire file system for integrity

17 Emple: Publishing 2 blocks of dt Inde H(dt) H(dt) version epirtion signture dt dt Digitlly sign version & hshes of blocks - Cn verify one block without hving the other - Two blocks must come from sme version of file Generlize technique to n entire file system

18 directory nme i-number Trditionl FS dt structures inode metdt dt ptr dt ptr indirect block contents dt dt indirect ptr double indir dt ptr dt ptr dt dt In dtbse rbitrry key cn replce disk loction

19 CFS dt structures RO-inode contents metdt dt directory H(dt) H(dt) dt indirect nme H(ind blk) block H(inode) H(dbl ind) H(dt) H(dt) dt dt Inde ll dt & metdt with cryptogrphic hsh

20 CFS sclbility & relibility CFS built on Chord not Pstry, but ides similr Blocks must be replicted for relibility - Esy: Store ech item t k successor nodes round circle Blocks must be replicted for sclbility - Eg, Imgine everybody reds the sme block - Don t wnt to overlod poor successor node Solution: Store blocks long the lookup pth - Suppose you re looking up block B on node n 0 - You my trverse nodes n 3, n 2, n 1, n 0 to get B - Now store B on n 1 - Net lookup tht converges t n 1 will store on prev, etc