Recap Consistent cuts. CS514: Intermediate Course in Operating Systems. What time is it? But what does time mean? Drawing time-line pictures:

Transcription

1 CS514: Interediate Course in Oerating Systes Professor Ken iran Vivek Vishnuurthy: T Reca Consistent cuts On Monday we saw that sily gathering the state of a syste isn t enough Often the state includes tricky relationshis Consistent cuts are a way of collecting state that could have arisen concurrently in real-tie What tie is it? In distributed syste we need ractical ways to deal with tie E.g. we ay need to agree that udate occurred before udate Or offer a lease on a resource that exires at tie 1:1.15 Or guarantee that a tie critical event will reach all interested arties within 1s ut what does tie ean? Tie on a global clock? E.g. with GPS receiver or on a achine s local clock ut was it set accurately? nd could it drift, e.g. run fast or slow? What about faults, like stuck bits? or could try to agree on tie Laort s aroach rawing tie-line ictures: Leslie Laort suggested that we should reduce tie to its basics Tie lets a syste ask Which cae first: event or event? In effect: tie is a eans of labeling events so that If haened before, TIME() < TIME() If TIME() < TIME(), haened before snd () 1

2 rawing tie-line ictures: rawing tie-line ictures: snd () snd () C,, C and are events. Could be anything eaningful to the alication So are snd() and rcv() and deliv() What ordering clais are eaningful? haens before, and C before Local ordering at a single rocess Write C and C rawing tie-line ictures: rawing tie-line ictures: snd () snd () C C snd () also haens before rcv () istributed ordering introduced by a essage Write M snd ( ) rcv ( ) haens before Transitivity: haens before snd ( ), which haens before rcv (), which haens before rawing tie-line ictures: snd () C and are concurrent Looks like haens first, but has no way to know. No inforation flowed Haens before relation We ll say that haens before, written, if 1. P according to the local ordering, or 2. is a snd and is a rcv and M, or 3. and are related under the transitive closure of rules (1) and (2) So far, this is just a atheatical notation, not a systes tool 2

3 Logical clocks sile tool that can cature arts of the haens before relation First version: uses just a single integer esigned for big (64-bit or ore) counters Each rocess aintains LT, a local counter essage will carry LT Rules for anaging logical clocks When an event haens at a rocess it increents LT. ny event that atters to Norally, also snd and rcv events (since we want receive to occur after the atching send) When sends, set LT = LT When receives, set LT = ax(lt, LT )+1 Tie-line with LT annotations snd () LT C LT LT() = 1, LT(snd ()) = 2, LT() = 2 LT(rcv ( ))=ax(1,2)+1=3, etc Logical clocks If haens before,, then LT()<LT() ut converse ight not be true: If LT()<LT() can t be sure that This is because rocesses that don t counicate still assign tiestas and hence events will see to have an order Can we do better? One otion is to use vector clocks Here we treat tiestas as a list One counter for each rocess Rules for anaging vector ties differ fro what did with logical clocks Vector clocks Clock is a vector: e.g. VT()=[1, ] We ll just assign index and index 1 Vector clocks reuire either agreeent on the nubering, or that the actual rocess id s be included with the vector Rules for anaging vector clock When event haens at, increent VT [index ] Norally, also increent for snd and rcv events When sending a essage, set VT()=VT When receiving, set VT =ax(vt, VT()) 3

4 Tie-line with VT annotations Rules for coarison of VTs VT 1 snd () We ll say that VT = VT if I, VT [i] = VT [i] nd we ll say that VT < VT if VT()=[2,] C VT Could also be [1,] if we decide not to increent the clock on a sndevent. ecision deends on how the tiestas will be used. 2 4 VT = VT but VT? VT That is, for soe i, VT [i] < VT [i] Exales? [2,4] = [2,4] [1,3] < [7,3] [1,3] is incoarable to [3,1] Tie-line with VT annotations snd () VT VT()=[2, ] C VT VT()=[1,]. VT()=[2,4]. So VT()<VT() VT()=[3,]. So VT() and VT() are incoarable Vector tie and haens before If, then VT()<VT() Write a chain of events fro to Ste by ste the vector clocks get larger If VT()<VT() then Two cases: if and both haen at sae rocess, trivial If haens at and at, can trace the ath back by which learned VT [] Otherwise and haened concurrently Consistent cuts If we had tie, we could revisit these using logical and vector clocks In fact there are algoriths that find a consistent cut by Ileenting soe for of clock sking everyone to record their state at tie now+δ (for soe large δ) nd this can be ade to work well Relication nother use of tie arises when we talk about relicating data in distributed systes The reason is that: We relicate data by ulticasting udates over a set of relicas They need to aly these udates in the sae order nd order is a teoral notion 4

5 and relication is owerful! Relicate data or a service for high availability Relicate data so that grou ebers can share loads and irove scalability Relicate locking or synchronization state Relicate ebershi inforation in a data center so that we can route reuests Relicate anageent inforation or araeters to tune erforance Let s look at tie vis-à-vis udates Maybe logical notions of tie can hel us understand when one udate coes before another udate Then we can think about building relicated udate algoriths that are otiized to run as fast as ossible while reserving the needed ordering r s t rill down: Life of a grou If tries to reliably ulticast to s, it won t get an ack and will wait indefinitely. Q does an udate. ut how It needs can to be sure that s has failed? If reach the three is ebers wrong, s will be issing an udate! Initial grou ebershi Now is {r, s s, goes t} u offline for a while. Maybe it crashed u 1 Here, Here two we dates see the occur udate concurrently: ordering r issue sees again u followed by in a u ure 1, but t for. sees u Which 1 first, then cae u first,. Process s s is still offline; udate it sees Now neither. or s is back one This fro online illustrates? and two issues: udate resuably order and grou should ebershi receive the udate is sending. Questions to ask about order Who should receive an udate? What udate ordering to use? How exensive is the ordering roerty? Questions to ask about order elivery order for concurrent udates Issue is ore subtle than it looks! We can fix a syste-wide order, but Soeties nobody notices out of order delivery Syste-wide ordering is exensive If we care about seed we ay need to look closely at cost of ordering Ordering exale Syste relicates variables x, y Process sends x = x/2 Process sends x = 83 Process r sends y = 17 Process s sends z = x/y To what degree is ordering needed? 5

6 Ordering exale x=x/2 x=83 These clearly conflict If we execute x=x/2 first, then x=83, x will have value 83. In oosite order, x is left eual to 41.5 Ordering exale x=x/2 y=17 These don t see to conflict fter the fact, nobody can tell what order they were erfored in Ordering exale z=x/y This conflicts with udates to x, udates to y and with other udates to z Coutativity We say that oerations coute if the final effect on soe syste is the sae even if the order of those oerations is swaed In general, a syste worried about ordering concurrent events need not worry if the events coute Single udater In any systes, there is only one rocess that can udate a given tye of data For exale, the variable ight be sensor values for a teerature sensor Only the rocess onitoring the sensor does udates, although erhas any rocesses want to read the data and we relicate it to exloit arallelis Here the only ordering that atters is the FIFO ordering of the udates eitted by that rocess r s t Single udater If is the only udate source, the need is a bit like the TCP fifo ordering 6

7 Mutual exclusion nother iortant case we ll study closely rises in systes that use locks to control access to shared data This is very coon, for exale in transactional systes (we ll discuss the next week) Very often without locks, a syste raidly becoes corruted Mutual exclusion Suose that before erforing conflicting oerations, rocesses ust lock the variables This eans that there will never be any true concurrency nd it silifies our ordering reuireent r s t Mutual exclusion ark blue when holding the lock How is this case siilar to FIFO with one sender? How does it differ? Mutual exclusion re these udates in FIFO order? No, the sender isn t always the sae ut yes in the sense that there is a uniue ath through the syste (corresonding to the lock) and the udates are ordered along that ath Here udates are ordered by Laort s haened before relation: Tyes of ordering we ve seen Tyes of ordering we ve seen cheaest Still chea More costly Most costly eliver udates in an order atching the FIFO order in which they were sent eliver udates in an order atching the order in which they were sent For conflicting concurrent udates, ick an order and use that order at all relicas eliver an udate to all ebers of a grou according to ebershi view deterined by ordering udates wrt view changes fbcast cbcast abcast gbcast eliver udates in an order atching the FIFO order in which they were sent eliver udates in an order atching the order in which they were sent For conflicting concurrent udates, ick an order and use that order at all relicas eliver an udate to all ebers of a grou according to ebershi view deterined by ordering udates wrt view changes 7

8 Recoended readings In the textbook, we re at the beginning of Part III (Chater 14) We ll build u the virtual synchrony relication odel in the next lecture and see how it can be built with 2PC, 3PC, consistent cuts and ordering 8