istributed lgorithms in Networks EES : Lecture 7 epartment of Electrical Engineering and omputer Sciences University of alifornia erkeley Network Protocols often have unintended effects TP Eample TP connections detect congestion after it has happened May cause packet drops from uncongested well behaved flows Non congested flows back off Eample Two TP flows sharing the same router get uneven bandwidths because one has a much smaller RTT than the other Routing Oscillation and countless other pathologies It is very difficult to avoid these unintended effects March, 00 KP: EES Lecture 7 The Internet is a HUGE istributed System Nodes are local processors Messages are echanged over various kinds of links Nodes contain sensors which sense local changes Nodes control the network jointly Method for doing this is a distributed algorithm Eample: Routing Time taken to solve the problem has two components: omputation time taken for local processing ommunication time for messages to be received over the links Today Focus on protocol design issues How to move from entralized to istributed lg. Synchronous and synchronous computation Why does the synchronous ellman Ford converge? Selfish behavior distributed systems March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 Solving Global Problems in a istributed Setting Eamples: Minimum Spanning Tree Shortest Path Leader Election Topology roadcast Much easier to think in terms of centralized algorithms reativity needed to convert to the distributed case The Network is Heterogeneous Speed ialup to terabit fiber Reliability Hosts: istributed Server farms to 8 P Links: Noisy wireless to virtually error free fiber ongestion Trustworthiness What is a general enough model to cover all of this? March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7
onsensus over an Unreliable Link and in a connection over an unreliable link They both want to terminate the connection only if they are certain that no more packets will arrive from the other user messages to all their neighbors who then flood. won t terminate unless it knows that knows it is about to terminate. won t terminate unless it knows that knows it is about to terminate slow link March, 00 KP: EES Lecture 7 7 March, 00 KP: EES Lecture 7 0 onsensus Problem Suppose tells it can terminate and receives this message, say M can terminate, but will never know if actually received M and so it can t terminate neighbors who then flood. fails sends K(M) to, but then needs to makes sure that received this message, so it must wait for K(K(M)) never terminates. In fact, NO protocol eists to solve this problem! Worth convincing yourself of this fact. March, 00 KP: EES Lecture 7 8 March, 00 KP: EES Lecture 7 Link model Error correction ssume that errors can eventually corrected Propagation elay Fied. fails marks the link down Variable but no more than d Variable with no upper bound Other components of delay Queueing elay Transmission elay Packet order FIFO an be delivered in arbitrary order March, 00 KP: EES Lecture 7 9 March, 00 KP: EES Lecture 7
. fails marks the link down. comes back up msg lost This can be fied with sequence numbers, but then other problems emerge. fails marks the link down. comes back up marks the link up. marks the link down. fails message lost thinks is down when it is actually up! March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7. fails marks the link down. comes back up Synchronous v/s synchronous lgorithms Synchronous algorithms can be described in terms of global iterations. The time taken for a given iteration is the time taken for the slowest processor to complete that iteration: time driven E.g. TM or SONET synchronous algorithms eecute at a processor based on received messages and internal state: event driven E.g. IP protocols which must run over heterogeneous systems March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 7 Slotted Time. fails marks the link down. comes back up marks the link up. marks the link down Slotted system,,, ll nodes agree on slot boundaries Have access to a global clock Helps to co-ordinate the nodes Every node can run the same algorithm Proving correctness is generally tractable if the centralized algorithm is analyzable Easier to understand the sequence of communication between nodes March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 8
Synchronous ellman-ford (SF) Every node runs the same algorithm Time is slotted and in every tick each node sends its distance vector. t time h, node i has as an estimate of the shortest path to node that has <= h+ hops h+ (I,j) = min kεn(i) { h (k,j) + c(i,k)} Synchronization Penalty Slow nodes can create a penalty as well Node Node Node Penalty can be huge! March, 00 KP: EES Lecture 7 9 March, 00 KP: EES Lecture 7 Synchronous Timing Great when links are reliable and similar Node Implementing a Synchronous lgorithm Suppose the slowest process can complete an iteration in time T p Node Link delay is always less than T l Then a slot size of T p +T l or more is sufficient ut most processors may be most of the time What if T p and or T l are not known? March, 00 KP: EES Lecture 7 0 March, 00 KP: EES Lecture 7 Synchronous Timing ut what when some links are much faster? Node Node Node Node suffers synchronization penalty Locally Synchronous omputation Forget about fied slots When a node has received all round k- messages from its neighbors, it computes and sends out its round k message Worst-case: s slow as synchronous computation Generally much faster ny synchronous algorithm that isn t using time as a part of the computation will also work when run in a locally synchronous manner. March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7
Local Synchronization Send update k after you ve heard update k- from all neighbors. Node Node Node Why bother with synchronous lgorithms To reduce the synchronization penalty ifficult to get the synchronous algorithm to start The network is dynamic Flows Topology Think of the algorithm having to restart with a new set of initial conditions, every time there is a failure hanges create events which may or may not have global impact Event-driven algorithms better suited March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 8 ompare with Synchronous Slot size is affected by the slow node Node Node Node synchronous ellman Ford (F) on t even wait to hear from all neighbors! Use most recent information to compute new distance vectors i.e. use last received values of () and d Whenever ready, each node i computes (i) = min k ε N(i) [(k) + c(i,k)] Sends the result to each of its neighbors No notion of global iterations In general, nodes are using different and possibly inconsistent estimates March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 9 synchronous computation No notion of slot size at all! 7 8 9 0 Node Node Node Why should this work? synchronous ellman Ford Regardless of how asynchronous the nodes are, the algorithm will eventually converge to the shortest path Links can go down and come up but as long as the topology is fied after some time t, the algorithm will eventually converge to the shortest path Why? There s some hope because the (j) can only go up if one of j s neighbors estimates has gone up. March, 00 KP: EES Lecture 7 7 March, 00 KP: EES Lecture 7 0
Idea Trustworthiness There are too many different runs of F, so lets try to bound the range of distance estimates of (j) over time o this by two different runs of Synchronous F Set different initial estimates One run U, uses the familiar ones, i.e. estimate is infinity if no edge The other, L, uses -if no edge! One bounds the estimates from above, one from below and both find the correct the shortest paths eventually For every iteration k of the two SF runs L k (j) L k+ (j) *(j) U k+ (j) U k (j) For any asynchronous run,, it is possible to show that for any k, there is a time t such that L k (j) L k+ (j) t (j) U k+ (j) U k (j) Since both lower and upper runs converge to the optimal, so will F eventually Three levels Honest: lways in conformance of the protocol Selfish: May lie to get better performance out of the protocol (GP) Malicious: Unpredictable Internet Protocols (for the most part) assume Honest protocol agents Unreliable infrastructure Infrastructure has gotten more reliable, and agents have gotten less honest raess s Parado: Eample of how Greediness and distributed algorithms can lead to suboptimality March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 Soft State ongestion Sensitive Routing State with Time-Out Eample: host joins a group by sending a join message to a host manager. The manager adds the host to the group for the net T seconds. If the host wants to stay in the group it must send a refresh message within T seconds to the manager. Otherwise it is dropped. dvantage: Manager robust to host failure isadvantage: Too many messages Most internet protocols use this way of communicating Trades of simplicity of correctness with compleity of communication epends on traffic S R Q T Weights are delays/bit unit of traffic from s to t u bits on the upper path -u bits on the lower path March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7 The nature of asynchronous distributed protocols Generally non-intuitive Limited theory to work with orrectness etremely hard to prove Robustness hard to analyze Networking gurus have a vast knowledge of special cases that can lead to strange behaviors Misconfiguration is a big cause of errors Soft state helps a lot, but wastes many messages! What about just broadcasting topology information accurately so that these problems go away Each Node is Greedy epends on traffic S R Q T Weights are delays/bit unit of traffic from s to t u bits on the upper path -u bits on the lower path Node S minimizes Total delay = u(u+) + (-u)(-u) = (u^ u +) elay minimized at u=. So Total elay =. s March, 00 KP: EES Lecture 7 March, 00 KP: EES Lecture 7
Greediness leads to suboptimality S still sends. on each path. R Weights are delays/bit unit of traffic from s to t S 0 T u bits on the upper path -u bits on the lower path. Q RESS S PROX R is greedy R diverts all. units on to the new link Now total delay is! March, 00 KP: EES Lecture 7 7 onclusions istributed lgorithms are not intuitive There is no systematic way to design them ctive research area is making some progress Until then use Hacking bilities Simulation ontrol Theory Optimization Graph Theory Game Theory. Greedy and malicious users complicate the protocol design problem even more nother active research area making progress This is why it is hard to build networks March, 00 KP: EES Lecture 7 8 7