Analysis of Shortest-Path Routing Algorithms in a Dynamic Network Environment

Transcription

1 Analysis of Shortest-Path Routing Algorithms in a Dynamic Network Environment Zheng Wang Jon Crowcroft Department of Computer Science, University College London Gower Street, London WC1E 6BT zwang@ukacuclcs, jon@ukacuclcs ABSTRACT In a dynamic network environment under heavy traffic load, shortest-path routing algorithms, particularly those that attempt to adapt to traffic changes, frequently exhibit oscillatory behaviors and cause performance degradation In this paper we first examine the problems from the perspective of control theory and decision making, and then analyze the behaviors of the shortest-path routing algorithms in details 1 INTRODUCTION Shortest-path routing algorithms have been widely used in today s computer networks In such algorithms, each node attempts to route packets to their destinations over paths of minimum distance and updates the distances periodically to adapt topological and traffic changes There are two main classes of algorithms: distance-vector algorithm and link-state algorithm In a distance-vector algorithm, each node maintains a routing table containing the distance of the shortest path to every destination in the network A node only informs its immediate neighbors of any distance changes to any particular destinations Examples of distance-vector algorithms include the old ARPANET routing algorithm [1], NETCHANGE algorithm of the MERIT network [2] and cisco s IGRP [3] In a link-state algorithm, each node keeps track of the entire network topology and computes the routing table based on the link distance information broadcast by every node in the network Link-state algorithms have been used in the new ARPANET routing protocol (SPF) [4], OSPF [5] and ISO s IS-IS [6] Shortest-path routing algorithms have served remarkably well in the network environment where traffic is light and network conditions change slowly Shortest-path algorithms are able to respond to topological changes automatically and adjust routing decisions when traffic changes In the presence of congestion, shortest-path routing algorithms can reduce the traffic away from the overloaded the paths However, as the networking speed increases and new applications proliferate, the network environment becomes much more dynamic and the traffic patterns less predictable The range of traffic rates which the network has to deal with is much wider and the distribution of traffic can also be extremely uneven In a network environment where traffic approaches the capacity of paths and changes dynamically, shortestpath routing algorithms, particularly those that attempt to adapt to traffic conditions, frequently exhibit oscillatory behaviors and cause performance degradation [7, 8] In this paper we first examine the problems from the perspective of control theory and decision making and then analyze the behavior of the shortest-path routing algorithms in a dynamic environment 2 OVERVIEW Routing algorithms can be classified as static, quasi-static and dynamic according to how adaptive they are In a static routing algorithm, the choice of routes is predetermined and fixed for relatively long time period (months or even years) Static routing algorithms are simple to implement but vulnerable to resource failures and traffic changes A dynamic routing algorithm, in contrast, allows continuous changes in routing decisions to reflect the current traffic and topological changes But such routing algorithms are often very complex and require large amounts of information exchange and computation The shortest-path routing algorithms widely used today fall approximately into the quasi-static category, in which the link distances remain constant for short period of time (eg 10 seconds in SPF) but can be updated when significant changes occur ACM SIGCOMM 63 Computer Communication Review

2 A quasi-static routing algorithm has two key procedures: distance estimation and route computation The distance estimation procedure predicts the link distances for the next route updating period based on the information collected in the past The estimated link distances are then propagated over the network and used to derive the routing table for packet forwarding From the control theory point of view, the distance estimation procedure belongs to the problem of adaptive parameter tracking [9] When traffic changes slowly, the distances between two consecutive route updating periods are assumed to be closely correlated, so that, based on observations of the distances in the present route updating period, the distances in the next period can be estimated The difference between the estimated value and the measured value is called estimation error When the estimation error increases, the estimated distances which the route computation is based on become less valid therefore the quality of the routes deteriorates The unpredictable nature of the traffic and the delayed feedback information make accurate estimation extremely hard In shortest-path routing algorithms, the link distances are decided based on the particular routing metric used (eg delay, queue length, throughput, hops) With traffic-sensitive routing metrics (eg delay), each node estimates the link distances for the next route updating period by averaging the link distances in this period For example, in the SPF routing algorithm, the delay of each link is averaged every 10 seconds and then used for calculating the routing table for the next route updating period The underlying assumptions of this approach are that the route updating does not affect the traffic distribution significantly and the statistics of the traffic remain unchanged As we will see in the next section, such assumptions are approximately true only when the traffic load is light The problem is further complicated by the interaction between the distance estimation procedure and route computation procedure The routing decisions for the next route updating period are made based on the current estimated link distances However, the distribution of the traffic in the next period is determined by the resultant routing decisions This recursive nature makes it difficult to obtain accurate distance estimation with simple algorithms The route computation can be considered as a problem of individual decision making under uncertainty [10] When one node chooses a path for a destination, it must consider not only the effects of its own decision but also the effects of routing decisions made by the other nodes With only partial information about the traffic distribution and limited resource for route computation, it is difficult for the node to determine the overall effects of its routing decisions In shortest-path routing algorithms, each node simply chooses the paths of minimum distances based on information collected in the previous route updating period This strategy is based on individual rationality rather than group rationality [11] Each individual attempts to maximize its use of the resource with no regard to effects of its action on the other individuals It usually works well when there is adequate resource However, when the resource is scare, the conflict of interests dominates the result and individual users compete with each other for more resource, which often leads overload and collapse It is obvious that a shortest-path routing algorithm has only one path for a pair of source and destination at any given time In other words, no matter how many possible paths exist from source to destination, only the best path is used We call it the single-path restraint The single-path restraint limits the maximum flow between a pair of source and destination when there are more than one paths available When the traffic approaches the capacity of the best path and none of the paths can accommodate the traffic, shortest-path routing algorithms become unstable and often oscillate between different paths The nature of individual rationality only makes the oscillation worse When distances of different paths vary widely, The paths which have comparatively shorter distances may attract too much traffic and become congested while the paths reported high distances may be abandoned and become idle 3 DETAILED ANALYSIS In the following sections, we examine the problems in six different perspectives and discuss in details the behavior of the shortest-path algorithms in a dynamic environment Estimation Error Link distance estimates provide the criteria on which the routing decisions are based The quality of the routes ultimately depends on the accuracy of the link distance estimation no matter what route computation algorithms are used In some respect, the estimation error can be used as a measure of routing performance The common approach of distance estimation is to average the link distances in current route updating period and use them as estimated distances for the next period Let us now examine the distance estimation process of link i Suppose that D i (t T, t ) is the measured average delay of link i during the time period (t T, t ), where T is the route updating interval The delay information ACM SIGCOMM 64 Computer Communication Review

3 D i (t T, t ) is then propagated and received by node j at time (t +t j ), where t j is the propagation delay for the information to rach node j Node j updates the distance of link i with D i (t T, t ) and uses it as the estimated delay for the next time period (t +t j, t +t j +T ) Let D i (t +t j, t +t j +T ) be the actual measured delay of link i during the time period (t +t j, t +t j +T ) The estimation error of link i during the time period (t +t j, t +t j +T ) is given by i = D i (t +t j, t +t j +T ) D i (t T, t ) We now consider a G/G/1 system under heavy traffic load The approximate average queueing delay D q is given by [12] (σ D q 2 a +σ 2 b ) 2(1 ρ)t where σ 2 a, σ 2 b and t are respectively variance of interarrival, variance of service time and average interarrival time Suppose ρ 1 and ρ 2 are respectively the utilization factors during time period (t T, t ) and (t +t j, t +t j +T ) The estimation error is given by (σ i = 2 a +σ 2 b ) (σ 2 a +σ 2 b ) 2(1 ρ 2 )t 2(1 ρ1 )t Note that t is given by 1 so we have µ ρ i = µ σ 2 a +σ 2 b ρ 2 ρ 1 2 (1 ρ 2 )(1 ρ 1 ) σ Let o = µ 2 a +σ 2 b and ρ = ρ2 ρ 2 1, we have i = o ρ (1 ρ 1 ρ)(1 ρ 1 ) The estimation error is a function of ρ 1 and ρ 1 If we fix ρ and increase ρ 1 gradually, the estimation error remains very low until ρ 1 approaches to 1 The estimation error then increases sharply (Fig1) This threshold behavior results from the fact that the delay changes much more rapidly under heavy traffic Normalized Estimation Error Utilization Factor Fig1: Normalized Estimation Error as a Function of ρ 1 ρ=30%ρ 1 ρ=20%ρ 1 ρ=10%ρ 1 Fig2 shows the normalized estimation error as a function of percentage of changes in the utilization factor The estimation error is low when the traffic load is decreasing but rises sharply when the traffic load is increasing The change in traffic load ρ depends on the nature of traffic and the propagation delay When the propagation delay is large, the difference between ρ 1 ρ 2 tends to increases In general, the estimation error is low only when the traffic load is light or decreasing ACM SIGCOMM 65 Computer Communication Review

4 Normalized Estimation Error ρ 1 =07 ρ 1 =06 ρ 1 = Change in Utilization Factor Fig2: Normalized Estimation Error as a Function of ρ/ρ 1 The route updating itself can also significantly change the traffic distribution and increase the estimation error When the measured delay is very high or very low, it is very likely that the route updating will alter the traffic distribution in the next period therefore the measured delay is less valid as the bases for estimating the link distance for the next route updating period The change of shortest-paths shifts traffic between different paths When the total traffic consists of many small flows, the estimation errors as a result of traffic shifting are small However, if the traffic is dominated by several large flows, traffic shifting can cause large estimation errors and may lead to oscillation Stability Stability is an important issue in routing algorithms Routing algorithms have to be able to reach an equilibrium in finite time, provided no continuous topological and traffic changes occur However, due to the single-path restraint, shortest-path routing algorithms tend to oscillate traffic flows between different paths even there are no traffic changes take place Suppose that node x has two identical paths P 1 and P 2 to destination z Initially they have equal distances If during one route updating period, P 1 is chosen to be the shortest-path, the distance of P 1 rises as the traffic increases At the next route updating point, path P 2 is preferred since it has shorter distance But this will be reversed during the next route updating The oscillation will continue and no equilibrium can be reached Such oscillation may be avoided by allowing route updating only when the reported distance is significantly different from the previous one For example, in SPF algorithm, a delay updating threshold is set to 64 ms The latest measured delay is compared with the previous one If the difference does not exceed the threshold, no updates are generated but the threshold is decreased by 125 ms which allows long lasting changes to be updated eventually Under light traffic, the distances change slowly A small threshold can effectively damps most oscillations But when the traffic approaches the capacity of the path, the distances increase sharply The routing algorithms desperately attempts to search for a path which can accommodate the traffic rate When such a path is not found, traffic simply oscillates between different paths The oscillation resulted from heavy traffic load has severe effects on the performance of routing If the distance of a link between two nodes is greater than a path between the two nodes, the link will be abandoned by all the traffic Therefore, while the traffic is heavy, many links may be left without any traffic while other links are severely congested When the traffic flow between two nodes oscillates among different paths, all traffic that shares any links on the paths are affected In some circumstance, the oscillation can be propagated like a wave across the network ACM SIGCOMM 66 Computer Communication Review

5 A Route Updating Period: 1 second 400(kbps) 100(kbps) Delay Updating Threshold: 40 ms Maximum Queue Length: 10 Average Packet Length: 512 Bytes TCP Connections: A-B and A-C B 200(kbps) C Fig3: Simulation Topology We now describe a simulation experiment which shows the severe degradation caused by wild oscillation Fig3 shows the topology of a three-node network and the corresponding link capacity The simulator uses TCP source with show-start and congestion avoidance [13] and the SPF routing algorithm which measures the link delay every 1 second and updates the routing table if the change of delay exceeds 40 ms Initially there is no traffic in the network and then node A initiates two TCP connections to node B and C in an arbitrary order Links Used AB BC AC CB Time (seconds) Fig4: Shortest-Paths Used by Two Connections The links which forms the shortest-paths for 25 route updating periods are recorded in Fig4 The solid lines and dashed lines indicate respectively the shortest-paths for connection A-B and A-C Fig4 shows that no equilibrium is achieved and the traffic of both connections oscillates between two paths In fact, at any given time, one of the two links AB and AC is used by both connections and the other one is left idle This is caused by the fact that routing decisions are made based on individual rationality Each user always attempts to choose the best path As a result, the two users always select the same path Fig5 shows the sending sequence number of the two connections with the SPF routing algorithm The oscillation has severely affected the throughput The total average throughput of the two connections during the 25 route updating periods is about 240 kbps which only accounts for less than half of the total bandwidth of the two links (500 kbps) When the SPF routing algorithm can not converge to an equilibrium, its performance is in fact worse than that of a simple fixed routing scheme Fig6 shows the result of the experiment with a fixed routing scheme in which ACM SIGCOMM 67 Computer Communication Review

6 Sending Sequence Number (KB) Time (seconds) Fig5: Sending Sequence Number with SPF Routing connection A-B and A-C use fixed paths link AB and link AC respectively The total average throughput is about 450 kbps Sending Sequence Number (KB) Time (seconds) Fig6: Sending Sequence Number with Fixed Routing In our experiment, we use a sufficiently large receiving window to allow traffic load approaching the link capacity However, due to the effect of the slow-start and congestion avoidance in TCP source, no severe congestion occurs in the simulation experiments The loss of bandwidth is not because of the retransmission but a result of poor routing decisions Without congestion control in the TCP source, the performance of the SPF routing algorithm can be much worse as congestion caused by the poor routing decisions only further decreases the throughput Responsiveness Shortest-path routing algorithms are designed to adapt changing conditions It is desirable that they respond to any traffic changes dynamically However, in a large network, there is a limit as to how fast the routing algorithm ACM SIGCOMM 68 Computer Communication Review

7 can adapt to changing conditions [14] Route updating is a costly operation in terms of CPU and bandwidth resources Particularly to the link-state algorithms, information of changes has to be propagated to all the nodes in the network and routing table has to be re-computed Frequent route updating may consume substantial amount of link bandwidth and CPU resource Route updating is often too slow as compared with the traffic changes, as route updating involves propagation of information across the network The communication delay poses a limit on how fast the routing algorithm can react to the traffic changes In a large network, it may take long time for routing information to travel across the network The routing algorithm can not respond to the network conditions at a rate faster than the rate at which relevant information can reach the points concerned and corresponding action can be taken During the route updating, the network is in a state of transition therefore the routing tables among the nodes may be inconsistent and temporary routing loops may be formed Moreover, the routing updates usually have higher priority than users traffic Transmission of large amount of such packets can affect the the users traffic Hence, routing updating in a large network is often too costly and too slow for adapting dynamic traffic changes such as congestion Frequent route updating, under heavy traffic, can lead to wild oscillation and degradation of performance In practical, routing updates are generated only when the traffic changes are long lasting Maximum Flow Maximum flow is one of the important characteristics of a network It represents the maximum traffic rate that the network is able to cope with for one connection When the traffic is bursty, the momentary traffic rate can be many times higher than the average rate Some time-critical applications may not allow the their traffic to be flow controlled at source It is highly desirable that the network be able to handle traffic bursts of high-speed It is known from the Max-Flow Min-Cut Theorem, the maximum flow between any two arbitrary nodes in a network is equal to the capacity of the minimum cut separating those two nodes However, the shortest-path routing algorithms can usually achieve far less than the theoretic potential due to the single path restraint Since the nodes can not route traffic simultaneously to more than one paths, the maximum throughput for one connection can not exceed the capacity of the best path unless the routing can change so frequently that more than one paths are kept busy at one time Consider that node x has two paths P 1 and P 2 to destination z If at the time when the shortest-path changes from P 1 to P 2, the output queue for P 1 is full and the shortest-path changes back to P 1 before the queue for P 1 drops to zero, node x may keep both paths busy all the time and therefore increase the throughput Suppose that Q m, P s and C are respectively the maximum output queue length, the average packet size and capacity of the paths To keep n paths busy, the routing algorithm has to select each path at least once before Q m packets drain out in Q m P s time Therefore, the maximum route updating period is Q m P s C C (n 1) The maximum route updating period is often too small even in low-speed networks Take the ARPANET for example (Q m = 8, P s = 512 Byte, C = 50 kbps), to keep two output lines busy, the node has to at least update routing tables every 655 ms In high-speed networks, the figure is much smaller and in impractical In fact, the stability problems basically eliminate the possibility of using route updating as means of load sharing In practice, the route updating period is usually set quite long (eg 10 seconds in ARPANET) to ensure the stability of the network The maximum flow is usually around the capacity of the best path Congestion Control Routing algorithms with traffic-sensitive metrics can, in theory, adapt to congestion When a path is overloaded, the reported link cost increases The routing algorithms re-computes the routing table and reduces the traffic over the congested path In practice, however, the ability of congestion control is rather limited The amount of traffic which can be shed from the congested link is difficult to predict in advance and largely depends on the composition of the traffic flow When the traffic consists of many small flows over different source-destination pairs, appropriate amount of traffic may be shed by carefully turning the link metrics But if traffic in the network is dominated by large flows, a stable solution is often difficult to find The routing algorithm tends to shift traffic around without reaching an equilibrium Congestion occurs when the traffic and resource mismatch at some points of the network It is therefore usually local and changes rapidly Changing routing table, which involves information exchange and route computation across the entire network, is often too costly and also too slow When congestion does occur, the routing algorithm has to wait until next updating period to respond At that time, the congestion may have already dissipated And the reported high distances caused by the congestion, which are no longer valid, nevertheless has a misleading effect on the traffic that shares the same link when the next route updating is due ACM SIGCOMM 69 Computer Communication Review

8 Failure Transparency Shortest-path routing algorithms provide a certain degree of failure transparency to the network users When resource failures are detected, the routing algorithms are able to search for alternative paths and update the routing table For some real-time applications, it is important that the alternative paths are found quickly so that the transition can be smooth and transparent However, it is usually difficult to detect the failures within short time after they occur Nodes usually rely on timeout mechanisms or reports from network management systems to monitor the availability of the resources It can take several seconds or even minutes before a failure is detected The traffic may still be routed along the failed path until the next routing updating During this period, some applications can be severely affected and some connection may not survive at all The packets, which are sent before the alternative paths can be found, may build up inside the network and cause further congestion in other areas 4 CONCLUSIONS AND FUTURE WORK The analysis of the behavior of shortest-path routing algorithms has shown that in a dynamic environment shortest-path routing algorithms often exhibit oscillatory behaviors and lead to performance degradation When the traffic is heavy and changing rapidly, the estimation errors of the link distances increase and the routing algorithm often can not converge to a stable solution and result in wild oscillation We now discuss some possible solutions which may provide some directions for future work One of the problems we have discussed is that the measure used for distance estimation, ie averaging the current distances, is not accurate under heavy traffic When the estimation errors is large, it is simply impossible for routing algorithms to produce good routes Link metrics such as delay used in SPF routing algorithm have wide range of permissible values which often causes oscillation under heavy traffic load To dump oscillation, the link distances have to be transformed in appropriate ways before they are propagated across the network A simple transformation function described in [8] shows substantial improvement in performance However, as the traffic is time-dependent and network conditions vary widely, the link metrics should be able to establish the transformation model based on the collected data and modify the parameters accordingly [9] The routing algorithms must be able to predict the effects of route updating on the traffic shifting With limited information, the conflict between different traffic is hard to be avoided Routing algorithms have to provide more information to ensure some degree of global coordination is maintained, particularly when the network is loaded with heavy traffic Routing updates from individual nodes should be subject to modification to meet some global requirements (eg stability) As the traffic approaches the capacity of a path, it is often impossible for a single-path routing algorithm to derive satisfactory routes It is necessary to provide some degree of load sharing by using multi-path routing Although most multi-path routing algorithm are too complex to be used in an adaptive fashion, some simple form of multi-path routing can be easily adopted and can still improve the performance For example, both OSPF [5] and IS-IS [6] provide load splitting on paths of equal value To make full use of the multi-equal-path routing with traffic-sensitive metrics, it may be necessary to transform the values of link distances into a set of discrete values When the traffic is heavy and changing rapidly, it is difficult for the routing algorithm to determine the entire routes based on the past link distance information It has to allow continuous changes in routing decisions However, route updating is costly and often too slow and may lead to oscillation A possible solution is to find alternate routes without route updating When the traffic accumulated in some areas of the network exceeds a limit, an alternate route can be provided to ease the congestion An alternate routing algorithm based on this approach is described [15] REFERENCES [1] J M McQuillan and D C Walden, "The ARPA Network Design Decision", Computer Networks, Vol1, pp , 1977 [2] W D Tajibnapis, "A Correctness Proof of a Topology Information Maintenance Protocol for a Distributed Computer Network", Communication of the ACM, Vol20, pp , 1977 [3] C L Hedrick, "An Introduction to IGRP", Preprint, RUTGERS, Centre for Computers and Information Services, The State University of New Jersey, Oct 1989 [4] J M McQuillan, et al, "The New Routing Algorithm for the ARPANET", IEEE Transactions on Communications, Vol COM-28, May 1980 ACM SIGCOMM 70 Computer Communication Review

9 [5] J Moy, "The OSPF Specification", RFC 1131, SRI International, Menlo Park, Calif, Oct 1989 [6] International Standards Organization, "Intra-Domain IS-IS Routing Protocol", ISO/IEC JTC1/SC6 WG2 N323, Sept 1989 [7] D Bertsekas, "Dynamic Behavior of Shortest Path Routing Algorithms for Communication Networks", IEEE Transactions on Automatic Control, Vol AC-27, No1, Feb 1982 [8] A Khanna, J Zinky, "The Revised ARPANET Routing Metric", in Proc of ACM SIGComm 89, pp 45-56, Sept 1989 [9] H Sorenson, "Parameter Estimation", Marcel Dekker, Inc, 1980 [10] R Luce, H Raiffa, "Games and Decisions", John Wiley & Sons, Inc, 1957 [11] R Cyert, M DeGroot, "Bayesian Analysis and Uncertainty in Economic Theory", Chapman and Hall, 1987 [12] L Kleinrock, "Queueing Systems", Volume II, John Wiley & Sons, 1976 [13] V Jacobson, "Congestion Avoidance and Control", in Proc of ACM SIGComm 88, pp , Sept 1988 [14] J Seeger, A Khanna, "Reducing Routing Overhead in a Growing DDN", in Proc of MILCOM 86, pp , Oct 1986 [15] Z Wang, J Crowcroft, "Shortest Path First With Emergency Exits", in Proc of ACM SIGComm 90, Sept 1990 ACM SIGCOMM 71 Computer Communication Review