A QoS Control Method Cooperating with a Dynamic Load Balancing Mechanism

Transcription

1 A QoS Control Method Cooperating with a Dynamic Load Balancing Mechanism Akiko Okamura, Koji Nakamichi, Hitoshi Yamada and Akira Chugo Fujitsu Laboratories Ltd , Kamikodanaka, Nakahara, Kawasaki, , Japan Telephone: Fax: address: {akikoo, nakamichi, hitoshi, chugo}@flab.fujitsu.co.jp Abstract A dynamic traffic engineering architecture is described that provides QoS-guaranteed service (GS) in addition to existing best effort (BE) service. It is designed to provide bandwidthguaranteed paths for GS traffic along optimum routes and to provide one or more paths for BE service traffic along routes that will best avoid congestion. Simulation showed that the proposed routing algorithm achieves both load balancing and efficient resource usage and that dynamic load balancing improves the throughput of BE traffic. Use of this architecture will help to provide QoS-guaranteed service, effectively utilize network resources, and avoid degradation in BE traffic throughput. Keywords:IP Network Management, Traffic Engineering, MPLS, Load Balancing, QoS, Network Optimization 1

2 Introduction Current IP network problems End user's viewpoint Degraded of performance (e.g., lowered throughput and increased delay) due to congestion Network operator's viewpoint Profits do not improve though traffic increases every year Frequent bandwidth increases needed to support traffic increases Services supporting usage-based billing are limited To overcome these problems Provide QoS guaranteed service Develop fee/charge system based on network QoS Use network resources efficiently --> IP traffic control mechanism is required Fujitsu Laboratories Ltd Introduction As the number of users accessing the Internet through broadband connections continues to accelerate and the needs of the users become more diversified, quality of service (QoS) guarantees are becoming more important for both Internet service providers (ISPs) and end users. Since only best effort (BE) service is now provided, network performance is sometimes degraded by congestion. Moreover, even though the amount of Internet traffic is increasing, the providers are still not making much profit. This is because frequent increases in bandwidth are needed, and optional services that support usage-based billing are limited. To overcome these problems so that we can build a new network based on Internet technologies as an infrastructure for the coming broadband, ubiquitous era, we need to develop a QoS control system and a fee/charge system based on network QoS [1]. Doing this will enable ISPs to provide usage-based billing services and enable end users to enjoy reliable networks. Furthermore, network resources should be used efficiently to avoid installing too many network facilities. All of this requires that networks have an IP traffic control mechanism. In short, our challenge is to provide QoS guarantees without over-investing in network facilities. Additionally, new network services should come with a range of QoS-guaranteed service, such as bandwidth guarantees and delay guarantees. 2

3 Level of network efficiency Our Approach to IP Traffic Control High Low +Dynamic flow splitting +Multi route & path control +Explicit routing Minimum hop routing QoS: quality of service SLA: service level agreement SPF: shortest path first Low Traffic Engineering (TE) Static load balancing Hop-by-hop forwarding Best Effort Dynamic load balancing Explicit path setup Connectionless approach (Diffserv) + Packetpriority mechanism Proposed method + Accountrefund Connectionoriented approach (IntServ) + Admission policing shaping Level of QoS/SLA + Performance monitoring reporting Future High Fujitsu Laboratories Ltd Our Approach to IP Traffic Control This diagram illustrates our approach. The final goal, shown on the horizontal axis, is to offer various service level agreements (SLAs). Simply introducing a QoS mechanism is not enough. End users may want to know the actually provided QoS so they can compare it with the target in the SLA. ISPs may have to refund service fees if the target is not achieved. The vertical axis shows how much freedom and flexibility there is in selecting the route. In legacy IP networks, a single route between the ingress and egress routers is selected based on a single criterion, such as shortest path first. This can result in traffic congestion and does not support flexible routing. A network operator achieves its netowork operation policy by using constraint-based routing. Reliable network operation can be achieved by using two or more routes between the ingress and egress routers. However, the static use of two or more routes works poorly in dynamic network situations such as congestion due to burst traffic or faults. Dynamic control can improve performance, for example, by moving traffic from a congested route to one or more non-congested routes. A network should be used as a two-dimensional space, not as a linear line, to improve its tolerance of such dynamic situations. As shown in the figure, several technologies, such as differentiated services (diffserv) [2] and integrated services (intserv) [3], provide QoS and traffic engineering techniques that enable the effective use of networks. However, the use of these techniques does not satisfy the goals of offering QoS and effectively using networks simultaneously. We thus developed an IP traffic control method based on traffic engineering that can provide guaranteed QoS while using network resources efficiently. 3

4 Traffic Engineering (TE) Description Improves traffic performance Facilitates reliable network operations A main application of multiprotocol label switching (MPLS) Constraint-based routing (explicit routing) Example Applications Static/dynamic load balancing Achieves highly reliabile network by avoiding congestion/failure Enables efficient use of bandwidth resources --> Functions of dynamic load balancing have been proposed Fast Reroute Achieves highly reliabile network through high-speed failure recovery Fujitsu Laboratories Ltd Traffic Engineering (TE) Traffic engineering is used to improve traffic performance and to effectively use network resources by avoiding network congestion [4]. It is one of the main applications of multi-protocol label switching (MPLS) [5]. In particular, it enables network congestion to be avoided by using constraint-based routing. A label switched path can be flexibly established not only along the shortest route but also along the route with the largest residual bandwidth or a route that satisfies the network operator s policy(e.g., prohibit to transit a particular node). 3.1 Example Applications Traffic engineering is commonly used for static and dynamic load balancing and fast rerouting. Static and dynamic load balancing use multiple routes to a destination to avoid congestion/failure. For static load balancing, traffic is split statically based on a predefined ratio. For dynamic load balancing, it is split based on the actual traffic load. Fast rerouting establishes detour paths in case of a failure. If the network detects a link or node failure, it quickly moves traffic from the original route to a new route on the detour path. Using fast rerouting, a network can achieve high reliability through high-speed failure recovery. 3.2 Dynamic Load Balancing Dynamic load balancing has been proposed as an application of dynamic TE [6]. It optimizes network resource usage and maximizes network throughput. The main functions of network load balancing are - traffic statistics monitoring, - detour route establishment, - explicit path setting, and - load balancing. The load balancing algorithms are described elsewhere [7]. 4

5 Basic Architecture of Proposed Method - Cooperation between dynamic load balancing and providing QoS guarantee - Admission control TE Controller Statistics monitoring (Network, Application servers) QoS request Load balancing control for best effort traffic User Application Server QoS path control for guaranteed class traffic QoS path Bandwidth reserved High priority at scheduling Optimum route considering both network and application server resources MPLS Network Best Effort path Original path = minimum hop route Detour route used when there is congestion MPLS: multi-protocol label switching Fujitsu Laboratories Ltd Proposed method 4.1 Basic Architecture Our proposed QoS control architecture works in cooperation with a dynamic load balancing mechanism. Here, we consider bandwidth to be the QoS metric. The network provides services via bandwidth-guaranteed paths (QoS paths) using MPLS. The architecture achieve efficient use of network resources, which helps avoid over investment in infrastructure, by providing QoS paths along the optimum routes for the guaranteed service (GS) traffic and by providing one or more paths along routes that will best avoid congestion for the best effort (BE) service traffic. This diagram shows a centralized architecture using a TE controller, one approach to implementing our proposed method. We assume that BE and GS traffic share the network resource (i.e., link bandwidth). The TE controller maintains the network topology, monitors the network and server statistics, accepts user requests, and sets label-switched paths to routers for GS and BE traffic. Each output interface of each router in the network has a priority queue as a scheduling mechanism; it sends GS traffic packets at a higher priority. The path for GS traffic is calculated as an optimum route in order to use both the network and application server resources efficiently. The calculation is done using a topology graph constructed from links satisfying the requested bandwidth, and that bandwidth is reserved on each link along the path. When a single path accommodates aggregated flows, each user request must pass through admission control. If the bandwidth of the path is insufficient to accommodate more flows, an additional path is established. Since bandwidth is not reserved on each link along a path for BE traffic, the shortest route is selected for BE traffic using the entire network topology graph. However, if the route becomes congested, an alternative path is set up and the load is dynamically balanced. As an example application of our proposed method, we will describe a contents delivery network providing a streaming service with a QoS guarantee. Multiple streaming servers are connected to edge routers, and the streaming contents are distributed among these servers. End users can request the contents with the QoS guaranteed or with best effort; with the QoS guaranteed, users pay an extra charge for a guaranteed bandwidth. The TE controller accepts the request, selects the optimum streaming server and route, sets up the QoS path, and directs the user s request to the selected server. 5

6 QoS Routing Algorithm Minimize total cost of link and server total cost = server cost + link cost server cost: 1/(residual available output rate) link cost: 1/(residual available bandwidth) B server cost is high User A link cost D E C server cost TOTAL COST is MINIMUM! Link D-E cost is high Fujitsu Laboratories Ltd QoS Routing Algorithm The basic idea of our algorithm for selecting the optimum server and route for guaranteed QoS is to assign a cost to each link and to each application server based on its load (the higher the load, the higher the cost) and to search for the route and the server that minimize the sum of the costs of the links along the route and the cost of the server. Link cost is based on the non-reserved bandwidth of the link, so we can define link cost = 1 / (R_MAX R ), (1) where R_MAX denotes the link s maximum reservable bandwidth and R denotes the bandwidth actually reserved. Server cost represents the load on the server based on its bottleneck (CPU, IO, etc.). Several values can be used to represent the cost: CPU load, memory consumption, disk access load, number of user connections, traffic output rate, etc. A streaming server, for example, generally has an IO or NIC bottleneck rather then a CPU load bottleneck, so we can define server cost = 1/(S_MAX S), (2) where S_MAX denotes the maximum output rate (NIC rate) and S denotes the current output rate. After determining the cost of each server and each link, we search for the route and the server that minimize the total cost of the links along the route and of the server. We calculate the minimum cost route to each candidate server, from the server's edge node to the user's edge node, using the Dijkstra algorithm. We then select the set of server and route with the minimum total cost. We thus balance both the server load and the network load at a same time, using resources efficiently. 6

7 Evaluation of QoS Routing:Metrics and Model Metrics Number of QoS requests accepted Average number of hops in QoS paths Compared with LSL method (lowest server load) DNS method (domain name server) Simulation model 1 st step Select server with lowest load Select nearest server ISP network, 19 nodes Application servers Four Capacity of 500 Mbps QoS requests 1-10 Mbps bandwidth guarantee (random) User s edge selected at random (user s edge server s edge) 2 nd step Select minimum cost route to server Server location candidate OC3 (155 Mbps) T3 (45 Mbps) Fujitsu Laboratories Ltd Evaluation 5.1 QoS routing We evaluated the effectiveness of the proposed server and route selection algorithm for QoSguaranteed service by simulation. The metrics were the number of requests accepted and the average number of hops in the QoS paths. For reference, we also simulated the performance of two methods that can be achieved by combining existing techniques. - LSL (lowest server load) method: 1) select server with lowest load; 2) select minimum cost route to server. - DNS (domain name server) method: 1) select nearest server (based on location of user s edge); 2) select minimum cost route to server. The criterion for route calculation with both methods was the same as with our proposed method available bandwidth Model We used an ISP network model with 19 nodes [8], as shown in this slide. The network consisted of 155- and 45-Mbps links. There were four servers; the edge routers connected to them are selected at random from the routers shown as slash-pattern circles. End users connected with the other edge routers and requested QoS paths with a specified bandwidth requirement. For each request, a user's edge was selected at random and the requested bandwidth was selected randomly (between 1 and 10 Mbps). The capacity of the server (i.e., the maximum transmission rate) was 500 Mbps. For simplicity, QoS paths were not released. 7

8 Evaluation of QoS Routing: Results Number of requests accepted Effect of QoS routing considering both server and network loads Accommodates many more requests Provides QoS path with the smallest number of hops Number of QoS requests Number of QoS requests Average number of hops LSL method DNS method Our Proposal Fujitsu Laboratories Ltd Results These graphs show the results. The one on the left shows the number of accepted requests as a function of the number of requests. More requests were accepted with our method. With the DNS method, a user's request is rejected when there is no available route to the nearest server, even if a route to another server is available. With the LSL method, the lowest load server is selected first even if the route to the server is long. A lot of link bandwidth can thus be consumed unnecessarily, limiting the number of requests accepted. With our method, a medium load server to which there is a short route can be selected, even if there is a lower load server with a longer route to the server. Moreover, when there is a short route to a server with a high load, that combination of server and route is unlikely to be selected with our method. In short, our method flexibly selects a server and route based on both the server and network conditions, resulting in a lower blocking probability. The graph on the right shows the average number of hops as a function of the number of requests. The number of hops was the lowest with our method, meaning that network resources were used more efficiently. With the LSL method, long routes tend to be selected, as we mentioned above. It is interesting that even the DNS method, which selects the nearest server, suffered an increase in the number of hops as the number of requests increased. This is because a longer detour route is likely to be selected if the link utilization of the shortest route to the nearest server becomes high. With the DNS method, the longer route tends to be selected even if there is a shorter route to another server, so the number of hops is higher than with our method. The average link utilization after 800 requests were received was 43.6% with our method, 50.3% with the LSL method, and 49.2% with the DNS method. Our method can thus achieve a lower load even if it accepts more requests. In this way, our method optimizes the network to achieve efficient resource utilization. 8

9 Evaluation of Dynamic Load Balancing: Metric, Model, and Conditions Evaluate effect of dynamic load balancing under GS traffic conditions Metric Throughput of BE traffic BE: 50 Mbps Model 5-node-ring model with 100 Mbps links GS Conditions 50 Mbps BE traffic Bandwidth reserved for GS traffic is increases to 80 Mbps. Reserved Actual amount of GS traffic fluctuates Congestion detection conditions A) Actual (GS+BE) traffic > 80 Mbps B) (Reserved GS + Actual BE) traffic > 80 Mbps 100 Mbps Actual GS traffic Fujitsu Laboratories Ltd Dynamic load balancing We evaluated how well dynamic load balancing (D-LB) avoids performance degradation of BE traffic due to GS traffic with a time-varying rate. The performance of BE traffic is easily degraded due to actual GS traffic because GS packets have higher priority, as mentioned above. Dynamic load balancing can prevent this degradation. The metric for this evaluation was the throughput of BE traffic Model To show the effect simply, traffic flowed between two points connected by two paths. The link capacity was 100 Mbps, the maximum reservable bandwidth for QoS traffic was 80 Mbps, and the input BE traffic rate was fixed at 50 Mbps. GS flows were added at intervals of 5 seconds with the bandwidth reserved for GS increased in 6-Mbps steps. The actual amount of GS traffic varied based on the sine function Congestion detection conditions When congestion was detected under one of two conditions, load balancing was triggered. Under condition (A), the total amount of actual GS and BE traffic in a link exceed the threshold parameter, the condition detection threshold (CDT), which was set to 80 Mbps. Under condition (B), the sum of the bandwidth reserved for GS traffic and the actual amount of BE traffic exceed CDT. Dynamic load balancing was initiated when the specified condition continued for more than 30 seconds. If condition (A) was specified, dynamic load balancing was done based on the actual state of the GS traffic. This means that if there was enough available bandwidth along the shortest route (because the total amount of actual GS traffic was less than the bandwidth reserved for GS), BE traffic could be sent using the shortest route. If condition (B) was specified, BE flows were moved to the detour path by dynamic load balancing when the bandwidth reserved for GS traffic was increased. In this case, degradation in BE traffic performance was avoided even when the actual GS traffic rate suddenly reached 100% of the reserved bandwidth. 9

10 Evaluation of Dynamic Load Balancing: Results Throughput [Mbps] BE traffic with D-LB D-LB1 (condition A, solid line) D-LB2 (condition B, signs) Bandwidth reserved for GS traffic Actual GS traffic Time [s] Throughput Throughput remains remains almost almost maximum. maximum. BE traffic without D-LB Input BE Traffic = 50 Mbps (fixed) D-LB:Dynamic load balancing Throughput Throughput decreases decreases because because load load cannot cannot be be moved moved to to other other available available links. links. Fujitsu Laboratories Ltd Results This graph shows throughput as a function of time. The throughput of BE traffic without D-LB (dynamic load balancing), with D-LB under condition (A) (D-LB1), and with D-LB under condition (B) (D-LB2) are shown in addition to the bandwidth reserved for GS traffic and the actual throughput of GS traffic. Without D-LB, BE traffic throughput degraded when the GS traffic rate exceeded 50 Mbps. In contrast, with D-LB, BE traffic throughput remained almost always at 50 Mbps. With D-LB1, the throughput decreased for a while due to a lag in detecting the congestion caused by the increase in the actual GS traffic. With D-LB2, congestion was detected immediately, so there was no degradation in throughput of BE traffic. Dynamic load balancing prevents degradation in the performance of BE traffic because it uses other available links efficiently. If the actual rate of GS traffic fluctuates intensely, it is safe to use condition (B) to avoid degradation in BE traffic performance. 10

11 Implementation - Snapshot of Operation Screen - Control status display panel (Path setting, load balancing, etc.) Path-setting status display Detailed path information Click Fujitsu Laboratories Ltd Implementation We implemented a prototype traffic engineering(te) controller capable of dynamic load balancing and QoS path setting using software under SunOS. It has the following functions. - Admission control: accept end user service requests with bandwidth guarantee if they can be satisfied using existing resources - QoS control: calculates optimum route for GS flow - Dynamic load balancing control: calculates detour route for BE traffic when congestion is detected; if there are multiple available paths for BE traffic, periodically calculates, based on network situation, ratio of flows to be moved to each path - Statistics monitoring: periodically monitors statistics for each router and each application server - Router setting: controls each router via a command line interface (CLI) to set up label switched paths, accommodate GS flows into each path, and split BE flows into multiple paths As an actual application for QoS (bandwidth) guaranteed service, we selected video-on-demand service. Users request streaming content with a bandwidth guarantee from a contents menu screen on a Web server. The TE controller calculates the optimum route on acceptance of the user s request, considering the hop count, available bandwidth, and streaming server load. After determining the route, the controller sends commands to the ingress router via the CLI to set up the QoS path for the requested flow and then responds to the user via the Web server screen. Concurrently, the controller collects network information from the routers by using SNMP and maintains the information as a database of the topology, paths, and statistics. Based on this information, the controller detects congestion, executes route search, and balances the BE traffic load by sending commands to the edge routers. We built a test-bed system using this prototype TE controller. The behavior of the system was monitored using an operation GUI we developed. The GUI shows the control status of the controller such as congestion is detected or QoS path searching. It also shows the network topology, the statistics collected from the routers, and the path information for GS and BE traffic. 11

12 Conclusion Conclusion & Future Work Proposed method effectively utilizes resource while providing QoS QoS routing based on network and server loads Number of QoS requests accepted is improved Server and network load balancing are achieved Use of dynamic load balancing effectively provides QoS-guaranteed service Degradation in BE traffic throughput when GS traffic is fluctuating is avoided Future work Evaluation of performance in large-scale network Development of more advanced QoS control method based on TE Fujitsu Laboratories Ltd Conclusion In this paper, we proposed an architecture and a method for dynamic traffic engineering that both sets a QoS path along an optimum route for guaranteed service traffic and performs dynamic load balancing for best effort traffic. The result is efficient use of network resources and provision of QoS-guaranteed services. Simulation showed that our QoS routing algorithm achieves both load balancing and efficient resource usage and that dynamic load balancing improves the throughput of BE traffic. Introducing a dynamic load balancing function that works with a QoS control mechanism when QoS-guaranteed services are provided offers several advantages to both providers and end users. We plan to study and evaluate the scalability of our architecture and to develop an advanced QoS control mechanism based on TE techniques. References [1] Paul Ferguson and Geoff Huston, Quality of Service: Delivering QoS on the Internet and in Corporate Networks, John Wiley and Sons, [2] S. Blake, et al., An Architecture for Differentiated Services, RFC2475, IETF, December [3] R. Braden, D. Clark and S. Shenker, Integrated Services in the Internet Architecture: an Overview, RFC 1633, IETF, June [4] D. Awduche, et al., "Overview and Principles of Internet Traffic Engineering," RFC3272, IETF, May [5] D. Awduche, et al., Requirements for Traffic Engineering Over MPLS, RFC2702, IETF, September [6] H. Yamada, K. Nakamichi, M. Fukazawa and M. Katoh, "Dynamic Traffic Engineering for Network Optimization - Architecture and Evaluation," Proceedings of APNOMS2002, pp , Jeju Island, Korea, September [7] K. Nakamichi, H. Yamada and A. Chugo, "Dynamic Traffic Engineering under different traffic patterns," 16th International Workshop on Communications Quality & Reliability (CQR2002), Okinawa, Japan, May [8] Q. Ma and P. Steenkiste, "On path selection for traffic with bandwidth guarantees," Proceedings of International Conference on Network Protocols (IEEE ICNP97), pp , Atlanta, USA, October