INGRESS RATE CONTROL IN RESILIENT PACKET RINGS

Transcription

1 ABSTRACT THOMBARE, ASAVARI ANIL. Ingress Rate Control in Resilient Packet Rings. (Under the direction of Dr. Ioannis Viniotis.) Resilient Packet Ring (RPR) Protocol is becoming popular in Metro Networks due to its rich features. One of these features is the Fairness and Flow Control mechanism. RPR is standardized as IEEE and claims that there will be no frame loss inside the network under normal operation. This is achieved through feedback flow control messages. Traffic rate is controlled at the ingress itself based on these messages. We concentrate on the Ingress Rate Control scheme where the local station adds traffic to the ringlet based on the allocated rate limits and FAIR rate limits to achieve fairness in the ring network. The research presented in the thesis tries to achieve three different goals. We implement the Rate Control scheme in software to identify early implementation level issues and then implement it in hardware. We achieve a speed of 10 Gbps with our hardware implementation. We also achieve a full throughput at minimum frame size of 24 bytes. The hardware implementation is tested for compliance with the standard IEEE In the simulation study part of the research, we observe the response of individual shapers under compliance testing. We concentrate on the feature of bandwidth reclamation in detail and show that the standard implementation is able to provide bandwidth reclamation under different scenarios. We also study the effect of alternative methods for implementation of Fairness Eligible Shaper. The implementation and simulation parts of the thesis concentrate on the Single Transit Queue Implementation. In the third part we perform analysis of the buffer size requirement under worst case conditions in Dual Transit Queue Architecture. We compare our analysis with the standard and identify various components which affect the buffer size. The buffer size is chosen such that there will be no frame loss due to overflow of the Secondary Transit Queue.

2 INGRESS RATE CONTROL IN RESILIENT PACKET RINGS by Asavari Thombare a thesis submitted to the graduate faculty of north carolina state university in partial fulfillment of the requirements for the degree of master of science department of electrical and computer engineering raleigh July 2005 approved by: Dr. Ioannis Viniotis Chair of Advisory Committee Dr. Paul Franzon Dr. Michael Devetsikiotis

3 Biography Asavari Thombare was born on February 13, 1979, in Pune, India. She graduated with a Bachelor of Engineering (B.E.) degree in Electronics Engineering from Vishwakarama Institute of Technology, Pune. After undergraduate studies, she worked with Paxonet Communications in India for three years. She joined North Carolina State University in Fall 2003 as a graduate student. Her interest is in the field of Computer Network Engineering. ii

4 Acknowledgements First of all I would like to thank my husband Omkar for his constant encouragement. I also thank him for teaching me how to be well organized. I would also like to thank my parents Anil Thombare, Uma Thombare and my brother Amit. Their confidence in me has always motivated me to take all decisions in my life. I express my thanks toward my parents-in-law for encouraging me to complete my higher studies. I thank my cousin brother Saurabh for making me feel at home when I was alone here away from my family for the first time. My sincere thanks to my advisor Dr. Ioannis Viniotis. His deep knowledge about the field has always inspired me. His guidance is of utmost important in this research. I learnt from him how to approach a particular problem with having a big picture in mind. I thank him for his constant support and encouragement. I thank Dr. Paul Franzon and Dr. Michael Devetsikiotis for being my committee member. I also thank Dr. Franzon for introducing me to minute details in ASIC Design. Thanks to my roommates Paramita, Jaya, Sridevi and Subathra, I never had to think about anything else apart from my work. Thanks Anindita for giving me some tips in Latex. Thanks Mohit for reviewing my thesis document. I really enjoyed working with you in the initial phase of our research. I would also like thank IEEE Working Group for clearing some of my initial doubts. In the, thank you 09, 73, L5 fris and Deepak for your wonderful company. Thank you Department of Electrical and Computer Engineering for making available all the tools I used in my research. Thanks to all my fris in EGRC and Partners I for giving response to my queries. iii

5 Table of Contents List of Tables List of Figures vii viii 1 Introduction What is RPR? Demands of Metropolitan Network Emerging Metro Architecture: RPR RPR Station Architecture Why RPR? RPR Features RPR Benefits and Drawbacks Comparison with other technologies Flow Control in RPR Components of RPR Flow Control Need of Shaping and Scheduling Transit Queuing Options Store and Forward Vs Cut-through Architecture Single Vs Dual Queue Architecture Outline of Thesis Background Work The Early Schemes Need of Flow Control Max-Min Fairness Proportional Fairness Flow Control in a Ring Spatial Reuse Protocol Issue of Fairness RPR Flow Control Criteria to choose a Flow Control Algorithm RIAS IEEE Non-standard approaches to Fairness control in RPR iv

6 2.4 Ingress Rate Control Rate Control using Shapers Service Classes and Scheduling Transit Traffic Single Transit Queue Architecture Features of Implementation Implemented Functionality Features of Implementation Behavioral Model in C Functional Description Verification Strategy Hardware Implementation Functional Block Diagram Architectural Features and Design Challenges Implementation Details Hardware Specifications Simulation Study Simulation Environment Objectives of Simulation Study List of Assumptions and Simulation Variables Results of Simulation Performance Analysis Optimization Analysis Compliance Testing Analysis for Dual Transit Queue MAC Purpose of Secondary Transit Queue Buffer Size Requirement Analysis Motivation List of Assumptions Detail Analysis Conclusion and Future Plans Summary of Findings Conclusion Future Plans List of References 91 A Sample Test Script 94 B Data Structures used in Packet Generator and Queue Management Logic 97 v

7 C Verilog Code 99 vi

8 List of Tables 3.1 Gate Count Details User Programmable Parameters which are not part of Test Configuration Test Configuration Parameters Analogy between the Standard Terminology and Our Symbols vii

9 List of Figures 1.1 RPR Ring Architecture RPR Station Architecture Wrap and Steer: Methods of protection in RPR Picture of SRP Credit update mechanism of Shapers Ingress Rate Control: Top Level Functionality Behavioral Model in C Verification Strategy for Behavioral Model Top Level Block Diagram of Hardware Implementation Verification setup for Hardware Implementation Timing Diagram: Selection Controller Handshake Shaper Internal Block Diagram Logic Level Implementation of Shaper and Timer Scheduler Internal Block Diagram State Machine flow in Scheduler Rate Controller (Shaper-Scheduler) Interface Shaper Interface Scheduler Interface Average Rate Calculations (Method1) Average Rate Calculations (Method2) (a) Bandwidth Reclamation in Software Implementation (Method 2) (b) Bandwidth Reclamation in Hardware Implementation (Method 2) (a) Bandwidth Reclamation in Software Implementation (Method 1) (b) Bandwidth Reclamation in Hardware Implementation (Method 1) Bandwidth Reclamation with Dual Transit Queue Architecture (a) Bandwidth Reclamation with return of high priority traffic in Software Implementation (b) Bandwidth Reclamation with return of high priority traffic in Hardware Implementation (a) Bandwidth Reclamation with different FAIR rate values in Software Implementation (b) Bandwidth Reclamation with different FAIR rate values in Hardware Implementation viii

10 4.8 (a) Bandwidth reclamation with sudden change in rate in Software Implementation (b) Bandwidth reclamation with sudden change in rate in Hardware Implementation (a) Unreserved rate limit in Software Implementation (b) Unreserved rate limit in Hardware Implementation FE Shaper response for Byte-wise and Packet-wise credit decrement (a) ClassA shaper response: Software Implementation (b) ClassA shaper response: Hardware Implementation (a) ClassB shaper response: Software Implementation (b) ClassB shaper response: Hardware Implementation (a) Fairness Eligible shaper response for classc: Software Implementation (b) Fairness Eligible shaper response for classc: Hardware Implementation The scenario for analysis of STQ size The steps in analysis of STQ size in a Timing diagram One Propagation Delay Expanded Drain Time Calculation Comparison of the two STQ Size Results ix

11 Chapter 1 Introduction 1.1 What is RPR? Demands of Metropolitan Network After achieving high speed in Local Area Networks (LAN), the technology tr in Networking is moving towards optimizing Metropolitan Area Networks (MAN) to support both voice as well as data-centric traffic. Networks of the size of a city are identified as Metropolitan Area Networks (MAN). For example, often a business located in a city has different departments spread over the city. Their information exchange takes place through MAN. A research conducted by Infonetics Research [1] discusses why Metro Networks are targeted for rapid growth today. The increasing Metro traffic demands for reliable and cost-effective network topology. Ring serves to be the most suitable topology compared to the others like Star, Bus or Mesh Topology. Ring Network does not have a hub like Star Network has, which puts the entire traffic load on the hub. Failure of one link of the Bus breaks the network and hence breaks the communication. This is not acceptable in heavily loaded Metro Network. Mesh becomes very costly as the number of stations increases. As opposed to all these methodologies, Ring provides a reliable structure with two possible paths for the same source and destination pair. Failure of one path does not break the communication between two stations. There is no hub which should work at heavy load and should be capable of handling the traffic for all the source-destination pairs. Every node is connected to only two adjacent nodes making the Ring cost effective. Metro Networks impose certain demands as indicated below. 1

12 Chapter 1. Introduction 2 Integrated Service - Metro Networks demand support to Voice, Video as well as Data Traffic. They demand for multiple service classes with different bandwidth, delay and jitter requirements. Utilization - The available ring bandwidth should be utilized in an efficient way. Static allocation ts to waste the bandwidth. Thus it is required to manage bandwidth dynamically to achieve maximum utilization. Low Cost - The expanding Metro Networks demand for low installation as well as low runtime cost. Robustness - The networking environment today cannot afford break in communication due to network failures. Hence Metro Networks should be robust and less prone to frame loss. QOS - Quality of Service is an important factor especially for the traffic with stringent quality requirements like delay and jitter. The nodes should follow the Service Level Agreement (SLA) for a service class. Scalability - Network should be scalable to support the increasing number of nodes. Overhead of a node addition or deletion should be the least possible. The demands of Metro Networks are explained in more detail in [2] by Luminous Networks. The most popular technologies deployed in Metro Networks today are SONET (Synchronous Optical Network) and Ethernet. SONET is a circuit switched technique with static bandwidth allocation. SONET has inbuilt protection capability in case of a ring failure. On the other side, Ethernet is bandwidth efficient and is optimized for point to point communication. Recovery from network failure is very slow in Ethernet. Thus SONET provides protection but fails to make efficient use of bandwidth and Ethernet makes efficient use of bandwidth but fails to recover fast from the link or station failures. The pitfalls in the existing technologies produced the need for a new technique which will support both protection and better utilization. That motivated the work over a new protocol called Resilient Packet Ring (RPR). But where does RPR fit? The answer to this question is obtained from the name itself. Resilient Packet Ring provides resiliency in the packet transport on Ring that is nothing but in a Metro Network. RPR is targeted towards satisfying all the demands from Metro Networks. The features of RPR mentioned in Section emphasize

13 Chapter 1. Introduction 3 this discussion Emerging Metro Architecture: RPR RPR is standardized as IEEE by IEEE Resilient Packet Ring Working Group which is part of IEEE 802 LAN/MAN Standards Committee. The standard is completely described as IEEE Information technology Telecommunications and information exchange between systems Local and metropolitan area networks Specific requirements Part 17: Resilient packet ring (RPR) access method & physical layer specifications [3]. The standard provides details of Ring as well as station architecture. Station architecture is described in Section RPR ring structure as defined in the standard [3] is shown in Figure 1.1. Station Link S0 Ringlet0 Ringlet1 Congestion Domain S3 S1 S2 Span Figure 1.1: RPR Ring Architecture RPR consists of unidirectional and counter-rotating rings called ringlets. The packets are transferred on the ring and removed at the destination which employs spatial reuse. The term spatial reuse is defined in Section Both the rings carry

14 Chapter 1. Introduction 4 working traffic. Each station is aware of the topology map and hence can forward data on the optimal ringlet towards destination. Here are some architectural definitions from the standard [3] Span - It is a portion of a ring between two stations. Link - Span is composed of unidirectional Links. Station - Station is a node on the Ring. It is identified by a 48-bit MAC address. Congestion Domain - The set of contiguous stations affected by a common congestion point is known as congestion domain. Upstream station and Downstream station - The station receiving traffic from another station is downstream station for the source, while the source becomes the upstream station for the destination on a particular ringlet. For example, consider in the figure above, station S0 ss traffic to station S2 on ringlet 0 then station S0 becomes upstream to S1 and S2 and S2 becomes downstream to S0 and S RPR Station Architecture RPR station architecture as defined in the standard [3] is shown in Figure 1.2. The architecture is mainly divided into following blocks - 1. Control - The Control block is responsible for functionality like OAM (Operations, Administration and Maintenance), Fairness, Topology and Protection. Control signals, which are generated in the control block, control the data flow. It also communicates with the client (higher layer entity) for transfer of control information e.g. OAM messages for management purposes. 2. Ringlet Selection - The control and data packets can be sent on any ringlet. Generally, it is the ringlet with shortest path to the destination. In case of control frames, the Control block decides the ringlet on which the frame should be sent. 3. Ringlet 0 and Ringlet 1 Datapath - The datapath MAC functionality includes shaping, rate control, transmission and reception of a data frame. The receive logic analyzes the frame for the destination address. The frame destined to the

15 Chapter 1. Introduction 5 Transmit Control Received Control Transmit Data Received Data MAC Service Interface MAC Control MAC Control Sublayer MAC Datapath layer Ringlet Selection Ringlet0 Datapath Ringlet1 Datapath Physical Layer Interface Figure 1.2: RPR Station Architecture local station is stripped off and the rest of the traffic is added to the transit queues. The shapers and rate control logic shape the client traffic. Traffic is throttled at the ingress to avoid the frame loss. Transmit logic prepares the RPR frame and schedules it on the ring according to its priority. 4. Transit queues (not shown in figure) - Transit traffic is stored in transit queues. There are two options to implement these queues. In single queue option all the transit traffic goes to only one queue known as Primary Transit Queue (PTQ). In dual queue option there are two queues - Primary Transit Queue (PTQ) carries the high priority traffic while the Secondary Transit Queue (STQ) carries the low priority traffic. Details of Single and Dual Queue Architecture are presented in Section

16 Chapter 1. Introduction Why RPR? RPR Features The key features of RPR, those distinguish it from other network interconnects [3] are given below. 1. Addressing - Unicast, multicast and broadcast. 2. Services - Multiple service classes are supported. Per-service quality flow control protocols regulate traffic introduced by clients. (a) Class A - Allocated/guaranteed bandwidth with a low circumference-indepent jitter. (b) Class B - Allocated/guaranteed bandwidth with a bounded circumferenceindepent jitter. Allows for transmissions of excess information rate (EIR) bandwidths (with class C properties). (c) Class C - Provides best effort service. 3. Efficient - Strategies increase the effective bandwidths beyond those of a broadcast ring. (a) Concurrent - Simultaneous clockwise counter-clockwise transmissions allowed. (b) Reallocated - Allocated bandwidths can be reallocated on non-overlapping segments. (c) Reclaimed - Unused allocated bandwidths can be reclaimed by opportunistic services. (d) Reused - Used opportunistic bandwidths can be consumed by others. 4. Fairness - Fairness ensures proper partitioning of opportunistic traffic. (a) Weighted - Weighted fair access to available ring capacity. (b) Simple - Point-of-congestion flow-control facilitates per-destination queuing in the client.

17 Chapter 1. Introduction 7 (c) Detailed - The (optional) multi-choke fairness allows the client to selectively throttle its transmissions based on multiple congestion point indications. 5. Plug and Play - Automatic topology discovery, initialization of operational parameters, and advertisement of station capabilities allows system to become operational without manual intervention. 6. Robust - Multiple features support robust frame transmissions. (a) Responsive - Service restoration time is less than 50 ms after a station or link failure. (b) Lossless - Queue and shaper specifications avoid frame loss in normal operation. (c) Tolerant - Fully distributed control architecture eliminates single point of failure. (d) OAM - Operations, administrations and maintenance supports service provider environments. These features make RPR stand out in the existing technologies. The existing technologies concentrate on a particular feature and prove to be efficient there, while RPR tries to satisfy all the requirements. Support to static as well as dynamic allocation for the bandwidth is a distinguishable feature of RPR. To provide simultaneous service to low jitter, low delay traffic at the same time to the best effort traffic is a challenge. Static allocation of the bandwidth provides the fixed share to the high priority traffic, but fails to serve opportunistic traffic in an efficient way wasting the bandwidth. If one tries to provide the bandwidth allocation for opportunistic traffic also, it becomes difficult to satisfy the jitter and delay bounds once the traffic enters the network. The solution is to be aware of the network conditions and inject the traffic accordingly in the network to avoid problems at the intermediate nodes or at the destination node. RPR is exactly doing that by providing service to different classes and being efficient in bandwidth utilization. To support this feature, RPR node needs to be aware of the network topology and the congestion information. This is achieved through exchange of information between the nodes, which further makes

18 Chapter 1. Introduction 8 RPR robust. Service classa is subdivided into SubclassA0 and SubclassA1. Bandwidth is strictly reserved for SubclassA0. ClassB is also divided into ClassB-CIR and ClassB-EIR. ClassB-CIR satisfies the Committed Information Rate while the rest of the traffic is best effort. SubclassA1 and ClassB-CIR bandwidth can be reclaimed when not used while ClassB-EIR and ClassC are Fairness Eligible and opportunistic RPR Benefits and Drawbacks The knowledge of basic architecture and features gives us the idea of the benefits and drawbacks of RPR. RPR benefits and drawbacks are summarized here for later comparison with other technologies. Dynamic Bandwidth Allocation - Best effort data traffic does not follow a predictable nature. This traffic generally consists of bursts and idle periods. If the bandwidth is statically allocated, it will be wasted during the idle period which can be utilized by some other flow. RPR provides dynamic bandwidth allocation. Thus it is possible to reclaim the unused bandwidth leading to better utilization. One advantage is that the user can be billed for the usage and need not be for larger circuit regardless of the use. Resiliency - Traffic which has already entered in the ring is not lost in normal conditions. RPR also ensures recovery from the failures in 50ms [3]. RPR provides resiliency without reserving bandwidth for station or link failure conditions. The protection to a failure condition is achieved by two methods known as Wrapping and Steering. In case of Wrapping, the traffic is wrapped at the station at which the link failure takes place. In case of Steering, the traffic is steered by the source to other ringlet which follows a fault free path. The Wrap and Steer methods are shown in Figure 1.3. Plug and Play - A very important demand of today s network is scalability. The size of the network is increasing every day. It is not tolerable to loose traffic or reconfigure the entire system every time a new node is added to the network. The plug and play feature of RPR is a solution to this

19 Chapter 1. Introduction 9 Wrap here Source Failure Failure Steer the traffic on this route Wrap here WRAP STEER Figure 1.3: Wrap and Steer: Methods of protection in RPR problem. RPR nodes periodically exchange the Topology information. Thus addition of a new node need not reset and reconfigure entire system. Instead the modified topology information changes the database of all the existing nodes. Hence runtime addition as well as deletion of a node is possible in RPR. Complexity - Support to all these features mentioned in Section is not an easy task. It definitely has some repercussions. It introduces complexity in the design and architecture. The implementation complexity of RPR MAC (Media Access Control) is more as it needs to take care of many factors simultaneously. Especially the dynamic allocation algorithm is difficult to implement Comparison with other technologies Various technologies were developed using the Ring topology. For example, FDDI, DQDB and Token Ring use the ring topology but are very inefficient. Only one packet is transmitted at a time and the packet travels the entire ring. A protocol called Metaring [4] employs spatial reuse explained in Section A control message SAT controls the transmission of packets by each node. But it suffers from some drawbacks due to fixed quota size in SAT [5]. As described earlier, SONET and Ethernet are the widely used technologies today in the Metro Networks.

20 Chapter 1. Introduction 10 The benefits of SONET can be summarized as follows. SONET is circuit switched. Hence it is capable of providing reserved resources to the voice or video traffic. It is a suitable technology for low jitter, low delay requirement traffic. It is resilient. SONET provides backup for the link or span failure scenarios. Thus SONET makes efficient use of the Ring topology. SONET provides point to point data transport with the help of Packet Over SONET (POS) technology. The benefits of Ethernet are summarized below. Ethernet makes efficient use of the bandwidth available to data traffic. It does not reserve a bandwidth and hence can use the available bandwidth whenever required. Ethernet is a cost effective solution. Ethernet is most popular and widely deployed in Local Area Networks. Looking at the benefits of RPR explained in Section 1.2.2, it is understood that RPR is nothing but a combination of both - SONET and Ethernet. It covers the pitfalls of both technologies and hence becomes the most suitable technology for Metro Networks. Here are some parameters to compare the performance of these three technologies. 1. Robustness - Ethernet implements spanning tree protocol for path setup. Thus in case of a failure, the spanning tree needs to be recomputed. This is time consuming. Hence fault recovery in case of Ethernet is very slow compared to SONET and RPR. SONET and RPR provide protection mechanisms. 2. Throughput and Utilization - SONET provides poor utilization when supporting opportunistic traffic. The fixed resources waste the bandwidth when not used. Ethernet and RPR make efficient use by reusing the idle bandwidth. 3. Jitter - SONET uses circuit switched technique and hence can satisfy the jitter requirements. RPR also provides reserved bandwidth for the traffic with

21 Chapter 1. Introduction 11 stringent jitter requirements and thus tries to satisfy them. Ethernet does not reserve resources hence may not satisfy the stringent requirements. 4. Ease of Implementation - SONET has a complexity in implementation due to requirement of the synchronization. RPR is complex as explained in Section Compared these two techniques, Ethernet is easy to implement. 5. Fairness - Detail explanation on this point is given in Section Frame Loss - Frame loss is minimized in RPR by controlling the rate at the ingress point itself. Ethernet suffers from frame loss in case of congestion. Wong et al. provide the comparison on POS, Gigabit Ethernet and RPR in [6] based on simulation studies. RPR Alliance also gives a comparison chart for RPR, Ethernet and SONET [7]. This comparison strengthens the choice of RPR for Metro Network. 1.3 Flow Control in RPR Components of RPR Flow Control Flow control in RPR is necessary to satisfy the claimed features. It is important that each station manages the traffic on the ring to achieve the objectives of Integrated service, Maximum utilization, QOS and Fairness. Flow Control includes service class priorities to support Integrated service and QOS, Feedback control to adjust the rates such that there is no congestion downstream, proper rate distribution to meet Fairness and Bandwidth reclamation to achieve maximum utilization. To achieve these objectives, RPR employs a Feedback Flow Control Algorithm. Downstream stations indicate upstream stations about their congestion status. Upstream node is expected to reduce the rate to FAIR rate value in case of downstream congestion. Integrated service, Maximum utilization, QOS are directly related to the demands of the flow but Fairness does not relate to a single flow. Then why do we need Fairness? The reason is the nature of traffic. The traffic today is dominated by the best effort data traffic. Everybody wants to s at a higher rate. But ring is a shared medium. The need is now to have proper bandwidth allocation for all the stations so that

22 Chapter 1. Introduction 12 source stations which are close to destination do not get advantage of bandwidth. All stations should get a FAIR chance to s the traffic on the shared medium. There are two main components of RPR flow control. Feedback Messages - The dynamic rate control can be achieved with the help of feedback messages. Every station advertises the congestion status through the fairness message to the upstream stations. The fairness message consists of the FAIR rate value. In case of congestion, the station advertises its local rate (the rate at which the local client traffic is transmitted) to the upstream station. Rate control at ingress point - The feedback fairness messages are used to do rate control at the source station. The idea is to throttle the traffic at the ingress point itself to avoid further problems. Rate control is achieved with the help of Shapers in the Transmit logic of the MAC datapath. The FAIR rate advertised by downstream station is used to shape the traffic. There are some alternative approaches also available to implement the fairness. First comes the static rate allocation. But this is very inefficient for the opportunistic nature of the best effort data traffic. Window based Control is also one of them [8]. In this case the feedback travels from destination to source. Thus the feedback delay affects the convergence time to achieve fair rate. Another one is per flow control [9]. The problem with flow based control is the granularity of control. Complexity increases with the finer granularity. There are many such algorithms which are suggested for FAIR rate control. IEEE chose the described approach for the reasons that the rate adjustments are done on the ingress traffic and not on flow granularity. The closest downstream station controls the rate and hence the delay in feedback message transmission is minimal. At this stage it is possible to compare the fairness property of RPR with the other technologies. SONET provides static allocation and hence does not explicitly implement fairness. Ethernet does not implement any distributed fairness algorithm. Thus the results are generally undesirable. The station closest to the destination ts to get the maximum share of bandwidth. RPR implements a distributed fairness algorithm. Congestion notification travels through the stations, controlling their FAIR rates. Thus desirable FAIR rates can be achieved through RPR fairness algorithm.

23 Chapter 1. Introduction Need of Shaping and Scheduling We concentrate on the second component of RPR Flow Control i.e. Rate Control at Ingress. Traffic is throttled to the FAIR rate advertised by the downstream station. Two main sub-components are identified to achieve rate control - Shaping and Scheduling. Shaper is required to shape the traffic to allocated rate. It is the responsibility of the station to prevent any service class traffic from exceeding its allocated rate. In case of RPR, it is also the responsibility of the shaper that the rate of fairness eligible traffic does not exceed FAIR rate value. RPR provides flow control for the unreserved traffic i.e. traffic for which the bandwidth is not reserved. Hence Shaper should also control the aggregate rate of unreserved traffic so that reserved bandwidth remains intact even when it is not used. The shaped traffic needs to follow the service class priorities. Scheduler takes care of this. It is responsible not only for the service class priority of the client traffic but also for the priorities of transit traffic over client traffic. In dual queue architecture, client traffic gets priority over the Secondary Transit Queue (STQ) when the occupancy of the queue is below certain threshold. After that threshold, STQ gets priority over the client traffic. Scheduler takes care of this priority inversion. Shaping and Scheduling together implement Bandwidth Reclamation. When subclassa1 and classb-cir traffic is not using its allocated bandwidth, it can be reclaimed by the Fairness eligible traffic i.e. classb-eir and classc. 1.4 Transit Queuing Options Store and Forward Vs Cut-through Architecture IEEE standard gives option of implementing the transit queues with either store and forward or cut-through architecture. In store and forward option, frame forwarding can only if the complete frame is received in the transit queue. The advantage of store and forward architecture is simplicity in implementation. Complete frame has been analyzed before forwarding it. Latency experienced by frame is more

24 Chapter 1. Introduction 14 as the entire frame needs to be received before forwarding s. In cut-through architecture, frame forwarding can start as soon as the header is received in the transit queue. Only header information is available when forwarding is done. Advantage of cut-through architecture is that it is faster. Latency experienced by the frame inside node is minimal. Another advantage is that the storage space required is less as compared to store and forward. We choose to implement store and forward as the frame length is already available when the forwarding takes place. Hence it is easy to make credit adjustments on the frame by frame basis and no need to count the bytes during transmission. The credit adjustments are required by the Shaper and they are explained in Section Single Vs Dual Queue Architecture Single transit queue consists of Primary Transit Queue (PTQ). Entire transit traffic is stored in this queue. Transit traffic always gets priority over the client traffic. This ensures simplicity and lesser storage space. But with single queue it is not possible to provide service class differentiation within transit traffic. High priority client traffic can also starve if transit traffic gets priority always irrespective of its service class. Dual transit queue Architecture consists of Primary as well as Secondary Transit Queue. ClassA transit traffic is stored in PTQ and hence always gets priority over the client traffic. STQ stores classb and classc frames. Client traffic gets priority over STQ till it reaches a certain threshold. Once STQ occupancy goes above this threshold, priority inversion takes place and STQ gets priority over client traffic. STQ needs to be bigger to avoid frame loss due to overflow. But it ensures service class priority in transit traffic and also avoids the starving of client due to high priority transit traffic. Our initial implementation considers Single Transit Queue Architecture. We also present analysis for dual queue architecture in Chapter Outline of Thesis The thesis concentrates on the Ingress Rate control component of RPR Flow Control Algorithm. Chapter 1 presents an Introduction to the Resilient Packet Ring (RPR)

25 Chapter 1. Introduction 15 protocol and introduces the concept of Rate Control at Ingress. Chapter 2 then discusses Background Work in this area. In Chapter 3, Single Transit Queue Rate Control Implementation is explained. It covers both Hardware as well as Software Implementation. Chapter 4 presents Simulation Study of the Shaping and Scheduling schemes. Here we observe the response of individual shapers as well as the combined effect of shaping and scheduling schemes. Chapter 5 moves on to Dual Queue Architecture. Here we present the analysis for buffer size estimate of the Secondary Transit Queue. Discussion of conclusion and future plan wraps up the document in Chapter 6.

26 Chapter 2 Background Work The background work described in this chapter discusses early flow control schemes followed by the advancements proposed to achieve better performance. This discussion then leads to SRP and RPR flow control. The components of RPR Flow control are introduced along with discussion of different schemes. We concentrate on Ingress Rate Control component of flow control mechanism. Thus the of the chapter concentrates on the discussion of Ingress Rate Control mechanism which is part of the RPR MAC. 2.1 The Early Schemes Need of Flow Control Ring is a shared medium for several stations situated on it. Hence the ring bandwidth needs to be distributed among the stations in a FAIR way. Increasing number of stations and opportunistic nature of traffic both demand for a fair bandwidth allocation algorithm. The demands of Metro Networks mentioned in Section 1.1 also need to be satisfied. This can be dynamically achieved with the help of Flow Control. Several algorithms are suggested for Flow Control and Fairness. The early schemes like Max- Min Fairness and Proportional Fairness are discussed next. The later algorithms can be traced backward to the concepts presented in these early schemes. 16

27 Chapter 2. Background Work Max-Min Fairness Max-Min fairness explained in [10] concentrates on achieving maximum utilization by allotting fair rates to the competing stations. The concept is to allocate equal rate to every flow and then distribute the remaining bandwidth to achieve the maximum utilization. By dividing the rate equally may not result into maximum bandwidth utilization. The idea is to give the largest possible share to the poorly treated user [11] or maximize the minimum share of a source whose demand is not satisfied [12]. In case of different service requirements, the flows get weighted share of the bandwidth instead of getting equal share. The definition dictates the feasible rate matrix for the traffic flows. Definition [10] [11] : Let 1,...,R be the set of users accessing the network, let L be the set of all links in a network, and let C l be the capacity of link l L. Let H l be the set of all users who pass through link l. We call a rate allocation (x 1,..., x R ) feasible, when for every link l L we have that r H l x r C l We call a feasible allocation (x 1,..., x R ) max-min fair, when it is impossible to increase the rate of a user without losing feasibility or reducing the rate of another user r with a rate x r x r Proportional Fairness Proportional Fairness is also one of the early schemes like Max-Min fairness explained in previous section. Detail explanation of Proportional Fairness can be found in [8] [13]. The goal here is to achieve efficiency again. Giving maximum possible share to the poorly treated user might result into some inefficiency. Proportional Fairness tries to overcome that. The definition of Proportional Fairness algorithm is given below. Definition [14] : An allocation of rates x is proportionally fair if and only if, for

28 Chapter 2. Background Work 18 any other feasible allocation y, we have: Ss=1 y s x s x s 0 This means that any change in the allocation will have negative average. Max-Min fair rate allocation may not always result into Proportional fair rates. Proportional fair rates can result into proportionally increased bandwidth for the flow traversing less number of hops [15]. 2.2 Flow Control in a Ring Spatial Reuse Protocol Ring networks are identified as the most suitable topology for Metro Networks. But the existing technologies do not make efficient use of it. Thus there was a requirement for a protocol which makes use of the unused spans of the ring to improve the efficiency. One such effort which is very popular today is SRP i.e. Spatial Reuse Protocol. But what is SRP or Spatial Reuse Protocol? Ring is a shared medium and there are multiple stations which access the ring. In earlier technologies like IBM Token Ring, only one station used to access the ring at a time. In later technologies, multiple stations shared the bandwidth and could transmit at the same time by multiplexing the traffic. But still every packet travels the entire ring and is stripped off when it returns to the source. Instead the packet can be stripped of at the destination itself so that the part of the ring from destination to the source can be reused. This is known as Spatial Reuse. Figure 2.1 gives a picture of SRP in ring. Simultaneous transmissions from S0 to S1, S2 to S3 and S0 to S2 can go on the ring. CISCO presented SRP MAC Layer Protocol in [16]. As explained in earlier section, the packet transmitted on a ring is removed at the destination to achieve spatial reuse. To control the access to the ring every station on the ring executes a distributed transmission control algorithm known as SRP Fairness Algorithm (SRP-fa) [17]. SRPfa is developed to achieve following goals [17] Global Fairness - Every station on the ring should get a fair share of bandwidth.

29 Chapter 2. Background Work 19 S0 S3 S1 Spatial Reuse Reuse the unused span S2 Figure 2.1: Picture of SRP Local Optimization - Stations can transmit at more than their fair share on the local unused link. Scalability - The algorithm should be scalable enough so that it can rapidly adapt the changing traffic and the increasing number of stations. Apart from the fairness feature, SRP also supports Intelligent Protection Switching (IPS) e.g. the traffic is wrapped at the failure point. SRP also supports Topology Discovery mechanism to ease the addition or deletion of the node to the ring Issue of Fairness We have seen the necessity of Fairness in the Flow Control Algorithm. The mechanism of Fairness control in RPR is similar to that of SRP. Each station monitors the local traffic added by it and the traffic that transits through it. Once it detects the congestion, it ss its usage through a feedback message to upstream stations. Upstream station is expected to adjust its transmission rate according to the received

30 Chapter 2. Background Work 20 rate advertisement from the downstream congestion node. SRP-fa restrictions are applicable to the low priority traffic only. The high priority traffic can transmit at the committed rate. The station detects the congestion by monitoring the occupancy of the low priority transit buffer. When the occupancy exceeds the threshold set, congestion feedback message is sent upstream. If the receiving station is also congested, it transmits the minimum of its own usage and the received rate. Otherwise, the station transmits the same rate to upstream station. Fairness messages always travel on the opposite ringlet as that of the data. Fairness messages are also used as keep-alive messages in case of no congestion. There were several algorithms suggested for fairness on rings with spatial reuse before that. The popular one was SAT based algorithm which had a SAT (satisfied) feedback signal controlling the transmission on the ring at each station [4]. But it suffered from starvation problem [5]. [5] suggests some modifications to improve the performance of the SAT based algorithm which includes improvement in throughput and reduction in latency. One suggestion is to monitor the active stations and the other is to keep a hop counter of non-starved downstream node for better control on ingress rates. These improvements bring it closer to the SRP-fa described above. There are some more extensions suggested for SRP-fa. These extensions are discussed in [18]. One suggestion is to give the feedback indication to all the stations on the ring and not only the next upstream station. This is required so that all the stations on the ring will be aware of the congested points and will throttle the traffic at the ingress itself. It also suggests destination queuing so that the traffic going to congested station can be throttled accordingly. The MAC architecture can be improved to Minimize the transit buffer size. Support destination based queuing. Shape the transmit traffic to provide better delay and jitter. Support single as well as multiple transit queues. IEEE concentrates on these problems to provide a better solution.

31 Chapter 2. Background Work RPR Flow Control Criteria to choose a Flow Control Algorithm There are some improvements required in SRP which are discussed in the previous section. RPR flow control mechanism tries to make these improvements keeping in mind the goals described below. Throughput - RPR tries to achieve maximum aggregate ring throughput by implementing spatial reuse and appropriate bandwidth allocation. Utilization - Maximum utilization is achieved by allowing the reclamation of unused bandwidth on every link. Latency - Latency can also be viewed as response time. Once the rate advertisement is received, how much time the station takes to throttle the ingress traffic is the latency. Minimum latency is desired. Convergence time - This is the time required to converge to the FAIR rate after the congestion notification is initiated on the opposite ringlet. More convergence time leads to more throughput loss due to oscillating rates. Ease of implementation - The implementation of the flow control logic should not be very complex. Complex designs are always difficult to upgrade. The optimal Flow Control algorithm will be the one which satisfies all the above mentioned goals. To achieve the above mentioned goals, the decisions must be taken at the source. If the traffic is rate controlled at the ingress itself, there is less possibility for traffic loss when it is traversing the ring. Thus the RPR flow control algorithm aims on providing a distributed protocol to throttle traffic at their ring-ingress points to their ring-wide FAIR rates [19] RIAS It is required to have a clear target for the design of Flow Control algorithm for RPR especially to achieve Fairness. Thus there is a need for idealized reference model. The algorithms then can be compared with this reference model than comparing them with each other. Ring Ingress Aggregated with Spatial Reuse (RIAS) fairness

32 Chapter 2. Background Work 22 is that reference model which was presented in [20] and then described in [15]. The definition of RIAS is given below. Definition [15] : A matrix of rates R is said to be RIAS fair if it is feasible and if for each flow (i,j), R ij cannot be increased while maintaining feasibility without decreasing R i j for some flow, j ) for which (i R i j R ij, when i = i IA(i ) IA(i) at some common link otherwise (IA - Ingress Aggregated) The constraints for matrix R to be feasible are described below. R ij > 0, for all flows (i, j) F n C, for all links n R ij is the rate allocated to flow (i, j), C is the capacity of the link and F n is the allocated rate on link n which is F n = all flows (i,j) crossing link n R ij The definition of RIAS looks similar to that of Max-Min Fairness but they are different in terms of granularity. RIAS implements two levels of granularity one is the Ingress Aggregated flow for all flows originating at same source and the second is for the individual flows under it. RIAS ensures that a station will always receive the minimum bandwidth of C/N where N is the number of stations on the ring. The weights can be applied to ingress nodes to provide them with the weighted share according to the service requirements IEEE IEEE is standardized by IEEE RPR working group. It describes the fairness control mechanism to be followed. The fairness control mechanism was evolved from different proposals. Aladdin and Gandalf proposals were one of the initial proposals. Darwin proposal was developed as a combination of these two proposals. Darwin provides the RIAS fair shares to the stations [20]. The comparison

33 Chapter 2. Background Work 23 of different proposals is given in [21]. The Objectives/Procedures to be followed are described by the standard [3] as shown below. 1. Support indepent fairness operation per ringlet. 2. Carry control information on the ringlet opposing that of the associated data flow. 3. Regulate only classc and classb-eir (i.e. fairness eligible) traffic. 4. Compute fair rates associated with a source station (vs., for example, a sourcedestination station pair). 5. Separately regulate aggregate fairness eligible traffic added to the ringlet and that portion of added fairness eligible traffic that transits a point of congestion. 6. Scale fair rates in proportion to an administrative weight assigned to each fairness instance. 7. Allow ringlet capacity not explicitly allocated to be treated as available capacity. 8. Allow ringlet capacity explicitly allocated to subclassa1 or classb-cir, but not in use, to be treated as available capacity (i.e., bandwidth reclamation). 9. Support either single transit queue or dual transit queue deployment. 10. Support either the aggressive or the conservative rate adjustment method. 11. Optionally report rate information to MAC client. 12. Adjust fair rates within a few fairness round trip times (FRTTs) of the occurrence of a change in traffic load (i.e., fast response time). RPR fairness control mechanism works on the basis of feedback messages and rate control at the ingress as already explained in Section Some details about the fairness messages and rate control are given here. The important steps in fairness control as explained in IEEE [3] are described below.