Traffic Management Shaping. White Paper. Northforge Innovations Inc.

Transcription

1 Traffic Management haping White aper Northforge Innovations Inc. eptember 017 1

2 Traffic Management roviding appropriate quality of service is increasingly important as the variety of networking applications increases. ome applications depend on low delay, others on low delay variation, some depend on low levels of frame loss, and many depend on combinations of these performance attributes. Achieving the appropriate quality of service for each flow through the network requires the participation of both the user and the network provider. Both parties need to ensure that at each transmission opportunity higher priority traffic is favored over lower priority traffic. This is call port scheduling and was the topic for the first whitepaper in this series. The network provider needs to ensure that the network is not overloaded which causes both delay and loss by engineering the network to carry the committed amount of traffic and then by ensuring that each flow only sends its committed amount of traffic and does not generate unreasonably long bursts of traffic. Limiting each flow to its committed traffic levels is called policing and was the topic of the second whitepaper in this series. The last piece of the puzzle is for the user to present a well-behaved traffic stream to the network that meets the agreed upon bandwidth and burst limits. This process is called shaping and is the topic of this whitepaper. haping When purchasing data services, subscribers don t want to purchase more bandwidth than they need. If an application requires 00Mbps of bandwidth to do its job, buying a gig is overkill and more importantly, it s expensive. But Ethernets come in fixed sizes (10M, 100M, 1G, 10G, etc.) and therefore, even if the service provider is willing to sell a 00Mbps service, the service will likely be delivered on a 1Gbps interface (assuming all Ethernet connectivity). But if the subscriber has a 1Gbps interface then, in theory, it can send 1Gbps. o the service provider has to police the incoming traffic to ensure that the subscriber is not sending more than it purchased. As noted in the previous whitepaper, policers are not friendly. If the data stream does not match the agreed to bandwidth profile (which includes bandwidth and burst limits) the policer just drops frames until it does match. The policer doesn t try to make the stream conform to the bandwidth requirements (for example, by spacing out packets to reduce the average bandwidth), it just meters the stream as presented. If the subscriber has multiple sessions running across a network service, and each session is creating bursts of traffic, it is possible for several sessions to end up bursting at the same time and creating an aggregate stream that exceeds the bandwidth profile and packets will be dropped when they hit a policer. This is not desirable and, in most cases, is not necessary. If the subscriber aggregation device (usually an Ethernet switch) can buffer the incoming streams and play them out at a rate that matches the bandwidth profile, then no packets will be dropped by the policer. A win-win. This process of making the traffic stream meet a bandwidth profile is called shaping. How Is haping Done? There are a few ways to implement shaper functionality. Normally it is implemented in the switching device (switching AIC or network processor) so you don t have to worry about how to implement it, but describing the implementation of a simple shaper can make it easier to understand what it does and provide insight into proper configuration. A shaper builds a buffer for all traffic that is associated with a single bandwidth profile (e.g. all traffic with Class of ervice Medium ). It is possible that at an interface there are different streams with different classes of service, in which case there is a shaper buffer and a shaper for each class of service, as shown in the following diagram.

3 Customer Aggregation Device (Ethernet witch) User App 1 (H) User App (H) User App 3 (M) User App 4 (M) haper Buffer H haper Buffer M H haper M haper ort cheduler To ervice For each time period, the shaper can send a particular number of bytes based on the bandwidth profile. For example, if the shaper is based on a 1ms timer and the Committed Information Rate or CIR is 80Mbps, then the shaper will send up to 10K bytes (80K bits) each millisecond (rounded to the nearest packet boundary). In the following diagram we see a burst of six packets arriving at the customer aggregation switch. In the top flow there is no shaper and the packets leave the customer switch with the same spacing that they were received with. In this case the policer at the service provider might (depending on the parameters) end up discarding the last packet because the token bucket was empty by the time it arrived. In the bottom flow the customer switch has a shaper. The shaper sends five of the six packets in the first millisecond and waits until the next millisecond to send the sixth packet. In this case all six packets are passed by the policer. When the port scheduler requests another packet from the shaper, the shaper will determine if it has sent its quota for the current time period. If it hasn t, it will give the port scheduler the next packet in the buffer. If it has, then the shaper will return nothing to send and the port scheduler will move on to the next class of service. Customer witch 1 Millisecond 1 Millisecond Millisecond haper olicer 1 Committed Information Rate is the bandwidth that the network provider is guaranteeing to the user. This parameter and related parameters are described in the olicing Whitepaper. The olicer is implemented using a Token Bucket Algorithm. The Token Bucket is a device (an abstraction) that, in effect, measures/maintains the running average of the user s bandwidth. 3 As noted in the olcing White aper, a ervice rovider can optionally sell excess bandwidth in addition to committed bandwidth. In that case the sixth packet in this example might have been declared yellow and forwarded without delivery guarantees instead of being declared red and discarded. 3

4 Where is haping Done? There are three common models for where shaping is done. haping is normally considered an egress or transmit function (as opposed to policing which is normally considered an ingress or receive function). The simplest model implements shaping in the subscriber s aggregation device. haping into a data service is, in theory, the subscriber s responsibility so doing it in the subscriber s device is the logical choice. This is shown in diagram a below. Many subscriber devices do not implement shaping (or the subscribers don t want to or don t know how to enable it). o, the second possibility is that the service provider can provide the subscriber with a device (frequently called a Network Interface Device or NID) that can do shaping for the user as well as other management functions. This is shown in diagram b below. The third possibility is that the service provider can implement a shaper at the ingress to the service either before the policer or instead of the policer. This is a little unusual since it requires multiple buffering steps at the receiver, but there are switching devices that support this (such as the Broadcom DNX) and there are real world environments where this approach is taken. Customer witch ervice rovider a: b: c: haper(s) olicer(s) Are There Any roblems Caused by hapers? haping is a pretty straightforward function and therefore there are very few issues associated with shapers. The two most common issues are: 1. hapers drop packets The goal of the shaper is to provide a conforming stream of packets to the policer so the policer does not drop packets. But if the traffic coming in to the shaper is at a much higher level than the CIR and it continues for a very long burst, then the shaper buffer will fill up and subsequent packets will be dropped. This is not really a problem, it is actually the intention of how the shaper works. If the users are generating packets that exceed CIR for a long period of time then packets must be dropped in order to get the stream to conform. It is a matter of when and how many, and in most cases, the shaper will drop fewer packets than the policer.. hapers add delay A shaper adds a buffering step to the data stream so it does, by definition, add delay. Again, this is really part of what a shaper does. If the data sources generate 1K bytes in a millisecond as in the diagram above (assuming the 80Mbps CIR described above) then it is possible that a policer will drop some of the later packets in the burst in order to get the stream to conform. A shaper will buffer the stream and play out the 1K bytes over two 1ms time periods thereby ensuring that no packets are dropped with the trade-off that some packets are held up for a short period. 4

5 Is added delay a problem? Yes, no, or maybe depending on the application. For most TC-based streams the answer is no. TC adapts its transmission rate to the actual delay in the network, so adding a shaper will, in effect, cause TC to throttle down its transmission rate which has the side effect of reducing the amount of buffering by the shaper. And it turns out that this is important (see the next section). For some realtime streams (frequently using UD) added delay may be a problem. For example, if the delay in a voice stream exceeds a threshold (around 30ms) there can be echo problems that would need to be dealt with. Also, high delay in voice applications can result in unacceptable results such as talkover where one or both sides of a conversation speak while the other side is speaking. Realtime streams, such as voice and video, are frequently more sensitive to delay variation than to absolute delay. hapers do, also, increase delay variation, so this needs to be considered. Consider the 1K example above. The first 10K bytes will be sent out in the first millisecond and the next K may be held up until the next millisecond. o not only is their absolute delay greater, but there is also a difference in the amount of delay experienced by the first part of the burst vs. the second part. Delay variation is an issue because these applications include a playout buffer at each receiver. The purpose of this buffer is to compensate for the reality that packet networks have delay variation which is anathema to a constant bit rate data stream. If the delay variation is high then the buffer needs to be larger which, of course, increases the delay further and if the delay variation exceeds the expected amounts the receiver can underrun resulting in a glitch in the audio or video stream (or the dreaded buffering ) One Important roblem olved By A haper Beyond the general function of creating a better behaved stream of packets resulting in fewer packets being dropped, there is another important benefit of using a shaper. It has long been recognized that TC sessions frequently have very poor performance when they are being policed. A TC session might only get 10Mbps through on a 80Mbps service. Although this problem has been around for a long time, very little analysis of the problem had been done and simple band aids were implemented to get around it, the most common being very large values for CB, the policer bucket size. Over the last few years the MEF Forum analyzed the problem and determined that the problem is caused by a negative interaction between the operation of the policer and TC s congestion control algorithm. TC was designed for a network (like the Internet) that was able to deliver bandwidth at some instant in time and the bandwidth varied up and down in an analogue fashion over time. With a policer, however, the model become more binary a lot of bandwidth is allowed through until the token bucket is empty at which time the bandwidth is zero until the bucket filled a little, then high bandwidth, then zero, etc. This, in effect, drives TC s congestion control algorithm crazy. The solution to this problem is to put a shaper into the stream. In reality the shaper can be anywhere in the subscriber s network or the provider s network. The shaper, in effect, smooths out the amount of available bandwidth as perceived by TC and also adds delay which causes TC to throttle down its transmission rate to adapt to the available bandwidth. With no changes other than including a shaper, TC performance can increase from, in some cases as low as 10% of CIR, to values in the 90%+ range! The analysis and explanation as well as additional description of the shaper algorithm can be found in Appendix G of the MEF Class of ervice Implementation Agreement (MEF3.). 5

6 ummary This is the final paper in the set of three white papers on Traffic Management. Three important functions were discussed, ort cheduling, Traffic olicing, and Traffic haping. Deploying these functions are an important part of developing network solutions that have the quality of service needed for correct operation of the various applications using the network, and to assist in the appropriate engineering for the network. But deploying these functions comes with a set of challenges. Choosing the correct values for the numerous parameters is an obvious challenge. Each decision comes with its own issues and tradeoffs. Implementing support for the functions in the switching AICs and network processors provides the next set of challenges. Monitoring the results, ensuring that the network is operating correctly, and meeting the Qo commitments to the users also requires implementation of appropriate tools and techniques. Developing these functions requires both network design knowledge and implementation expertise. Northforge Innovations has an experienced team of software engineers that has helped many clients with network device design and implementation. In addition, Northforge is a development partner with several major switching device (AICs and Network rocessors) vendors such as Broadcom and Cavium. If you are developing a network device that includes packet switching (Ethernet switching, routing, etc.) and need help designing and implementing the embedded software for the product, Northforge can help! NORTHFORGE INNOVATION INC. GATINEAU DEVELOMENT CENTER (Development Center) 7 Laval treet, 3rd Level Gatineau (QC) J8X 3H3 MONTREAL DEVELOMENT CENTER 40 aint-nicolas treet, uite 06 Montreal, QC HY 5 NORTHFORGE INNOVATION IRAEL LTD. 10 Zarchin t. Ra anana O Box 4318 Israel 6 UA OFFICE (ales Office) One Boston lace, uite 600 Boston, MA 0108 General Inquiries Consulting Inquiries info@northforgeinc.com