DEPLOYING QoS IN SERVICE PROVIDER NETWORK

Size: px

Start display at page:

Download "DEPLOYING QoS IN SERVICE PROVIDER NETWORK"

Alisha Bond
5 years ago
Views:

1 DEPLOYING QoS IN SERVICE PROVIDER NETWORK EDGE QoS FOR LAYER3 VPN Andy Chien Consulting System Engineer Cisco Systems 1

3 Service Provider High Level QoS Requirements A Service Provider Needs to: Support enterprises with diverse QoS policies (classes, markings, PHBs) Ensure contracted rates per SLA Ensure loss, latency, jitter per class per SLA Maintain QoS transparency for customers Measure SLA metrics and report Capacity planning based on SLA metric measurements Enterprise Enterprise Service Provider Network Enterprise Residential Small Business 3

4 How Many Enterprise Traffic Classes Expected? Ent1 Ent2 Class Voice Critical Trans BE Class Critical Trans BE IPP IPP Class Voice Critical Trans BE IPP Class Voice Critical Trans BE SP Network IPP Class Voice Critical Trans BE Class Critical Trans BE IPP IPP Ent1 Ent2 1. SP expects multitude of enterprise traffic classification schemes 2. Single classification scheme for service provider edge 3. Enterprise classes mapped to SP class at the ingress edge 4. SP classes mapped to enterprise classes at the egress edge 4

5 How Many Enterprise Traffic Classes Expected? A Service Provider s Classification Policy Should Be Able to Accommodate/Aggregate Enterprise Classes 4/5 Class Model Realtime Call Signaling Critical Data Best Effort 8 Class Model Voice Video Call Signaling Network Control Critical Data Bulk Data Best Effort QoS Baseline Model Voice Interactive-Video Streaming Video Call Signaling IP Routing Network Management Mission-Critical Data Transactional Data Bulk Data Best Effort Time Scavenger Scavenger Scavenger Possible Enterprise Traffic Class Evolution Over Time 5

6 Service Provider Edge Traffic Classes Service Provider (SP) Edge Traffic Class Design Goals 1. Edge class design should support Application isolation (min bandwidth) SLA differentiation (latency, loss, jitter) 2. Find optimal number of SP classes to accommodate expected customer classes today and in future Combine a set of enterprise classes with similar SLA attributes in to an SP edge class Note: Can t guarantee traffic Isolation among the enterprise classes within a single SP edge class 3. Protect SP own traffic, e.g., telnet to managed CE 6

7 Service Provider Edge Traffic Classes Edge Traffic Class Design Considerations 1. Combining/separating Voice and Video in the same class 2. Combining/separating Call Signaling and Voice in the same class 3. Combining/separating TCP and UDP in the same class 4. Marking considerations Use DiffServ recommended codes (EF vs. CS5)? 5. Protection needed for any SP class Telnet to managed CPE (NMS)? Routing to managed CPE (routing class)? solutions_white_paper0900aecd802296dd.shtml 7

8 Service Provider Edge Traffic Classes 11 Enterprise Traffic Classes Mapping to Three Edge Classes Application DSCP SP Edge Classes Routing Voice CS6 EF EF Realtime Interactive Video AF41 CS5 CS5 Streaming Video Mission Critical Data Call Signaling Transactional Data CS4 DSCP 25 AF31 AF31/CS3 CS5 AF21 CS3 X CS6 AF31 CS3 Critical Data Network Management Bulk Data CS2 CS3 AF11 X Scavenger CS1 0 Best Effort Best Effort 0 Three Edge Classes Can t Cleanly Accommodate All 11 Classes e.g., Streaming Video, Bulk Treated as Best Effort 8

9 Service Provider Edge Traffic Classes 11 Enterprise Traffic Class Transaction into Four Edge Classes Application DSCP SP Edge Classes Routing Voice Interactive Video Streaming Video Mission Critical Data Call Signaling Transactional Data Network Management Bulk Data Scavenger CS6 EF AF41 CS5 CS4 DSCP 25 AF31 AF31/CS3 CS5 AF21 CS3 CS2 CS3 AF11 CS1 0 Best Effort 0 X EF CS5 CS6 AF31 CS3 Realtime Critical 25% In Contract = 3 Out = 1 AF21 Transactional CS2Streaming Video Best Effort Either BULK or Video Can t Be Cleanly Supported 9

10 Service Provider Edge Traffic Classes 11 Enterprise Traffic Class Transaction into Five Edge Classes Application Routing Voice Interactive Video Streaming Video Mission Critical Data Call Signaling Transactional Data Network Management Bulk Data Scavenger EF AF41 CS5 CS4 DSCP CS6 DSCP 25 AF31 AF31/CS3 CS5 AF21 CS3 CS2 CS3 AF11 CS1 0 Best Effort 0 AF21 SP Edge Classes DSCP EXP EF CS5 CS6 AF31 CS3 AF21 CS2 AF11 CS1 CS0 Realtime Up To 35% Max Critical 20% Video 15% Bulk 5% 1 Five edge classes can support all the 11 enterprise classes Less than 11 classes more common possible to use < five classes 10

11 Service Provider s Own Edge Traffic Classes Class Nomenclature Varies Among Service Providers Realtime Business Latency Business Throughput Best Effort Realtime Critical Data Transactional Data (or Streaming Video) Best Effort Routing Management SP Specific Edge Classes: SP may Additionally Have Separate Classes for Its Own Traffic (e.g., Telnet, and/or for Routing to a Managed CE) 11

12 Service Provider Edge Traffic Classes Edge Traffic Class Design Approach 1. First, design the class carrying realtime traffic (Voice, Interactive Video) to meet its stringent latency, jitter, loss requirements End-to-end delay budget for realtime Zero loss design Jitter must be accommodated by the end device 2. Other classes with less tight SLA needs, will be easier to design 12

14 Latency Budget for Realtime Class: Example Voice Class Delay budget is decided first G.114 says <=150 ms for toll quality voice (Could be > 150 ms, for less than toll quality, or for intercontinental calls where people expect larger delays) Engineering SLA Delay budget is tighter than SLA budget One Way Voice Delay Budget Example L3 Edge L3 Backbone L3 Edge 35 ms Codec Delay (G.729) 15 ms Propagation= 30 ms, Queuing=5 ms 15 ms Edge Delay Backbone Propagation Delay + Switching and Queuing Delay = 35 ms Edge Delay 30 ms* Jitter Buffer Delay End-to-End Delay Allocation (Engineering SLA) = 130 ms, in This Example 14

15 Jitter Budget for Realtime Class: Example Jitter Budget Example L3 Edge Backbone L3 Edge 35 ms Codec Delay (G.729) Delay 15ms Jitter 15ms Jitter Buffer 1. Jitter is not additive 2. Jitter across any two points can t exceed the max delay across the same two points As long as max. delay within the edge network < max. jitter buffer size, no separate jitter budgeting necessary 15

17 Controlling Realtime Class Latency Packet Delay in a Router Can Be Due to: 1. Queuing delay: Delay due to a packet sitting in a class queue, depends on Queuing algorithm (PQ or non-pq) Level of congestion, and queue size De-queue speed (available bandwidth for the queue, or shaping rate for the queue) 2. Tx-ring delay: FIFO adds equal latency to all classes 3. Serialization delay: Delay for actually transmitting the bits of the packet (depends on link speed and packet size) 4. Switching delay: All other delays a packet encounters inside a router Small (few us) for hardwire routing platforms like GSRs Higher for software switching routers (e.g., 7200) Realtime Class Latency Can Be Reduced by Using QoS Features to Control First Three 17

18 Controlling Queuing Delay A Queuing Delay in General Can Be Reduced for a Class During Congestion Via One/More of Following: 1. Using a priority queue 2. Allocating larger min bandwidth to a class relative to its average expected load Drains the packets quicker, forms smaller queue 3. Reducing max. length of a class queue Limits queue size, thereby limiting delay But a smaller burst can lead to tail drop Need to balance max. latency vs. max burst size that can be accommodated without drop 18

19 Controlling Delay Due to TX-Ring Latency Control Via Reducing Delay in TX-Ring VoIP Bus BE RP Mgt RED RED TAIL DROP TAIL DROP Classifier Per-Class Policy FRTS Dual- FIFO TX-Ring TX-Ring: FIFO queue before hardware line driver maximizes line utilization TX-Ring adds latency to ALL traffic classes If TX-Ring size is too short, interface driver may not be able to maintain line rate (depending on forwarding performance of interface driver) 19

20 Controlling Delay Due to TX-Ring Latency Control Via Reducing Delay in TX-Ring TX-Ring tuning: trade-off between maximising line utilization, and minimising queuing delay Note: some platforms (GSR linecards) automatically adjust TX-Ring size without allowing manual reconfiguration Configure TX-Ring size less than the default value, only when the default does not satisfy the required delay constraints Use the largest TX-Ring limit that still satisfies the delay requirements Generally recommended that TX-Ring limit = 2 be the minimum value used TX-Ring Tuning for 7200: Some Platforms Like GSR (Line Cards) Have Optimized TX-Ring Size (Non-Configurable) 20

21 Link Traffic Volume Budget for Realtime Class How much of link bandwidth for realtime class? Voice latency via PQ increases with link utilization due to increase in self-induced queuing delay generated by VoIP traffic contention function of: % link utilization due to PQ traffic Serialization delay of a single VoIP packet (CODEC, packetization interval) Simulation can help to estimate self induced delay for various % link utilization for a specific CODEC Other ways to estimate/determine % link utilization for limiting latency: Use Corvil bandwidth estimation (helpful if call arrival distribution is unknown, or can t be determined) Lab testing with realistic data Existing real life measurement data, if available 21

22 Link Traffic Volume Budget for Realtime Class p99.9 Delay [ms] Delay Due to VoIP Contention for 512kbps Link Poisson Call Arrival Distribution PPP/HDLC Encapsulation +/- 3% Standard Error G ms [208 bytes] G ms [78 bytes] G ms (5.3kbps) [68 bytes] G ms [68 bytes] Source: Tight SLA Project (Cisco) % VoIP Traffic Assuming latency of unloaded Priority Queue = 10 ms, and edge delay budget = 15 ms self induced delay = 5 ms 35% link utilization for G.711 (20 ms) Other factors to consider: bandwidth needed for other traffic classes, and CPU Note: Assumes Poisson Call Arrival Distribution 22

24 Edge Traffic Class Design Realtime Class Design: Delay: per edge delay budget Jitter: max jitter <= edge delay budget Ensure that jitter buffer can accommodate the jitter PHP: Expedited Forwarding (EF) Implemented using Priority Queue (PQ) of LLQ Limit the queue length queue-length = xxx packets Queue-length = link-speed * max budgeted per-hop delay Ensures that a high burst will not exceed the delay budget TX-Ring size: reduce to a size sufficient for full link utilization during worst case traffic of all classes Max link traffic volume: estimated as discussed earlier 24

25 Edge Traffic Class Design Realtime Class Design (Cont.): Traffic Conditioning: Ingress Policing Ingress traffic is policed to contractual rate (per SLA) Out-of-contract traffic is dropped (not marked down) (no call admission control by SP) Policing burst = link-speed * max budgeted per-hop delay Always-on policer: traffic permanently policed (recommended) Congestion aware policer Policing works only when there is congestion realtime traffic can potentially use whole link bandwidth May be unsuitable for enforcing contractual rate 25

26 Edge Traffic Class Design Realtime Class Design (Cont.): Congestion Avoidance Don t use WRED on Realtime Class! UDP Traffic in Realtime Class won t respond to WRED drops Stringent packet drop requirements but WRED would drop packets randomly even before the max queue threshold is reached Hence: Tail drop recommended WRED can be OK, if configured to simulate tail drop (for the class) Min threshold = max threshold = link-speed * max budgeted per-hop delay 26

27 Edge Traffic Class Design Business Latency Traffic conditioning: policed per SLA, out of contract can be marked down PHB = Assured Forwarding (AF) Should be able to use additional BW, if available (queue rather than shape) Congestion avoidance: recommended (TCP) Latency optimization needed: queue bandwidth provisioning Business Throughput Same as above, except More latency acceptable (relatively smaller bandwidth overprovisioning vs. business latency) 27

28 %& # %& $$# % "!

29 Edge Traffic Class Design Best Effort Class: No delay/loss guarantee Typical marking: IPP 0, EXP 0, DSCP CS0 Minimum bandwidth guarantee typically 25% of link bandwidth Depends on the number of other classes and their bandwidth requirement Congestion avoidance (WRED) recommended Policing optional when total traffic from customer is policed (if policed, excess is dropped) 28

30 Edge Traffic Class Design Optional SP Classes SP Class NMS A small amount of bandwidth (e.g., Kbps) is guaranteed for telnet to managed CE (in band management, without DCN) Should be able to use available bandwidth SP Class Routing PE CE routing traffic (managed CE case) traffic should be protected from congestion from enterprise customer traffic Min class bandwidth guaranty = R/C, where R = number-of-routes x routing-packet-size x 8 C = desired route convergence time in seconds 29

32 QoS Transparency QoS Transparency Transferring IP packets through a SP network without modifying the IP DSCP/precedence of a packet, while having the ability to apply QoS policies in the SP network independent of the IP DSCP of the packet Benefit: When a service provider transfers enterprise traffic through its network Enterprise QoS markings are preserved SP can still apply its own QoS policy in its network 31

33 QoS Transparency Options Method 1: per-customer class translation at edge Customer marking is changed at ingress to SP s marking SP s marking is changed back to original marking at egress on a per-customer basis Can be done at the PE or at (managed) CE Doing it at PE increases complexity per-customer marking translation on PE Method 2: encapsulate customer packet with a header that can additionally contain the SP s classification MPLS, MPLS/VPN, traffic engineering tunnel, IP-based tunnels Tunnel header carries SP s marking Original header marking is untouched Simplifies edge (PE or managed CE) configuration 32

34 QoS Transparency QoS Transparency and Tunnel Modes: RFC 2983 Differentiated Services and Tunnels introduced generic requirement of tunneling IP packets through different QoS domains to preserve the IP DSCP of the packets RFC 3270 describes three MPLS tunneling modes: Uniform mode Short pipe mode Pipe mode 33

35 MPLS Uniform Mode DiffServ Tunneling Uniform Mode Operation (without PHP) Shaded Area Represents Customer/Provider DiffServ Domain Assume a Policer Remarks Out-of-Contract Traffic s Top-Most Label to MPLS EXP 0 Here CE Router PE Router MPLS VPN PE Router CE Router P Routers MPLS EXP 0 IPP3/DSCP AF31 MPLS EXP 3 MPLS EXP 0 MPLS EXP3 IPP0/DSCP 0 Packet Initially Marked to IPP3/ DSCP AF31 MPLS EXP 3 IPP3/DSCP AF31 By Default IPP Values Will Be Copied to MPLS EXP Labels MPLS EXP 3 IPP3/DSCP AF31 Top-Most Label Is Marked down by a Policer IPP3/DSCP AF31 Topmost Label is NOT popped (no PHP) MPLS EXP Value Is Copied to IP ToS Byte (and labels popped) Direction of Packet Flow 34

36 MPLS Uniform Mode DiffServ Tunneling Uniform Mode Operation (with PHP) Shaded Area Represents Customer/Provider DiffServ Domain Assume a Policer Remarks Out-of-Contract Traffic s Top-Most Label to MPLS EXP 0 Here CE Router PE Router MPLS VPN PE Router CE Router P Routers IPP3/DSCP AF31 MPLS EXP 3 MPLS EXP 0 MPLS EXP 0 IPP0/DSCP 0 Packet Initially Marked to IPP3/ DSCP AF31 MPLS EXP 3 IPP3/DSCP AF31 By Default IPP Values Will Be Copied to MPLS EXP Labels MPLS EXP 3 IPP3/DSCP AF31 Top-Most Label Is Marked down by a Policer IPP3/DSCP AF31 Top-Most Label Is Popped, and EXP Value Is Copied to Underlying Label MPLS EXP Value Is Copied to IP ToS Byte Direction of Packet Flow 35

37 MPLS Uniform Mode DiffServ Tunneling Uniform Mode Policies The MPLS EXP Value from the Topmost Label Is Temporarily Copied to a QoS Group Value on Ingress from the MPLS VPN Cloud and Then the Label Is Popped MPLS VPN CE Router CE Router P Routers (Ingress) PE Router (Egress) PE Router IP Precedence Value is Copied to EXP of imposed Labels (default) The QoS Group Value Is Copied to the IP ToS Byte on Egress, Setting the IP Precedence to Match the Topmost Label s EXP Value Direction of Packet Flow 36

38 MPLS Uniform Mode DiffServ Tunneling Remarking Considerations In Service Provider Environment, Enterprise Customers May Need to Remark packets received from the Service Provider to Restore Diffserv Markings That May Have Been Changed in Transit Through the Cloud Model may suit enterprises running their own MPLS Core Ingress Marking from LAN MPLS VPN CE Router Ingress Remarking from MPLS VPN 37

39 MPLS Short Pipe Mode DiffServ Tunneling Short Pipe Mode Operation (without PHP) Assume a Policer Remarks Out-of-Contract Traffic s Top-Most Label to MPLS EXP 0 Here CE Router Shaded Area Represents Provider DiffServ Domain PE Router MPLS VPN P Routers Unshaded Areas Represent Customer DiffServ Domain PE Edge (to CE) Policies Are Based on CUSTOMER MARKINGS PE Router CE Router MPLS EXP 0 IPP3/DSCP AF31 MPLS EXP 4 MPLS EXP 0 MPLS EXP 4 IPP3/DSCP AF31 Packet Initially MPLS EXP 4 Marked to IPP3/ IPP3/DSCP AF31 DSCP AF31 MPLS EXP Values Are Set Independently from IPP/DSCP Values MPLS EXP 4 IPP3/DSCP AF31 Topmost Label Is Marked down by a Policer IPP3/DSCP AF31 Top-Most Label Is not popped (no PHP) Original Customer- Marked IP ToS Values Are Preserved Direction of Packet Flow 38

40 MPLS Short Pipe Mode DiffServ Tunneling Short Pipe Mode Operation (with PHP) Assume a Policer Remarks Out-of-Contract Traffic s Top-Most Label to MPLS EXP 0 Here CE Router Shaded Area Represents Provider DiffServ Domain PE Router MPLS VPN P Routers Unshaded Areas Represent Customer DiffServ Domain PE Edge (to CE) Policies Are Based on CUSTOMER MARKINGS PE Router CE Router IPP3/DSCP AF31 MPLS EXP 4 MPLS EXP 0 MPLS EXP 4 IPP3/DSCP AF31 Packet Initially MPLS EXP 4 Marked to IPP3/ IPP3/DSCP AF31 DSCP AF31 MPLS EXP Values Are Set Independently from IPP/DSCP Values Direction of Packet Flow MPLS EXP 4 IPP3/DSCP AF31 Topmost Label Is Marked down by a Policer IPP3/DSCP AF31 Top-Most Label is popped (PHP), but egress policy is based on EXP 0 of topmost label Original Customer- Marked IP ToS Values Are Preserved 39

41 MPLS Short Pipe Mode DiffServ Tunneling Short Pipe Mode Policies MPLS VPN P Routers CE Router (Ingress) PE Router (Egress) PE Router CE Router MPLS EXP Values Are Explicitly Set by Service Provider Egress Policies Are Based on CUSTOMER S DiffServ Markings Direction of Packet Flow 40

42 Short PIPE Mode Config Ingress PE PE Classification Example for Traffic from CE Enterprise Class Routing Voice Interactive Video Streaming Video Critical Data Call Signaling Transactional Network Mgmt Bulk Data Scavenger Best Effort DSCP CS6 EF AF41 CS5 CS4AF21 DSCP 25 AF31 AF31/CS3 CS5 AF21 CS3 CS2 AF11 CS1 0 0 X SP Edge Classes REALTIME EF 25% CS5 CS6 Bus-Lat AF31 30% In-contract = 3 CS3 Out = 1 AF21 Bus-Thru CS2 25% BEST EFFORT 20% class-map match-any rt match ip precedence 5 class-map match-any bus-lat match ip precedence 3 6 class-map match-any bus-thru match ip precedence 2 class-map match-any rt match ip dscp ef cs5 class-map match-any bus-latency match ip dscp af31cs3 cs6 class-map match-any bus-thruput match ip dscp af21 cs2 41

43 Short PIPE Mode Config Ingress PE: From CE policy-map ing-pe-in class rt CE Ing-PE P P Egr-PE CE police cir percent 25 bc 10 ms conform-action set-mpls-exp-imposition-transmit 5 exceed-action drop class bus-latency police cir percent 30 bc 20 ms conform-action set-mpls-exp-imposition-transmit 3 exceed-action set-mpls-exp-imposition-transmit 1 class bus-throughput police cir percent 25 bc yy ms conform-action set-mpls-exp-imposition-transmit 2 exceed-action drop SP Edge Classes Note: Depending on SLA, Policing of Every Class May Not Be Necessary Ef CS5 REALTIME 25% CS6 Af31 Bus-Lat 30% Out Of Contract = CS1 CS3 Af21 Bus-Thru Bus-Thru 25% CS2 BEST EFFORT 20% 42

44 Short PIPE Mode Config Ingress PE: To Core class-map match-any realtimece match mpls experimental 5 match ip dscp ef cs5 class-map match-any business match mpls experimental match ip dscp cs6 af31 af21 cs3 cs2 cs1! policy-map ing-pe-out class realtime priority police cir percent 50 bc 500 us conform-action transmit exceed-action drop class business bandwidth remaining percent 90 random-detect random-detect precedence packets 1012 packets 1 random-detect precedence packets 9692 packets 1 random-detect precedence packets 9692 packets 1 random-detect precedence packets 9692 packets 1 class class-default bandwidth remaining percent 10 random-detect random-detect precedence packets 9692 packets 1 Ing-PE P P Egr-PE CE OC-48 Core: Optional Policer to Avoid PQ Starving Other Traffic Classes; Not Needed if RT Class Is Policed Appropriately At Ingress Policing Rate = 50% Line Rate (Worst Case) Burst = 500 us = Max Per-Hop RT Latency Budget Ef CS5 SP Core Classes REALTIME 25% CS6 Af31 Critical Out Of Contract = CS1 CS3 90% Remaining Af21 CS2 BEST EFFORT (10% Of Remaining) 43

45 Short PIPE Mode Config Egress PE: From Core CE Ing-PE P P Egr-PE No QoS policy necessary here to classify packets for egress queuing/wred Although the PE will get an MPLS label, packet classification for egress policy is NOT done on the basis of the MPLS label, rather on the DSCP Note: on GSR, tofab queuing would need ingress classification however (tofab queuing described in backbone QoS section) Note: For suppressing PHP, Egress PE needs to advertise Explicit NULL via mpls ldp explicit-null 44

46 Short PIPE Mode Config Egress PE: To CE class-map match-any rt match ip dscp ef cs5 class-map match-any bus-lat match ip dscp cs6 af31 cs3 cs1 class-map match-any bus-thput match ip dscp af21 cs2 policy-map egr-pe-out class rt priority CE Ing-PE police conform-action transmit exceed-action drop class bus-lat bandwidth percent 30 random-detect dscp-based randon-detect dscp cs packets 9692 packets 1 randon-detect dscp cs packets 9692 packets 1 randon-detect dscp af packets 9692 packets 1 random-detect dscp cs1 500 packets 1012 packets 1 class bus-thput bandwidth percent 25 random-detect dscp-based random-detect dscp af packets 9692 packets 1 random-detect dscp cs packets 9692 packets 1 class class-default bandwidth remaining percent 20 random-detect discard-class-based random-detect discard-class packets 9692 packets 1 P P Egr-PE CE PE Uses Customer s DSCP to Classify Traffic: Config on a per-customer basis (SP must know customers markings) 45

47 MPLS Pipe Mode DiffServ Tunneling Pipe Mode Operation (Without PHP) Shaded Area Represents Provider DiffServ Domain Assume a Policer Remarks Out-of-Contract Traffic s Top- Most Label to MPLS EXP 0 Here Unshaded Areas Represent Customer DiffServ Domain CE Router PE Router MPLS VPN P Routers PE Edge (to CE) Policies Are Based on PROVIDER-MARKINGS PE Router CE Router MPLS EXP 0 IPP3/DSCP AF31 MPLS EXP 4 MPLS EXP 0 MPLS EXP 4 IPP3/DSCP AF31 Packet Initially Marked to IPP3/ DSCP AF31 MPLS EXP 4 IPP3/DSCP AF31 MPLS EXP Values Are Set Independently from IPP/DSCP Values MPLS EXP 4 IPP3/DSCP AF31 Top-Most Label Is Marked down by a Policer IPP3/DSCP AF31 No Penultimate Hop Popping (PHP) Original Customer- Marked IP ToS Values Are Preserved Direction of Packet Flow 46

48 MPLS Pipe Mode DiffServ Tunneling Mode Translating MPLS EXPs to DSCP PHBs MPLS VPN PE Edge (to CE) Policies Are Based on PROVIDER-MARKINGS PE Router Pipe Mode PE-to-CE queuing policies are based off the SP s marking policies SP marking has been done via MPLS EXP but all MPLS labels are popped prior to PE-to-CE queuing MPLS EXP values must be mapped to temporary placeholder values: QoS groups for scheduling Discard Class for WRED 47

49 MPLS Pipe Mode DiffServ Tunneling Pipe Mode Policies MPLS VPN The MPLS EXP Value from the Final Label Is Temporarily Copied to a QoS Group (and Discard Class) on Ingress from the MPLS VPN Cloud and Then the Label Is Popped P Routers CE Router (Ingress) PE Router (Egress) PE Router CE Router MPLS EXP Values Are Explicitly Set by Service Provider Egress Policies Are Based on SERVICE PROVIDER s MPLS EXP Markings Direction of Packet Flow 48

50 PIPE Mode Config Ingress PE PE Classification Example for Traffic from CE Enterprise Class Routing Voice Interactive Video Streaming Video Critical Data Call Signaling Transactional Network Mgmt Bulk Data Scavenger Best Effort DSCP CS6 EF AF41 CS5 CS4AF21 DSCP 25 AF31 AF31/CS3 CS5 AF21 CS3 CS2 AF11 CS1 0 0 X SP Edge Classes REALTIME EF 25% CS5 CS6 Bus-Lat AF31 30% In Contract = 3 CS3 Out = 1 AF21 Bus-thru 25% CS2 BEST EFFORT 20% class-map match-any rt match ip precedence 5 class-map match-any bus-latency match ip precedence 3 6 class-map match-any bus-thruput match ip precedence 2 class-map match-any rt match ip dscp ef cs5 class-map match-any bus-latency match ip dscp af31cs3 cs6 class-map match-any bus-thruput match ip dscp af21 cs2 49

51 PIPE Mode Config Ingress PE From CE policy-map ing-pe-in class rt CE Ing-PE P P Egr-PE CE police cir percent 25 bc 10 ms conform-action set-mpls-exp-imposition-transmit 5 exceed-action drop class bus-latency police cir percent 30 bc 20 ms conform-action set-mpls-exp-imposition-transmit 3 exceed-action set-mpls-exp-imposition-transmit 1 class bus-throughput police cir percent 25 bc yy ms conform-action set-mpls-exp-imposition-transmit 2 exceed-action drop SP Edge Classes EF CS5 CS6 AF31 CS3 AF21 CS2 REALTIME 25% Bus-Lat 30% Out of Contract = EXP 1 Bus-Thruput 25% BEST EFFORT 20% 50

52 PIPE Mode Config Ingress PE To Core class-map match-any realtimece match mpls experimental 5 match ip dscp ef cs5 class-map match-any business match mpls experimental match ip dscp cs6 af31 af21 cs3 af41 cs2 cs1! policy-map ing-pe-out class realtime priority police cir percent 50 bc 500 us conform-action transmit exceed-action drop class bus-lat bandwidth remaining percent 90 random-detect random-detect precedence packets 1012 packets 1 random-detect precedence packets 9692 packets 1 random-detect precedence packets 9692 packets 1 random-detect precedence packets 9692 packets 1 class class-default bandwidth remaining percent 10 random-detect random-detect precedence packets 9692 packets 1 Ing-PE P P Egr-PE CE OC-48 Core: Optional Policer to Avoid PQ Starving Other Traffic Classes; Not Needed if RT Class Is Policed Appropriately at Ingress Policing Rate = 50% Line Rate (Worst Case) Burst = 500 us = Max Per-hop RT Latency Budget SP Edge Classes REALTIME 25% Business 90% Remaining Out of Contract = EXP 1 BEST EFFORT (10% of Remaining) 51

53 PIPE Mode Config Egress PE From Core CE Ing-PE P P Egr-PE class-map match-any realtime match mpls experimental 5 match ip dscp ef cs5 class-map match-any bus-latency match mpls experimental 3 6 match ip dscp cs6 af31 cs3 class-map match-any bus-latency-out match mpls experimental 1 match ip dscp cs1 class-map match-any bus-thruput match mpls experimental 2 match ip dscp af21 cs2 policy-map egr-pe-in class realtime set qos-group 5 class bus-latency set qos-group 3 set discard-class 3 class bus-latency-out set qos-group 1 set discard-class 1 class class-default set qos-group 0 set discard-class 0 52

54 PIPE Mode Config Egress PE To CE class-map match-any rt CE Ing-PE P P Egr-PE match qos-group 5 class-map match-any bus-lat match qos-group 3 match qos-group 1 class-map match-any bus-thput match qos-group 2 policy-map egr-pe-out class rt priority police conform-action transmit exceed-action drop class bus-lat bandwidth percent 30 random-detect discard-class-based random-detect discard-class packets 9692 packets 1 random-detect discard-class packets 1012 packets 1 class bus-thput bandwidth percent 25 random-detect discard-class-based random-detect discard-class packets 9692 packets 1 class class-default bandwidth percent 20 random-detect discard-class-based random-detect discard-class packets 9692 packets 1 CE 53

55 Long PIPE Mode Cisco Specific (Not in RFC 3270) Simplifies PE Configuration With IP Explicit NULL: CE imposes a explicit NULL label on pkts towards PE (although CE doesn t run MPLS) The label caries the EXP per SP s edge-classification, but the underlying IP DSCP is unchanged Suitable for managed CE case Benefits No need to translate disparate DSCPs of multiple customers to the SP edge classes on the PE Less configuration, and better efficiency 54

56 Long Pipe Mode CE Router Shaded Area Represents Provider DiffServ Domain Ingress PE Router Classifies Traffic Based on Incoming EXP and Imposes VPN Label Assume a Policer Remarks Out-of-Contract Traffic s Top-Most Label to MPLS EXP 0 Here MPLS VPN Core Unshaded Areas Represent Customer DiffServ Domain PE Edge (to CE) Policies Are Based on SP Markings PE Router P P PE Router CE Router MPLS EXP 0 MPLS EXP 0 IPP3/DSCP AF31 Packet Initially Marked to IPP3/ DSCP AF31 MPLS EXP 4 IPP3/DSCP AF31 MPLS EXP Values are Set Independently from IPP/DSCP Values (and per SP Policy) on the Managed CE via an Explicit Null Imposition MPLS EXP 4 IPP3/DSCP AF31 Top-Most Label Is Marked Down by a Policer MPLS EXP 4 IPP3/DSCP AF31 No PHP IPP3/DSCP AF31 PE Classifies per Incoming EXP, Then Pops Label; Original Customer DSCP Preserved Direction of Packet Flow 55

57 Long Pipe Mode A Null Label Is Pushed onto the Packet with the Provider s MPLS EXP Markings MPLS VPN The MPLS EXP Value from the Final Label Is Temporarily Copied to a QoS Group and/or Discard Class on Ingress from the MPLS VPN Cloud, and then the Label Is Popped P Routers CE Router (Ingress) PE Router (Egress) PE Router CE Router Ingress PE Classifies Packet per Incoming EXP Same as Pipe Mode Egress Policies are based on Service Provider s MPLS EXP Markings Direction of Packet Flow 56

58 Long PIPE Mode: Ingress CE Configuration class-map match-any rt match ip dscp ef cs5 af31af41 class-map match-any bus-lat match ip dscp cs6 25 af21 class-map match-any bus-thruput match ip dscp cs4 cs2 CE Ing-PE P P Egr-PE class-map match-any rt-out match mpls exp 5 class-map match-any bus-lat-out match mpls exp 3 class-map match-any bus-thru-out match mpls exp 2 CE policy-map i-ce-in class rt police conform-action set-mplsexperimental-imposition-transmit 5 exceed-action drop class bus-lat set-mpls-experimental-imposition-transmit 3 class bus-thput set-mpls-experimental-imposition-transmit 2 class class-default set-mpls-experimental-imposition-transmit 0 Interface serial 1/0 service-policy input i-ce-in Note: policing on other classes possible policy-map i-ce-out class rt-out priority percent 30 class bus-lat-out bandwidth percent 25 random detect class bus-thru-out bandwidth percent 20 random-detect class class-default bandwidth percent 25 random-detect Interface serial 2/0 mpls ip encapsulate explicit-null service policy out i-ce-out 57

59 Long PIPE Mode: Ingress PE (From CE) CE class-map match-any realtime match mpls exp topmost 5 class-map match-any bus-lat match mpls topmost experimental 3 Class-map match-any bus-lat-out match mpls topmost experimental 1 Class-map match-any bus-thru match mpls topmost experimental 2 Ing-PE P P Egr-PE CE OC-48 Core: Policer Not Necessary Since Realtime Class Is Policed or Shaped at Managed CE! policy-map ing-pe-in class realtime set mpls-experimental-imposition-transmit 5 class business set mpls-experimental-imposition-transmit 3 class business-out set mpls-experimental-imposition-transmit 1 class class-default set mpls-experimental-imposition-transmit 0 SP Core Classes REALTIME 25% Business Class In-Contract EXP = 3 Out-Contract EXP = 1 90% Remaining BEST EFFORT (10% of Remaining) 58

60 Long PIPE Mode: Config Imposes EXPLICIT NULL Label Classifies Based on Explicit NULL Label s EXP CE Ingress PE P P Egress PE QoS Configuration Same as PIPE Mode Long pipe config differs from PIPE mode only at ingress CE and at CE facing interface of ingress PE Rest of the configs match PIPE mode 59

62 Edge QoS Variation Variation: Low Speed Interface Frame Serialization Delay 1 Byte 64 Bytes 128 Bytes 256 Bytes 512 Bytes 1024 Bytes 1500 Bytes 56kbps 143us 9ms 18ms 36ms 72ms 144ms 214ms Link Speed 64kbps 128kbps 256kbps 125us 62.5us 31us 8ms 4ms 2ms 16ms 8ms 4ms 32ms 16ms 8ms 64ms 32ms 16ms 128ms 64ms 32ms 187ms 93ms 46ms 512kbps 15.5us 1ms 2ms 4ms 8ms 16ms 23ms 768kbps 10us 640us 1.28ms 2.56ms 5.12ms 10.24ms 15ms 1536kbs 5us 320us 640us 1.28ms 2.56ms 5.12ms 7.5ms Voice Packet May Suffer Increase in Serialization Delay on Low Speed Interfaces Due to Other Large Packets 61

63 Edge QoS Variation Variation: Low Speed Interface Voice support on low speed link support needs Link Fragmentation and Interleaving (LFI) to avoid excessive serialization delay MLPPP on ATM, Ethernet FR.12 Frame Relay Recommendation: LFI absolutely needed for link speeds < 768 Kbps May be necessary for higher speeds depending on edge router max delay budget crtp optional (depending on CPU load) 62

64 Edge QoS Variation CE Config for LFI with ATM MLPPP MLPPP CE PVC 10/105 ATM PVC 10/105 PE policy-map low-speed class voip priority 32 set ip dscp ef class bus-latency bandwidth remaining percent 80 set ip dscp 32 class class-default bandwidth remaining percent 20 set ip dscp 0! interface ATM2/0 no ip address no atm ilmi-keepalive! interface ATM2/0.1 point-to-point bandwidth 512 pvc 10/105 vbr-nrt tx-ring-limit 3 encapsulation aal5snap protocol ppp Virtual-Template1 interface Virtual-Template1 bandwidth 512 ip unnumbered Loopback0 service-policy output low-speed no peer default ip address ppp multilink ppp multilink fragment-delay 3 ppp multilink interleave 63

65 Edge QoS Variation PE Config for LFI with ATM MLPPP MLPPP CE PVC 10/105 ATM PVC 10/105 PE policy-map low-speed class voip priority 32 class bus-latency bandwidth remaining percent 80 class class-default bandwidth remaining percent 20! interface ATM2/0 no ip address no atm ilmi-keepalive! interface ATM2/0.1 point-to-point bandwidth 1024 pvc 10/105 vbr-nrt tx-ring-limit 3 encapsulation aal5snap protocol ppp Virtual-Template1! interface Virtual-Template1 bandwidth 1024 ip vrf forwarding V1.1.com ip unnumbered Loopback1 service-policy output low-speed no peer default ip address ppp multilink ppp multilink fragment-delay 2 ppp multilink interleave 64

66 Edge QoS Variation Latency Example with LFI PVC 10/105 MLPPP ATM PVC 10/105 CE 2600 PE (7200) 7200 PE, 2600 CE: PLP/LFI Imix Load LLQ Max Latency Latency (ms) SCR: 640 SCR: 768 SCR: 1024 SCR: 1536 SCR: 2048 SCR: 3072 SCR: Frag (Bytes) IMIX Load LLQ Average Latency 65

67 Edge QoS Variation Latency Example with LFI MLPPP Latency (ms) PVC 10/105 CE 7200 PE (7200) 7200 PE, 7200 CE: PLP/LFI Imix Load LLQ Average Latency Frag (Bytes) IMIX Load LLQ Average Latency ATM PVC 10/105 SCR: 640 SCR: 768 SCR: 1024 SCR: 1536 SCR: 2048 SCR: 3072 SCR:

68 Edge QoS Variation SUB Rate Via Hierarchical Policy Maps policy-map ds3_shaping class class-default shape average service-policy ds3_output_shaping policy-map ds3_output_shaping class voip police conform-action transmit exceed-action drop priority class bus-lat queue-limit 60 bandwidth 6092 bandwidth remaining percent 1 police conform transmit exceed set-dscp-transmit 24 Example with Virtual Rate = 80% of DS3 class bus-th bandwidth remaining percent 80 random-detect random-detect dscp-based random-detect dscp class class-default random-detect random-detect prec bandwidth remaining percent 19 More Variations Based on Customer Connectivity Models in Appendix 67

69 Q and A 68

70 69

WAN Edge MPLSoL2 Service

WAN Edge MPLSoL2 Service 4 CHAPTER While Layer 3 VPN services are becoming increasing popular as a primary connection for the WAN, there are a much larger percentage of customers still using Layer 2 services such Frame-Relay (FR).