ROUTERS IP TO MPLS TO CESR

WHAT IS ROUTING FORWARDING? R3 A R1 R4 D B E C R2 Destination D Next Hop R3 R5 F E R3 3 F R5

20 bytes WHAT IS ROUTING? R3 A R1 R4 D 1 Ver 4 HLen T.Service 16 32 Total Packet Length B Fragment ID Flags Fragment Offset E TTL Protocol Header Checksum C Source Address R2 Destination Destination Address Next Hop R5 F Options D (if any) R3 E Data R3 4 F R5

WHAT IS ROUTING? R3 A R1 R4 D B E C R2 R5 F 5

POINTS OF PRESENCE (POPS) POP2 POP3 A POP1 POP4 D B POP5 E C POP6 POP7 POP8 F 6

WHERE HIGH PERFORMANCE ROUTERS ARE USED (2.5 Gb/s) R1 R2 R5 R6 (2.5 Gb/s) R3 R4 R7 R8 R9 R10 R14 R11 R12 7 (2.5 Gb/s) R13 R15 R16 (2.5 Gb/s)

WHAT A ROUTER LOOKS LIKE Cisco GSR 12416 19 Juniper M160 19 Capacity: 160Gb/s Power: 4.2kW Capacity: 80Gb/s Power: 2.6kW 6ft 3ft 8 2ft 2.5ft

GENERIC ROUTER ARCHITECTURE Header Processing Data Hdr Data Hdr Lookup IP Address Update Header Queue Packet IP Address Next Hop 1M prefixes Off-chip DRAM Address Table Buffer Memory 1M packets Off-chip DRAM 9

GENERIC ROUTER ARCHITECTURE Data Hdr Header Processing Lookup IP Address Update Header Buffer Manager Data Hdr Address Table Buffer Memory Data Hdr Header Processing Lookup IP Address Update Header Buffer Manager Data Hdr Address Table Buffer Memory Data Hdr Data Hdr Header Processing Lookup IP Address Update Header Buffer Manager 10 Address Table Buffer Memory

WHY DO WE NEED FASTER ROUTERS? 1. To prevent routers becoming the bottleneck in the Internet. 2. To increase POP capacity, and to reduce cost, size and power. 11

Fiber Capacity (Gbit/s) WHY WE NEED FASTER ROUTERS 1: TO PREVENT ROUTERS FROM BEING THE BOTTLENECK Packet processing Power Link Speed 10000 10000 Spec95Int CPU results 1000 100 10 1 2x / 18 months 1985 1990 1995 2000 2x / 7 months 1985 1990 1995 2000 1000 100 10 1 0,1 TDM DWDM 0,1 12 Source: SPEC95Int & David Miller, Stanford.

WHY WE NEED FASTER ROUTERS 2: TO REDUCE COST, POWER & COMPLEXITY OF POPS POP with large routers POP with smaller routers 13 Ports: Price >$100k, Power > 400W. It is common for 50-60% of ports to be for interconnection.

WHY ARE FAST ROUTERS DIFFICULT TO MAKE? 1. It s hard to keep up with Moore s Law: The bottleneck is memory speed. Memory speed is not keeping up with Moore s Law. 14

0.001 WHY ARE FAST ROUTERS DIFFICULT TO MAKE? SPEED OF COMMERCIAL DRAM 1. It s hard to keep up with Moore s Law: The bottleneck is memory speed. Memory 100 speed is not keeping up with Moore s Law. Access Time (ns) 1000 10 1 0.1 1980 1983 1986 1989 1992 1995 1998 2001 Moore s Law 2x / 18 months 1.1x / 18 months 15 0.01

WHY ARE FAST ROUTERS DIFFICULT TO MAKE? 1. It s hard to keep up with Moore s Law: The bottleneck is memory speed. Memory speed is not keeping up with Moore s Law. 2. Moore s Law is too slow: Routers need to improve faster than Moore s Law. 16

ROUTER PERFORMANCE EXCEEDS MOORE S LAW Growth in capacity of commercial routers: Capacity 1992 ~ 2Gb/s Capacity 1995 ~ 10Gb/s Capacity 1998 ~ 40Gb/s Capacity 2001 ~ 160Gb/s Capacity 2003 ~ 640Gb/s Average growth rate: 2.2x / 18 months. 17

First Generation Routers Shared Backplane CPU Route Table Buffer Memory Line Interface MAC Line Interface MAC Line Interface MAC Typically <0.5Gb/s aggregate capacity 19

Second Generation Routers CPU Route Table Buffer Memory Line Card Line Card Line Card Buffer Memory Fwding Cache MAC Buffer Memory Fwding Cache MAC Buffer Memory Fwding Cache MAC Typically <5Gb/s aggregate capacity 20

Third Generation Routers Switched Backplane Line Card Local Buffer Memory Fwding Table MAC CPU Card Routing Table Line Card Local Buffer Memory Fwding Table MAC Typically <50Gb/s aggregate capacity 21

Fourth Generation Routers/Switches Optics inside a router for the first time Optical links 100s of metres Switch Core Linecards 22 0.3-10Tb/s routers in development

GENERIC ROUTER ARCHITECTURE Header Processing Lookup IP Address Update Header Buffer Manager Address Table Buffer Memory Header Processing Lookup IP Address Update Header Buffer Manager Address Table Buffer Memory Header Processing Lookup IP Address Update Header Buffer Manager 24 Address Table Buffer Memory

IP ADDRESS LOOKUP Why it s thought to be hard: 1. It s not an exact match: it s a longest prefix match. 2. The table is large: about 120,000 entries today, and growing. 3. The lookup must be fast: about 30ns for a 10Gb/s line. 25

IP LOOKUPS FIND LONGEST PREFIXES 65.0.0.0/8 128.9.176.0/24 128.9.16.0/21 128.9.172.0/21 142.12.0.0/19 128.9.0.0/16 0 128.9.16.14 2 32-1 Routing lookup: Find the longest matching prefix (aka the most specific route) among all prefixes that match the destination address. 26

ADDRESS TABLES ARE LARGE 140000 Number of prefixes 120000 100000 80000 60000 40000 20000 0 Source: Geoff Huston, Oct 2001 Aug-87 May-90 Jan-93 Oct-95 Jul-98 Apr-01 Jan-04 28

LOOKUPS MUST BE FAST Year 1997 1999 2001 2003 Line 622Mb/s 2.5Gb/s 10Gb/s 40Gb/s 40B packets (Mpkt/s) 1.94 7.81 31.25 125 30

GENERIC ROUTER ARCHITECTURE Header Processing Lookup IP Address Update Header Queue Buffer Manager Packet Address Table Buffer Memory Header Processing Lookup IP Address Update Header Queue Buffer Manager Packet Address Table Buffer Memory Header Processing Lookup IP Address Update Header Queue Buffer Manager Packet 32 Address Table Buffer Memory

FAST PACKET BUFFERS Example: 40Gb/s packet buffer Size = RTT*BW = 10Gb; 40 byte packets Write Rate, R 1 packet every 8 ns Buffer Manager Read Rate, R 1 packet every 8 ns Buffer Memory 33 Use SRAM? Use DRAM? + fast enough random access time, + high density means we can store data, but but - too low density to store 10Gb of data. - too slow (50ns random access time).

PACKET CACHES Small ingress SRAM cache of FIFO tails Small ingress SRAM cache of FIFO heads Arriving Packets 60 59 58 57 56 55 97 96 1 2 Buffer Manager 5 4 4 3 3 2 2 1 1 2 Departing Packets 91 90 89 88 87 Q SRAM 6 5 4 3 2 1 Q b>>1 packets at a time DRAM Buffer Memory 54 53 52 51 50 10 9 8 7 6 5 DRAM 89 88 87 86 Buffer Memory 15 14 13 12 95 94 93 92 91 90 11 10 9 8 7 6 1 2 1 86 85 84 83 82 11 10 9 8 7 Q 34

GENERIC ROUTER ARCHITECTURE Data Hdr Header Processing Lookup IP Address Update Header 1 1 Queue Packet Address Table Buffer Memory Data Hdr Header Processing Lookup IP Address Update Header 2 2 N times line rate Queue Packet Address Table Buffer Memory N times line rate Data Hdr Header Processing Lookup IP Address Update Header N N Queue Packet 36 Address Table Buffer Memory

GENERIC ROUTER ARCHITECTURE Data Hdr Header Processing Lookup IP Address Update Header Queue Packet 1 1 Address Table Buffer Memory Data Hdr Data Hdr Header Processing Lookup IP Address Update Header Queue Packet 2 2 Data Hdr Address Table Buffer Memory Data Hdr Header Processing Lookup IP Address Update Header Queue Packet N Scheduler N 37 Address Table Buffer Memory Data Hdr

OUTPUT BUFFERED SWITCHES The best that any queueing system can achieve. Delay 38 0% 20% 40% 60% 80% 100% Load

INPUT BUFFERED SWITCHES HEAD OF LINE BLOCKING The best that any queueing system can achieve. Delay 39 0% 20% 40% 60% 80% 100% Load 2 2 58%

40 HEAD OF LINE BLOCKING

41 VIRTUAL OUTPUT QUEUES

A ROUTER WITH VIRTUAL OUTPUT QUEUES The best that any queueing system can achieve. Delay 42 0% 20% 40% 60% 80% 100% Load

MAXIMUM WEIGHT MATCHING A 1 (n) A 11 (n) A 1N (n) L 11 (n) S*(n) 1 1 D 1 (n) A N (n) A N1 (n) D N (n) A NN (n) N N L NN (n) L 11 (n) * T S n L n S n S( n) ( ) arg max( ( ) ( )) Maximum Weight Match 43 L N1 (n) Request Graph Bipartite Match

OUTLINE Background What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques The Future The evolution of router architecture. IP address lookup. Packet buffering. Switching. More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? 44

EXTERNAL PARALLELISM: MULTIPLE PARALLEL ROUTERS What we d like: R IP Router capacity 100s of Tb/s R NxN R R The building blocks we d like to use: R R 45 R R

R R R MULTIPLE PARALLEL R ROUTERS LOAD BALANCING R R R/k 1 2 k R/k R R R 46

INTELLIGENT PACKET LOAD-BALANCING PARALLEL PACKET SWITCHING R/k Router R/k rate, R 1 1 1 rate, R 2 rate, R N N rate, R k 47 Bufferless

They are already there. Connecting linecards to switches. Optical processing doesn t belong on the linecard. You can t buffer light. Minimal processing capability. Optical switching can reduce power. DO OPTICS BELONG IN ROUTERS? 49

Optics in routers Optical links Switch Core Linecards 50

COMPLEX LINECARDS Typical IP Router Linecard Lookup Tables Buffer & State Memory Optics Physical Framing Layer & Maintenance Packet Processing Buffer Mgmt & Scheduling Buffer Mgmt & Scheduling Switch Fabric Buffer & State Memory Arbitration 51 10Gb/s linecard: Number of gates: 30M Amount of memory: 2Gbits Cost: >$20k Power: 300W

REPLACING THE SWITCH FABRIC WITH OPTICS Typical IP Router Linecard Typical IP Router Linecard Lookup Tables Buffer & State Memory Buffer & State Memory Lookup Tables Optics Framing Physical & Layer Maintenance Packet Processing Buffer Mgmt & Scheduling Buffer Mgmt & Scheduling Buffer & State Memory 1. MEMs. electrical Switch Fabric Buffer Mgmt & Scheduling Buffer Mgmt & Scheduling Candidate Arbitration technologies Buffer & State Memory Packet Processing Framing & Maintenance Physical Layer Optics 2. Fast tunable lasers + passive optical couplers. 3. Diffraction waveguides. 4. Electroholographic materials. Typical IP Router Linecard Lookup Tables Buffer & State Memory optical Typical IP Router Linecard Buffer & State Memory Lookup Tables 52 Optics Framing Physical & Layer Maintenance Packet Processing Buffer Mgmt & Scheduling Buffer Mgmt & Scheduling Buffer & State Memory Req/Grant Switch Fabric Arbitration Req/Grant Buffer Mgmt & Scheduling Buffer Mgmt & Scheduling Buffer & State Memory Packet Processing Framing & Maintenance Physical Layer Optics

Optics enables simple, low-power, very high capacity circuit switches. The Internet was packet switched for two reasons: Expensive links: statistical multiplexing. Resilience: soft-state routing. Neither reason holds today. EVOLUTION TO CIRCUIT SWITCHING 54

Processing Power Link Speed (Fiber) 10000 10000 FAST LINKS, 1985 SLOW 1990 1995 ROUTERS 2000 55 Spec95Int CPU results 1000 100 10 1 0.1 2x / 2 years 2x / 7 months Source: SPEC95Int; Prof. Miller, Stanford Univ. 1985 1990 1995 2000 1000 100 10 1 0,1 Fiber optics DWDM Fiber Capacity (Gbit/s)

REFERENCES General 1. J. S. Turner Design of a Broadcast packet switching network, IEEE Trans Comm, June 1988, pp. 734-743. 2. C. Partridge et al. A Fifty Gigabit per second IP Router, IEEE Trans Networking, 1998. 3. N. McKeown, M. Izzard, A. Mekkittikul, W. Ellersick, M. Horowitz, The Tiny Tera: A Packet Switch Core, IEEE Micro Magazine, Jan-Feb 1997. Fast Packet Buffers 1. Sundar Iyer, Ramana Rao, Nick McKeown Design of a fast packet buffer, IEEE HPSR 2001, Dallas. 57

REFERENCES IP Lookups 1. A. Brodnik, S. Carlsson, M. Degermark, S. Pink. Small Forwarding Tables for Fast Routing Lookups, Sigcomm 1997, pp 3-14. 2. B. Lampson, V. Srinivasan, G. Varghese. IP lookups using multiway and multicolumn search, Infocom 1998, pp 1248-56, vol. 3. 3. M. Waldvogel, G. Varghese, J. Turner, B. Plattner. Scalable high speed IP routing lookups, Sigcomm 1997, pp 25-36. 4. P. Gupta, S. Lin, N. McKeown. Routing lookups in hardware at memory access speeds, Infocom 1998, pp 1241-1248, vol. 3. 5. S. Nilsson, G. Karlsson. Fast address lookup for Internet routers, IFIP Intl Conf on Broadband Communications, Stuttgart, Germany, April 1-3, 1998. 6. V. Srinivasan, G.Varghese. Fast IP lookups using controlled prefix expansion, Sigmetrics, June 1998. 58

REFERENCES Switching N. McKeown, A. Mekkittikul, V. Anantharam, and J. Walrand. Achieving 100% Throughput in an Input-Queued Switch. IEEE Transactions on Communications, 47(8), Aug 1999. A. Mekkittikul and N. W. McKeown, "A practical algorithm to achieve 100% throughput in input-queued switches," in Proceedings of IEEE INFOCOM '98, March 1998. L. Tassiulas, Linear complexity algorithms for maximum throughput in radio networks and input queued switchs, in Proc. IEEE INFOCOM 98, San Francisco CA, April 1998. D. Shah, P. Giaccone and B. Prabhakar, An efficient randomized algorithm for input-queued switch scheduling, in Proc. Hot Interconnects 2001. J. Dai and B. Prabhakar, "The throughput of data switches with and without speedup," in Proceedings of IEEE INFOCOM '00, Tel Aviv, Israel, March 2000, pp. 556 -- 564. C.-S. Chang, D.-S. Lee, Y.-S. Jou, Load balanced Birkhoff-von Neumann switches, Proceedings of IEEE HPSR 01, May 2001, Dallas, Texas. 59

REFERENCES Future C.-S. Chang, D.-S. Lee, Y.-S. Jou, Load balanced Birkhoff-von Neumann switches, Proceedings of IEEE HPSR 01, May 2001, Dallas, Texas. Pablo Molinero-Fernndez, Nick McKeown "TCP Switching: Exposing circuits to IP" Hot Interconnects IX, Stanford University, August 2001 S. Iyer, N. McKeown, "Making parallel packet switches practical," in Proc. IEEE INFOCOM `01, April 2001, Alaska. 60

MULTI PROTOCOL LABEL SWITCHING (MPLS)

WHY INTERNET PROTOCOL IS POPULAR? Robustness Aggregation and Hierarchy

ISSUES WITH INTERNET PROTOCOL IP address lookup No QoS Best Effort

OBJECTIVES OF MPLS Speed up IP packet forwarding By cutting down on the amount of processing at every intermediate router Prioritize IP packet forwarding By providing ability to engineer traffic flow and assure differential QoS Without losing on the flexibility of IP based network

MPLS KEY IDEAS Use a fixed length label in the packet header to decide packet forwarding A path is established between two end points. At ingress a packet is classified into a Forwarding Equivalence Class. FEC to which it is assigned is encoded as a short fixed length value Label. Packet is forwarded along with label to next hop.

MPLS KEY IDEAS No further analysis of header by subsequent routers, forwarding is driven by the labels. Label is used as index for next hop and new label. At the Egress, label is removed and packet is forwarded to final destination based on the IP packet header

MPLS HEADER Label: 20-bit label value Exp: experimental use Can indicate class of service S: bottom of stack indicator 1 for the bottom label, 0 otherwise TTL: time to live 20 3 1 8 Label Exp S TTL

Forwarding Equivalence Class Forwarding Equivalence Class (FEC): A subset of packets that are all treated the same way by an LSR. A packet is assigned to an FEC at the ingress of an MPLS domain. Subset can be based on Address prefix Host address QoS

Forwarding Equivalence Class Assume packets have the destination address and QoS requirements as 124.48.45.20 qos = 1 FEC 1 label A 143.67.25.77 qos = 1 FEC 2 label B 143.67.84.22 qos = 3 FEC 3 label C 124.48.66.90 qos = 4 FEC 4 label D 143.67.12.01 qos = 3 FEC 3 label C

LABEL DISTRIBUTION PROTOCOL (LDP) LDP is the set of procedures to inform other LSRs about binding between FEC and label. Piggyback on existing protocols(bgp, OSPF,RSVP) Separate Label Distribution Protocol

MPLS LSPS ESTABLISHING Static Configuration -Operator can provision LSPs by statically configuring label mappings at each LSRs.

MPLS LSPS ESTABLISHING- LDP LDP can have two types of neighbors: 1. Directly connected neighbor LDP uses UDP hello messages sent on port 646 to all the routers Multicast address (224.0.0.2) to discover directly connected neighbors. 2. Non-directly connected neighbor LDP has the ability to establish LDP session with a router two or more hops away. Hellos in this case are unicast to the peer router using UDP port 646. 1.1.1.1 Hello DA: 224.0.0.2 Dest. Port: 646 2.2.2.2 Hello 1.1.1.1 DA: 3.3.3.3 Dest. Port: 646 2.2.2.2 3.3.3.3

MPLS LSPS ESTABLISHING- LDP Once two LSRs discover each other an LDP session is established over which label can be advertised. LDP session runs over TCP over port 646. Session is maintained by periodic exchange of Keep Alive Messages. Hello Exchange 1.1.1.1 TCP Session Session Keep Alive Label Exchange 2.2.2.2

MPLS LSPS ESTABLISHING- RSVP TE The ingress LSR computes a path to egress LSR using the CSPF algorithm. The computed path is encoded into an Explicit Route Object (ERO) and included in RSVP TE Path messages. Each router along the path creates state for the path and forwards the message to the next router in ERO. Egress LSR validates the path message and creates a Resv message containing a Label for the LSP. The Resv message is sent back to ingress LSR on the same path followed by path message.

LABEL DISTRIBUTION METHODS Ru Rd Ru Request for Binding Rd Label-FEC Binding Unsolicited Downstream Label Distribution Rd discovers a next hop for a particular FEC Rd generates a label for the FEC and communicates the binding to Ru Ru inserts the binding into its forwarding tables Label-FEC Binding Downstream on Demand Label Distribution Ru recognizes Rd as its next-hop for an FEC A request is made to Rd for a binding between the FEC and a label If Rd recognizes the FEC and has a next hop for it, it creates a binding and replies to the Ru

LABEL DISTRIBUTION AND MANAGEMENT Label Distribution Control Mode Independent LSP control Ordered LSP control

LABEL DISTRIBUTION AND MANAGEMENT Label Retention Mode Conservative LSR maintains only valid bindings. Liberal - LSR maintains bindings other than the valid next hop, more label, quick adaptation for routing change

LABEL INFORMATION BASE (LIB) Table maintained by the LSRs Contents of the table Incoming label Outgoing label Outgoing path Address prefix Incoming label Address Prefix Outgoing Path Outgoing label

NEXT HOP LABEL FORWARDING ENTRY (NHLFE) Used for forwarding a labeled packet. Contains the following information: - Packets next hop. - Operation to be performed on label stack i.e. (One of the following) Replace the label on the label stack with a specified new label. Pop the label stack. Replace the label on the label stack with a specified new label and then push one on more specified new label onto the label stack.

Incoming Label Map (ILM)- Maps incoming label to NHLFE. FEC to NHLFE Map (FTN)-Maps FEC to a set of NHLFE s.

NHLFE, ILM AND FTN Labeled ILM Incoming Packet NHLFE Unlabeled FTN

LABEL STACK A labeled packet can contain more than one label. Labels are maintained in FIFO stack. Processing always done on the top label. MPLS support a hierarchy and notion of LSP Tunnel.

OPERATIONS ON LABEL STACK Swap Pop Push

OPERATION ON LABEL STACK-SWAP Labeled Packet - LSR examines label at the top of the label stack of the incoming packet. - Uses ILM to map to the appropriate NHLFE. - Encodes the new label stack into the packet and forwards. Unlabeled Packet - LSR analyses network layer header to determine FEC s. - Uses FTN to map to an NHLFE. - Encodes the new label stack into the packet and forwards.

OPERATION ON LABEL STACK PUSH - A new label is pushed on top of the existing label, effectively "encapsulating" the packet in another layer of MPLS. - This allows hierarchical routing of MPLS packets. POP - The label is removed from the packet, which may reveal an inner label below. - If the popped label was the last on the label stack, the packet "leaves" the MPLS tunnel. - This is usually done by the egress router and in Penultimate Hop Popping.

OPERATION ON LABEL STACK L1 IP SWAP L2 IP L1 IP PUSH L2 L1 IP L2 L1 IP POP L1 IP

Label Switched Path For each FEC, a specific path called Label Switched Path (LSP) is assigned. To set up an LSP, each LSR must Assign an incoming label to the LSP for the corresponding FEC Inform the upstream node of the assigned label Learn the label that the downstream node has assigned to the LSP Need a label distribution protocol so that an LSR can inform others of the label/fec bindings it has made. A forwarding table is constructed as the result of label distribution.

Label Switched Path Penultimate Hop Popping : Label Stack is popped at the penultimate LSR of LSP rather than LSP egress. Egress LSP does a single look up. Egress may not be a a LSR.

Label Switched Path LSP Next Hop Labeled Packet - As selected by the NHLFE entry used for forwarding that packet. FEC - As selected by the NHLFE entry indexed by a label corresponding to that FEC.

LABEL SWITCHED PATH -HIERARCHY Lbl-2 Lbl-1 IP Payload Level 2 LSP Label Stack -2 Lbl -1 IP Payload Lbl -1 IP Payload LSR LSR LSR Level 1 LSP Level 1 LSP Label Stack -1 Label Stack -1 LSR

LABEL STACK - HIERARCHY Layer 2 Header Label 3 Label 2 Label 1 IP Packet MPLS Domain 1 MPLS supports hierarchy. Each LSR processes the topmost label. MPLS Domain 2 MPLS Domain 3 If traffic crosses several networks, it can be tunneled across them Advantage reduces the LIB table of each router drastically Slide by ByTamrat Bayle, Reiji Aibara, Kouji Nishimura

LABEL SWITCHED PATH -TUNNEL Hop By Hop Tunnel Level 2 LSP Label Stack -2 Explicitely Routed Tunnel

Label Switched Path - Control Independent - Each LSR on recognizing a particular FEC makes an independent decision to bind a label to it and distribute that binding. Ordered - An LSR binds a label to a FEC only if it is the egress LSR to that FEC or it has already a binding for that FEC from its next hop for that FEC.

LSP Route Selection Method for selecting the LSP for a particular FEC. Hop-by-hop routing: Each node independently choose the next hop for a FEC. Explicit routing (ER): the sender LSR can specify an explicit route for the LSP Explicit route can be selected ahead of time or dynamically

Explicitly Routed LSP Advantages Can establish LSP s based on policy, QoS, etc. Can have pre-established LSP s that can be used in case of failures. It makes MPLS explicit routing much more efficient than the alternative of IP source routing. 95

AGGREGATION In the MPLS Domain, all the traffic in a set of FECs might follow the same route. The procedure of binding a single label to a union of FECs which is itself a FEC, and applying that label to all traffic in the union is known as Aggregation. Reduces the number of labels. Reduces the amount of label distribution control traffic.

AGGREGATION Label L1, FEC-F1 Label L2, FEC-F2 Label L3, FEC-F3 Label L, FEC-F

LABEL MERGING Label Merging is the capability of forwarding two different packets belonging to the same FEC, but arriving with different labels, with the same outgoing label. An LSR is label merging capable if it can receive two packets from different incoming interfaces, and/or with different labels, and send both packets out the same outgoing interface with the same label. Once transmitted, the information that they arrive from different interfaces and/or with different labels is lost.

MPLS PROTECTION MPLS OAM*: Fault detection and diagnosis TRAFFIC PROTECTION: Route traffic away from failed node/link. NODE PROTECTION: Enhance node availability. * Operation, Administration and Maintenance.

SOME MPLS TRANSPORT PROBLEMS Data plane fails ( Black Holes ). Connectivity Problem, Broken link. What path is being taken?

LEVERAGING MPLS OAM Difficult to detect MPLS failure: Traditional ping may not be successful Difficult to troubleshoot MPLS failure: Manual hop/hop work MPLS OAM facilitates and speeds up troubleshooting of MPLS failures.

LSP PING/TRACEROUTE Requirements: Solutions: - Detect MPLS Black holes. - Isolate MPLS faults. - Diagnose connectivity problems. - LSP ping for connectivity checks. - LSP traceroute for hop-by-hop fault localization. - LSP traceroute for path tracing.

TROUBLESHOOTING SA DA = 127/8 ECHO R1 R3 does not forward R2 echo-req to R4, replies R3 back to R1. R4 30 50 MPLS Echo Reply LSP Broken

TRAFFIC PROTECTION Detect the fault. Divert the traffic away from fault. Mechanisms: - LDP signaled LSPs - Backup LSP - Fast Reroute

LDP SIGNALED LSPS Path protection depends on IGP reconvergence. In case of link/node failure: i) Remove label mapping for the LSP from FIB. ii) Wait for new shortest path and matching label.

BACKUP LSPS Backup Path for RSVP-TE signaled LSP R6 R7 Backup path R3 R1 R2 R4 R5 Path Err

FAST REROUTE Repair failure at the node that detects failure (Point of Local Repair or PLR). One to one backup: Each LSR creates a detour LSP for each protected LSP. Facility Bypass: Single bypass LSP for all protected LSPs

LINK PROTECTION(FR) R5 Merge Point PLR R1 R2 R3 R4 Merge point is one hop away from PLR. Next Hop protection

NODE PROTECTION(FR) R6 R7 Merge Point PLR R1 R2 R3 R4 R5 Merge point is two hops away from PLR. Next-Next Hop protection

NODE PROTECTION Resilient LSRs. Software support required to take advantage of Hardware redundancy. Nonstop Routing: - Replicate all state changes to backup control card. - State synchronization required. Nonstop Forwarding: - Graceful restart of the failed node. - Protocol extensions at neighbors.

MPLS QOS Network can meet Qos requirement ensures Control Forwarding enforce Qos in network elements

MPLS QOS P P P Prioritized Queues WRED for service differentiation

MPLS QOS MODELS Soft QOS (Class of Service): - Forwarding plane technique. - No need for per-flow state (Scales well) - Unable to provide guaranteed forwarding behavior. Hard QOS : - Resources reserved using control plane. - Per-flow state required. - Provides firm guarantees.

TRANSPORT PLANE MODELS E-LSP model - Implementation of Soft QOS model. - EXP field in MPLS label. - 3 bits => up to 8 different DiffServ points. L-LSP model - Implementation of Hard QOS model. - Both Label and EXP field are considered. - Label lookup and Exp bits determine output queue and priority.

TRANSPORT NETWORKS: THE SERVICE PROVIDER PERSPECTIVE Shift from best effort to guaranteed services. Focus on revenue bearing services. Support for varied application requirements. Flexible on-demand service granularity. High QoS, Reliability. Operations, Administration, and Maintenance features. Low CAPEX and OPEX networks 115 Yesterday Today Tomorrow

Efficiency and Flexibility Service Guarantees with Reliability PACKET TECHNOLOGIES (e.g. IP/MPLS, Carrier Ethernet) CIRCUIT TECHNOLOGIES POSSIBLE SOLUTION (e.g. SONET/SDH) Packet- Optical Transport System (P-OTS) Multi-Layer Optimized transport for flexible, reliable and scalable networks 116 9/21/2015

Very fast communication framework combining switching, routing and transport in single layer TCP/UDP Omnipresent Ethernet IP Application Layer Transport Layer Network Layer Ethernet Data-Link Layer OMNIPRESENT ETHERNET (OE) Physical Layer 117 9/21/2015

CONTEMPORARY NETWORKING HIERARCHY Access, Metro and core networks 118

Formulate physical network into a logical hierarchical tree Simplify the network into a fractal binary tree Fig. 1a. Physical topology of the network. Binary Routing Packet switching based on the bit value corresponding to the node 0 indicates right-ward movement while a 1 indicates left-ward movement Advantages: A I J K H O G Simple lookup Energy efficient Source routing Cost, cost, cost!!!!! B L D M N E F A B 0 1 I L J 1 D 1 0 H K M 1 N E O G F 119

EXAMPLE OF CONVERSION TO A BINARY TREE 0000 000 Campus node (large switch) Gateway Virtual (dummy) node created 00 0 0001 001 010 01 100 1 Campus node (large switch) 10 11 111 MTU - switch 11111 1111 111111 0110 011 1000 101 1110 11110 111100 0111 1001 10010 10011 01100 Laptop computer IBM Compatible IBM Compatible Mac Classic Workstation Laptop computer Source A C N j E I J N q 100100 MTU - switch 1100 100111 110 1101 11011 110111 D 10 N i B 10110 F G 101100110 K N 1011001101001 L O Destination 101100110100101 120 H M N r

C E Source A N j I J N q D 10 N i B 10110 F G 101100110 L K 1011001101001 O N Destination 101100110100101 H M N r 121

Creating Virtual Topology Binary Routing Binary Routing Network Graph Logical Tree Logical Binary Tree Source Routing CONCEPT OF OE 122 9/21/2015

Discard common MSBs 0 0 0 1 0 1 1 1 0 Source S-ARTAG Destination S-ARTAG Complement Source S-ARTAG except MSB Lowest common ancestor 1 0 Root 1 0 1 0 1 1 0 ADDRESSING AND ROUTING 0001 IN OE 0 1 1 BASED TREES Discard MSB of Destination S-ARTAG Reverse Destination S-ARTAG Final R-ARTAG 0 1 1 0 1 0 0 1 0 Source S-ARTAG 01110 Destination S-ARTAG 123 9/21/2015

Ethernet frame IP address with data Identifier IPV4 IP address IPV6 Binary Tag Ethernet 010001001 header 100101010 Omnipresent Ethernet another avatar of MPLS and SDNs dat a MAC 1010 Port 111010 S/CTAG 0010101010 1 Ethernet data mapped onto OE packet Ethernet header used for L2 protocol.

UNICAST PACKET FLOW Output Port Decoder Input Port Logic Output Port 0 TELL Table Route Calculator QDR II+ QDR II+ QDR II+ QDR II+ Output Port 1 Memory Arbiter Rate Limiter Write State Machine Memory Mgmt. Unit Read State Machine Port 1 Output Port 2 OE Framing/ De-framing Port Control Logic and Multicast Handler Port 2 Frame Buffer Port N-1 Contention Resolution Logic (Port 2) Switch Fabric Output Port N-1 126 9/21/2015

THE CESR HARDWARE Fig 1. The Carrier Ethernet Switch Router(CESR) hardware

THE V SERIES CORE ROUTER, LONG HAUL TRANSPORT, TUNABLE WDM SUPPORT WITH CARRIER ETHERNET State-of-the-art transport solution. 1000 km reach without regen. 96 Gbps cross connect OTN as ODU2 compliant. 1000 FEC entries PseudoWire emulation. Deep buffers for packet processing 3-5 microsecond latency. Multicast, 4 level QoS support. Dense Wavelength Division Multiplexing technology for super fiber utilization. Applications: metro transport, regional transport, multi-gigabit router, National Knowledge Network transport, enterprise backbone and Carrier Ethernet. 128

STATISTICS O1000 LX240T (IPv4) O1000 LX365T (IPv6) O1010 LX365T O100 LX150T FPGA device Utilization 92% 67% 71% 71% BRAM Utilization 59% 61% 74% 74% Lines of code (VHDL) 1,28,405* 1,28,405* 68,302** 68,302** Lines of code (NMS) 16,142 16,142 16,142 16,142 Lines of code Web based NMS PCB stats 50,000 + 50,000 + 50,000 + 50,000 + R ~ 500, C ~ 1000 135 different components R ~ 500, C ~ 1000 135 different components R ~ 300, C ~ 700 107 different components R ~ 700, C ~ 2000 176 different components Total components 1500+ 1500+ 2200+ 850 *O1000 code statistics includes test code **O100 and O1010 code statistics exclude common files from O1000.

Latency COMPARISON Juniper M- 120 Cisco GSR Cisco 6700 Cisco 3550 Arista Omnipresent Ethernet Protocol depth 130

Port-to-port latency = 1.2 microsecond OTN support for long reach without 3R DWDM optics supported E-LINE, E-LAN service support for MAC,IPv4,IPv6, CTAG, STAG, port based identifiers CESRS DEVELOPED 131 9/21/2015

RailTel Western Region Network 22 nodes 23 links Working Path Protection Path EXPERIMENTAL TEST-BED DESCRIPTION 132 9/21/2015

CESR 2 Fiber Links JDSU MTS-6000 Client Node-1 CESR 1 TEST-BED PHOTOGRAPH NMS JDSU MTS-6000 Client Node-2 133 9/21/2015