SURFnet7: PBB-TE and MPLS-TP technical description and comparison

Transcription

1 SURFnet7: PBB-TE and MPLS-TP technical description and comparison Author(s): Jörg Buesink Version: 1.0 Date: December 24th, 2013 Radboudkwartier CK Utrecht The Netherlands PO Box DA Utrecht The Netherlands +31 (0) admin@surfnet.nl Deutsche Bank C of C Utrecht VAT NL B01

2 Contents 1 Introduction Keywords Ethernet evolution Introduction to Carrier Ethernet Provider Backbone Bridging (PBB) and PBB-TE (Traffic Engineering) PBB PBB and PBB-TE Frame processing PBB-TE / PBT PBB-TE OAM Multi Protocol Label Switching (MPLS) and MPLS Transport Profile (MPLS-TP) MPLS-TP management and data plane Pseudowires MPLS-TP OAM MPLS-TP Protection Switching MPLS-TP Security PBB-TE and MPLS-TP comparison Conclusion Acronyms References This project was made possible by the support of SURF, the collaborative organization for higher education institutes and research institutes aimed at breakthrough innovations in ICT. More information on SURF is available on the website 2/32

3 3/32

4 1 Introduction This paper provides a technical description of PBB-TE and MPLS-TP and compares both technologies in relation to Carrier Ethernet. We describe the evolution of Ethernet, provide an introduction to Carrier Ethernet and then dive into the details of PBB-TE and MPLS-TP and compare both technologies. 1.1 Keywords Ethernet, Carrier Ethernet, PBB, PBB-TE, MPLS-TP 4/32

5 2 Ethernet evolution Ethernet, developed by Robert Metcalfe in 1972 is the most dominant Local Area Network (LAN) technology for decades. Its initial success is mainly because of massive adoption in Enterprise networks, its low cost and ease of deployment. Ethernet has evolved over the years, initially running over coax cables at low speeds and nowadays over copper and fiber with speeds ranging up to 100Gbps and even more. Because of several enhancements Ethernet became an attractive technology in Service Provider (SP) and other Telecommunication environments. Besides several encapsulation enhancements more recent developments like the 802.1Qbb (PFC) standard made Ethernet also a reliable technology for fiber-channel transport, which resulted in uptake for Ethernet in storage environments. It is clear that Ethernet is no longer considered a LAN-only technology anymore and is here to stay for a while. Traditionally Carrier Networks deployed legacy technologies like SDH/SONET, Frame Relay and ATM in the Wide Area Networks (WAN) or Metropolitan Area Networks (MAN) to transport their services. These legacy TDM technologies where expensive to deploy and maintain, therefore inefficient and could not cope with increasing customer bandwidth demands. In the past it was difficult for Carrier Networks to adopt Ethernet in the WAN or MAN, because: Native Ethernet does not support scaling up to millions of customers MAC addresses. Native Ethernet frames do not have fields available for Class of Service (CoS) identification Redundant Ethernet connections relied on the spanning tree protocol. The spanning tree protocol does not scale very well in large networks Lack of Operations Administration and Maintenance (OAM) support. To address the above issues and meet carrier requirements, the IEEE defined several adaptations of Ethernet in amendments to the standard. These changes comprise additional headers as well as changes of the Ethernet operational principles. Figure 1 5/32

6 In the figure above several IEEE adaptations of Ethernet are shown. The 802.1q VLAN tagging standard scales up to 4096 separated broadcast domains. So in theory with 802.1q it is possible to support 4096 customers within the provider network. With 802.1q the provider assigns VLANs, which the customers must use to avoid VLAN ID overlap in the provider network, which is inefficient and complex to coordinate. The 802.1q standard supports Class of Service (CoS) through the usage of three dot1p priority bits (PCP) in the VLAN header. The provider network needs to interact with customer layer 2 loop prevention mechanisms and makes forwarding decisions based on the learned customer MAC addresses. This is a potential security risk, as the provider core switches must act on the information received from the customer space (for example excessive MAC learning, spanning tree interaction). These limitations caused the 802.1q standard not to be adopted by providers as a carrier grade transport technology. The Provider Bridging (802.1ad) standard has some major improvements over the 802.1q standard as it uses two stacked VLAN tags, the service provider tag (S-TAG or outer tag), which represents a customer or service instance in the provider network and the customer (C-TAGS/ inner tag) representing the customer VLANs. The main advantage over 802.1q is that customer VLANs can be reused and are non-overlapping as long as the S-TAG is different. Forwarding based on the S-TAG allows for a maximum of 4096 customer/ service identifiers in the provider networks. The dot1p value in the S-TAG and C-TAG makes it possible to preserve customer Class of Service (CoS) markings, hence the ability to provide CoS transparent services towards customers. Like the 802.1q standard, with Provider Bridging the customer MAC addresses are still used for forwarding purposes in provider networks. An increase in customer MAC addresses results in an increase in the amount of MAC addresses learned in the provider network. This might result in scaling issues in the provider hardware or even cause outages due to broadcast storms (flooding). So although the 802.1ad standard has major benefits over the 802.1q standard, the 802.1ad standard still could not cope with the scaling and stability requirements needed in large carrier networks. Over the years several enhancements made Ethernet evolve into a Carrier Grade transport technology suitable for carrier networks. In this document two Carrier Grade transport technologies, Multi Protocol Label Switching Transport Profile (MPLS-TP) and Provider Backbone Bridging Traffic Engineering (PBB-TE), both capable of transporting Ethernet with Connection Oriented characteristics are discussed in detail. 6/32

7 3 Introduction to Carrier Ethernet Before diving into the details of PBB-TE and MPLS-TP an introduction to Carrier Ethernet (CE) and terminology is fundamental as CE terminology is used throughout this document. Carrier Ethernet provides carrier grade transport characteristics to Ethernet by providing scalability, protection, deterministic delay and OAM support. The Metro Ethernet Forum (MEF) is a nonprofit international industry consortium, dedicated to the adoption of Carrier Ethernet Networks (CEN) and services. The MEF has defined Carrier Ethernet as a standardized, carrier-class Service and/or Network defined by five attributes (i.e. standardized service, scalability, reliability, quality of services and service management) that distinguish Carrier Ethernet from LAN based Ethernet. The MEF defined several service attributes to convert Ethernet to a carrier grade service centric model. In Carrier Ethernet Networks, Ethernet service frames are transported across Ethernet Virtual Connections (EVCs) or Operator Virtual Connections (OVCs). Figure 2 An EVC is a service container with two or more User Network Interfaces (UNIs) within a single, or spanning multiple Autonomous Systems or layer 2 domains. An OVC is a service container, which has similar service characteristics as an EVC, but with at least one External Network to Network Interface (ENNI) used for inter-provider connectivity. A UNI and ENNI describe different aspects of the interface between the subscriber and provider (UNI) or between providers (ENNI). A UNI or ENNI provides various data, control and management plane capabilities required by Carrier Ethernet providers and demarcate different network domains. A UNI can be of two types (UNI-C and UNI-N). A UNI-C is located on Customer edge equipment where as the UNI-N is located on the Provider equipment. A UNI-C and UNI-N interface are always connected. Figure 3 7/32

8 An EVC that spans multiple Autonomous Systems consists of concatenated OVCs. OVCs do not span multiple autonomous systems. Figure 4 EVCs can be defined as an E-LINE or E-ACCESS service if point-to-point, E-LAN if multipoint-tomultipoint or E-TREE if rooted multipoint. EVCs or OVCs can by multiplexed at the UNI and ENNI, which make it possible to deliver virtualized services like EVP-LINE and EVP-LAN. The MEF defined several service attributes: EVCs and OVCs service attributes: Virtual connections Bandwidth profiles Class of Service Identification Frame Delay Inter Frame Delay Frame Loss Availability UNI/ENNI service attributes: Physical interface capabilities Service multiplexing capabilities (for example VLAN) C-VLAN bundling capabilities These service attribute definitions are a means to standardize Carrier Ethernet Services (CES). Provider Backbone Bridging Traffic Engineering (PBB-TE) and Multi Protocol Label Switching Transport Profile (MPLS-TP) are both Carrier Grade transport technologies capable of delivering CES as defined by the MEF. 8/32

9 4 Provider Backbone Bridging (PBB) and PBB-TE (Traffic Engineering) 4.1 PBB Provider Backbone Bridging (PBB) (also known as MAC-in-MAC) is specified in the 802.1ah standard and provides connection-less transport for Ethernet services. PBB employs an encapsulation scheme, which tunnels customer Ethernet frames in provider frames. There is a clear separation between the customer space and provider space. Customer MAC addresses are learned only on the provider Backbone Edge Bridges (BEB) and are not used in provider Backbone Core Switches (BCB) for forwarding decisions. This is accomplished by having the BEB encapsulate the customer frames that are received on the UNI-N in a provider MAC header. In a PBB network, forwarding is solely based on the Backbone VLAN (B-VID), which is a dedicated reserved Service Provider VLAN (S-VLAN). This means that the B-VID must be unique on each NNI within the PBB network. BCBs do not need to be PBB capable, as they just need to forward frames based on the B-VID. In other words, BCBs need at least 802.1q support to transport 802.1ah (PBB) tagged frames. With PBB, carrier grade resiliency can be achieved using technologies like Transparent Interconnection of Lots of Links (TRILL) or Shortest Path Bridging Mac (SPBM), instead of using a STP flavor for layer-2 loop avoidance. However these protocols currently do not have Traffic Engineering (TE) capabilities as with PBB-TE. With PBB, the number of transport tunnels is limited to 4096, as forwarding is based on a 12-bit B-VID only. In the following figure the Provider Backbone Bridging 802.1ah frame format is shown: Figure 5 As shown in the above figure the payload (original customer frame) is encapsulated in a PBB frame, which consists of a B-TAG and I-TAG. The B-TAG is equivalent to a Service Provider VLAN (S-TAG) and is used for forwarding decisions within PBB networks. This is the reason that switches with 802.1q support are able to transport 802.1ah frames. The I-TAG is a field that conveys the service instance information (I-SID) associated with the frame. The I-SID field is 24 bits and is used as a unique service identifier. Compared to the 802.1q standard where only 12 bits are reserved for service (VLAN) identification this is a huge improvement in terms of scaling. The B-TAG (32 bit) format: TPID (Tag Protocol Identifier), default 0x88a8, 16 bits Priority, 802.1p, 3 bits DEI (Drop Eligibility Indicator), default 0, 1 bit B-VID (Backbone Virtual Local Area Network Identifier), 12 bits 9/32

10 The I-TAG (48 bit) format: TPID (Tag Protocol Identifier), default 0x88e7, 16 bits Priority, 802.1p, 3 bits DEI (Drop Eligibility Indicator), default 0, 1 bit. Use Customer Address: default 0, 1 bit Reserved 1, default 0, 1 bit Reserved 2, default 0, 2 bits I-SID: Service Identifier, 24 bits By mapping customer frames to a 24-bit service identifier (I-SID) it is possible to support more than 16 million services in the provider network, which is a huge improvement over the 802.1ad standard. BEBs have two PBB components, each serving a particular task for PBB processing. The I-component is responsible for bridging functions within the customer Ethernet space. It receives customer frames and learns customer MAC addresses. The I-component maps customer frames to a backbone service instance. It encapsulates customer frames with an I- TAG and a backbone MAC address and service identifier (I-SID). The B-component is responsible for bridging functions within the provider space. The B- component relays customer frames between I-component and other B-components in the PBB network. Customer frames are encapsulated with a B-TAG. 4.2 PBB and PBB-TE Frame processing The following figure shows PBB-TE frame processing between two IB- BEBs. Figure 6 A service frame received from a PBN with S-VLAN 500 arrives at the I-component of IB-BEB1. The I- component is responsible for processing the received frame and forwarding to the B component. The I-component may pop the S-VLAN 500 and pushes the I-SID to the frame. It also pushes the IB-BEB1 source address and IB-BEB2 as the destination to the header. The frame is transmitted to the B- component of IB-BEB1 using the I-components outgoing Provider Instance Port (PIP) interface. The B- component processes the frame, pushes B-VID 2000 to the frame and forwards it to the BCB based 10/32

11 on a statically defined (PBB-TE) forwarding entry / label information. Optionally the B-Component may translate the I-SID and BEB Destination MAC. With PBB-TE all forwarding entries are programmed statically by a Network Management System, thus MAC-learning is not needed to find the remote IB- BEB. When the frame (with I-TAG and B-TAG) arrives at the BCB, the BCB performs a forwarding lookup and forwards the frame to the outgoing interface. When the frame arrives at IB-BEB2 the B- component pops the B-VLAN and forwards it the I-component of IB-BEB2 using the B-components outgoing CBP interface. The I-component pops the IB-BEB1 source MAC address, IB-BEB2 destination MAC address, pops the I-SID and may push the outgoing customer S-VLAN 501. Only the I-component of an IB-BEB learns customer mac-addresses. Figure 7 The I- and B-component may be located on different BEB devices, so it is possible to have an I-BEB and a separate B-BEB. Most vendors develop BEBs supporting both components on the same device, which are called IB-BEBs. An I-tagged interface provides access between the I-component (customer domain) and the B-component (provider domain). The I-tagged interface on the I-component facing the B-component is called the Provider Instance Port (PIP). The I-tagged interface on the B- component facing the I-component is called the Customer Backbone Port (CPB). Multiple I- Components may be connected to the same B-component. The UNI-C facing interface located on the I-Component is called the Customer Network Port (CNP). I-SIDs traveling over the I-tagged interface identify customer service instances. A customer service instance frame is send to the CPB interface, where it is encapsulated with a B-TAG and transported over the Provider Backbone Port (PBP) towards the PBT network. Figure 8 BCBs are not directly attached to I-Components, meaning they only have PBPs attached. Incoming frames received on BCB PBP ports are forwarded based on the B-TAG in PBB networks or forwarded based on the B-TAG + Destination BEB MAC label information in PBT networks. 4.3 PBB-TE / PBT Provider Backbone Bridging Traffic Engineering or also called Provide Backbone Transport (PBT) uses the PBB (802.1ah) encapsulation scheme. PBB-TE originally developed by Nortel and pushed by BT, tried to receive standardization in the ITU and IETF and eventually resulted in the IEEE 802.1Qay standard. PBB-TE provides connection-oriented services and builds upon the 802.1ah standard by providing Traffic Engineering support and carrier grade service resiliency. With PBB-TE true traffic engineering support is possible, since PBB-TE uses an external management plane, which populates the PBB-TE bridges with static forwarding entries called Ethernet Labeled Switched Paths (ESPs). 11/32

12 The PBB-TE management plane (NMS) ensures that the PBB-TE network is loop free, which means that STP is not needed and must be disabled for the PBB-TE VLAN range used in the PBBN. Carrier grade service resiliency (sub 50ms failover) is supported, by providing pre-provisioned backup paths (each with a different B-VID) in combination with OAM (CFM) for fault detection and service failover. Several drafts are proposed for dynamic control plane support for PBB-TE, where GMPLS is used to signal the bi-directional Ethernet Label Switched Paths (ESPs) and provisions OAM functions in the network. PBB-TE supports the construction of point-to-point and point-to-multipoint Traffic Engineered unidirectional ESPs. PBB-TE Service instances consisting of two ESPs between the same CBPs (one in each direction) are called point-to-point Traffic Engineered Service Instances (TESIs). Point-tomultipoint TESIs have one point-to-multipoint ESP from the CBP of the root BEB towards the CBP on each leaf BEB and also one point-to-point ESP for each leaf BEB towards the root BEB. The PBB-TE management plane is responsible for path congruency, meaning that two unidirectional ESPs for the same Service Instance must follow the same physical path in each direction. In PBB-TE forwarding is based on 46 bits of the 48 bit Destination BEB MAC address + the B-VID (12 bits), so 58 bits in total. Unlike PBB, with PBB-TE the B-VID may be reused on the NNI as long as the Destination BEB MAC is different. This means that with PBB-TE it is possible to have 4096 ESPs towards the same destination BEB, where as with PBB the maximum is 4096 tunnels for the whole PBBN (assuming per hop BVID rewriting is not used, which buys some additional room due to BVID re-usage). As all PBB-TE paths are statically provisioned by the NMS, there is no need to support the forwarding of broadcast and or unlearned frames for BEB MAC address reachability. These broadcast or unlearned frames are either dropped or forwarded as unicast traffic to remote BEBs. From a services perspective, PBB-TE only supports point-to-point services, thus E-LAN and E-TREE services are not natively supported. This does not mean that E-LAN services cannot be provisioned, however the design and applicability is often very limited and heavily relies on vendor specific / proprietary features. Figure 9 In the figure above an example of a proprietary hierarchical PBB-TE E-LAN service is shown. Bridges H1 and H2 have a dedicated active PBB-TE ESP established. BEB1, BEB2 and BEB3 have an active ESP towards bridge H1. BEB1, BEB2 and BEB3 also have a standby ESP established towards node H2, which only becomes active in the event that the ESP towards node H1 goes down. However it is clear that traffic between BEB1 and BEB2 must always transit bridge H1 or H2 or both. BEB1, BEB2 and BEB3 cannot have a direct ESP between each other, as it would create a layer 2 loop in the E- LAN service due to the lack of split horizon mechanisms on the NNI ports. Furthermore proprietary 12/32

13 failure detection mechanisms must be implemented to address MAC learning / black holing issues arising from BEB ESP failover, as the H nodes are unaware of these state changes. 4.4 PBB-TE OAM In PBB-TE networks, OAM is used to detect failures, measure performance and provides diagnosis functions. The IEEE developed the 802.1ag standard and the ITU developed the Y.1731 standard, both forming the PBB-TE OAM protocol suite. Both standards describe a similar connection fault management mechanism based on the exchange of continuity messages. The Y.1731 standard is complimentary to the 802.1ag standard as it also defines performance management. OAM Protocol Data Units (PDUs) transported over Ethernet are identified with an ether-type of 0x8809. The use of a unique ether-type makes it easy for network components to distinguish OAM traffic from Ethernet service traffic. Figure 10 OAM PDUs may carry different sorts of messages depending on the flags and codes set in the header. For example OAM flags may indicate: Link Fault, Dying Gasp, Critical Events and OAM Discovery, where OAM codes may indicate: Events, Requests and Responses, Loopback configuration and Information. Connection Fault Management (CFM) is used to verify connectivity of TESIs or ESPs. CFM is enabled by the exchange of Continuity Check Messages (CCMs). CCMs are exchanged between two or more PBB-TE nodes for fault detection. If CCM messages are not received within a certain time interval a fault is detected. Upstream or downstream PBB-TE nodes can also be informed about a failure with Y.1731 AIS (Alarm Indication Signals), FDI (Forward Defect Indication) and BDI (Backward Defect Indication) signals. CCM messages discover and verify the health of OAM neighbors, which are basically two or more MEPs exchanging CFM messages within the same MD. CFM can also be used for path discovery using link trace messages or fault isolation using the CFM loopback mechanism. Maintenance domains (MD) divide the Carrier Ethernet Network (CEN) into several individually manageable OAM domains. MDs are assigned a level ranging from 0 to 7. The smallest MD (the operator level) is assigned the lowest MD level, where as the largest MD (the customer level) is assigned the highest MD level. Maintenance associations (MAs) define a set of Maintenance association End-Points (MEPs) and Maintenance association Intermediate Points (MIPs) within MDs. Thus it is possible to have multiple MAs within the same MD. MEPs are defined at the edges of an MD and are able to initiate and respond to CFM messages. MIPs are always internal to MDs, are passive and only able to respond to CFM loopback and trace-route messages. MEPs can be defined in two directions: Up MEPs are always facing away from a remote Up MEP and send and receive CFM messages to and from a bridge relay on a PBB-TE switch. Down MEPs are always facing remote Down MEPs. 13/32

14 Figure 11 In the figure above several MDs are shown. The level 6 MD spans two CENs where the MD edges are defined with Down MEPs on the CE UNI-C ports. The level 4 MD spans two CENs, where the MD edges are defined with Up MEPs on the CEN BEBs. In the level 4 MD two MIPs are defined at the CEN ENNIs to be able to respond to loopback and CFM trace-route messages initiated by the MEPs. The level 2 MDs span both a single CEN and have the MD edges defined using UP MEPs on the CEN BEBs. In the example shown, the scope of the level 6 MD is within the customer space as CFM messages are sent between the customer CE devices. The level 4 MD scope is within the service provider space as the CFM messages are sent between a BEB device of CEN 1 and a BEB device of CEN 2. The level 2 MD scope is within the operator space/ single CEN space PBB-TE Protection Switching PBB-TE supports linear 1:1 protection switching. For each primary ESP a dedicated backup ESP is established. CFM is used to detect failures and triggers failover to the backup ESP in case the primary ESP fails. By default three CC (Continuity Check) messages must be missed to trigger failover. Depending on hardware performance, CFM CC messages can be sent at a 10ms or even 3.3ms interval, thus providing for sub 50ms protection switching support. ESP reversion ensures that a previously failed primary ESP, when restored, will become the primary ESP again. Figure 12 14/32

15 4.4.2 PBB-TE Security PBB-TE management plane security threats come in the form of attacks to the NMS system/ provisioning system or data plane attack. Attacks to the NMS are either directly focused on compromising the NMS or the communication channels between the NMS and the network devices. PBB-TE attacks include: disclosure of information, taking down PBB-TE tunnels, DoS and (in-band) OAM attacks. The following defense techniques can be implemented to protect against these attacks: authentication, encryption and monitoring and logging. 15/32

16 5 Multi Protocol Label Switching (MPLS) and MPLS Transport Profile (MPLS-TP) Multi Protocol Label Switching (MPLS) was originally developed by the IETF as a technique to address performance limitations in traditional IP routing platforms. MPLS gained huge popularity with Service Providers due to the support for converged services (IP, L2VPN and L3VPNs) in packet switched networks. Over the years packet switching has become the dominant technology in service provider networks. For this reason it made sense to enrich the MPLS protocol suite with a protocol able to transport connection-oriented services over a packet switched network. Alcatel invented the T-MPLS protocol and tried to standardize it in the ITU and IETF. The ITU and IETF worked in parallel on several MPLS subsets like Pseudowire (PW) and OAM mechanisms and made their own recommendations in relation to protection switching for MPLS connection-oriented transport. It was clear that the ITU was redefining major aspects of the IETF MPLS protocol without consent of the IETF. The IETF was concerned that T-MPLS developed by the ITU-T did not have interoperability with native MPLS. A Joint Working Team (JWT) was formed between the ITU and the IETF for co-development on MPLS-TP and the ITU-T stopped working on the T-MPLS standard. MPLS-TP is a connection oriented transport technology defined by the "IETF". With MPLS-TP it is possible to statically configure Labeled Switched Paths (LSPs) and Pseudowires (PWs) through the management plane (NMS). MEF E-LINE, E-LAN and E-TREE services are supported with MPLS-TP. MPLS-TP is typically targeted in the access/ distribution layer of service provider networks and is interoperable with MPLS, which usually runs in the provider core. MPLS-TP has the following planes defined: Management plane: Maintenance, provisioning and configuration through an NMS (Network Management System). Control plane: Optional (GMPLS and T-LDP). Data or forwarding plane: Takes care of forwarding of data using MPLS push, pop or swap operations. To enforce connection oriented transport characteristics, MPLS-TP differs from MPLS in the following: No PHP support: Penultimate Hop Popping ensures that the penultimate hop (the node before the LSP egress node) pops the outer MPLS label. This ensures that the egress MPLS device does not have to perform a double forwarding lookup (Lookup in label database, pop the MPLS header, lookup in IP forwarding table and then forward the payload based on the IP header). Unfortunately this also breaks end-to-end MPLS OAM support. For this reason PHP is not supported in MPLS-TP. No LSP merging: With LSP merging it is possible to forward traffic from different sources to the same destination, on the same link, using the same outgoing label. Although efficient in terms of label space usage and the number of forwarding entries, LSP merging might break OAM, as it s difficult to determine the origin of OAM packets. No ECMP: Equal Cost Multi Path forwards traffic belonging to the same LSP over multiple paths towards the destination. This breaks MPLS-TP OAM requirements as Connectivity Check Messages (CCM) may follow different paths. Support for co-routed bidirectional LSPs: For proper OAM support it must be ensured that LSPs between source and destination PEs follow the same path (transit nodes and links). This cannot be guaranteed with plain vanilla MPLS where LSPs are unidirectional and follow the IGP best path. 16/32

17 With MPLS-TP, IP support is optional. This means that features like neighbor discovery, OAM forwarding and protection switching must also be supported without IP forwarding capabilities, control plane and or routing functionality. For example in non-ip environments the ARP protocol cannot be used to discover the Layer-2 interface addresses of MPLS-TP neighbors. So another protocol like the Link Layer Discovery Protocol (LLDP) or the G-Arch Advertisement Protocol (GAP) must be used for neighbor discovery. Management and provisioning in MPLS-TP networks is based on using a centralized NMS. The NMS is used for static service provisioning and monitoring. It is also possible the enable a dynamic control plane using Generalized Multi Protocol Label Switching (GMPLS). 5.1 MPLS-TP management and data plane With MPLS-TP a Network Management System (NMS) is responsible for management and provisioning of the network elements. An optional dynamic control plane can be enabled, where GMPLS is used for LSP signaling and Targeted LDP (T-LDP) for PW signaling. The NMS takes care of LSP and PW label assignments, defines the primary and backup paths and OAM functions on MPLS-TP network elements. Using a centralized management plane for these functions will reduce costs, as hardware complexity on MPLS-TP devices is reduced. However a dynamic control-plane enabled on MPLS-TP devices can be useful in parts of the network where simplified provisioning or multi vendor support is required, but will likely increase component costs. Figure 13 Label Switched Paths (LSPs) define the path through the MPLS-TP network. LSP ingress or Egress nodes are called Label Edge Routers (LERs) and LSP transit nodes are called Label Switched Routers (LSR). LERs encapsulate packets received on the UNI-N with one or more MPLS labels (push operation) depending on the provided service and or protection mechanisms used. In MPLS networks packets are stamped with a MPLS header. This header is inserted between the Layer 2 and Layer 3 information of a packet. For this reason MPLS is often referred to as Layer 2.5 networking. Networking equipment is able to distinguish and process MPLS packets easily, as MPLS has its own Layer 2 Ether-type (8848) identifier when Ethernet encapsulation is used. A MPLS header can contain one or multiple labels. In the case of multiple labels, these labels are stacked upon each other. This process is called MPLS label stacking. Forwarding in an MPLS network is based on the topmost MPLS label information within the label stack. Each entry in the label stack consists of four fields: Label, TC, S and TTL as shown in the figure below. 17/32

18 Figure 14 Three traffic class (TC) bits in the MPLS header provide for Class of Service (CoS) support using the Differentiated Services (DiffServ) Architecture. Pipe and Short pipe DiffServ models can be used when the CoS requirements are different for the provider and the payload. In other cases Uniform mode is also acceptable and supported in MPLS-TP networks. The bottom of the Stack bit (S) is set for the last/ bottom label in the stack and ensures that egress LSRs can identify the last MPLS label to be processed and proceed with traditional IP forwarding. Each label has an 8-bit TTL value, which is used for loop prevention in the MPLS network. MPLS devices perform the following label operations: Push: Adding an outgoing label (to the stack) Pop: Removing an incoming label (from the stack) Swap: replace/ swap the incoming label with an outgoing label Figure 15 In the figure above MPLS forwarding and label operation is shown (in one direction). The MPLS ingress LSR receives a packet on its UNI-N interface and performs a forwarding lookup. When the next-hop is determined to be an LSP, a MPLS header is pushed onto the packet and forwarded. MPLS transit routers perform a forwarding lookup based on the received labeled packet. The transit LSR swaps the incoming label with an outgoing label and forwards the packet to next LSR. If the transit LSR is also the penultimate LSR, it will not pop the MPLS label. This behavior is different than the default behavior for native MPLS environments. The penultimate LSR will perform a label swap operation and forwards it to the egress LSR. Note that a label swap operation may be interrupted in case of TTL expiry in the MPLS header. When the TTL expires the local LSR must process the packet or discard the packet depending on MPLS header information. The egress LSR pops the MPLS label, performs a forwarding lookup and transmits the packet in its native form to the egress interface. 18/32

19 The following MPLS-TP LSP types are supported Point-to-point unidirectional; A (blue) unidirectional LSP from node A to node C. The green LSP in the opposite direction (node C to node A) is not related and may follow the same or a completely different path or might even not exist. Figure 16 Point-to-point associated bidirectional; Consists of two unidirectional LSPs (in this example one LSP from node A to node C and one LSP from node C to node A), which is seen by the management plane as one bidirectional LSP. The LSPs do not need to follow the same path (non congruent). Figure 17 Point-to-point co-routed bidirectional (default); Consists of two unidirectional LSPs (node A to node B and node B to node A), which is seen by the management plane as one bidirectional LSP with the constraint that both LSP's must follow the same path. P2P co-routed bidirectional LSP can be considered the default MPLS-TP LSP type, as it is required to deliver CE services. Figure 18 Point-to-multipoint unidirectional; A LSP with two or more egress nodes. P2MP LSPs are typically used to transport broadcast and multicast packets. The advantage of using a P2MP LSP is that there is no ingress replication for multicast traffic destined to node C and node F on node A. Data sent over a point to multipoint LSP is received on all LSP egress nodes (in this example, nodes C and node F) Figure 19 19/32

20 5.2 Pseudowires A Pseudo-Wire (PW) is a mechanism that emulates the essential attributes of a native layer 2 service while transported over a Packet Switched Network (PSN). PW services are identified with an additional MPLS label, which is pushed onto the MPLS label stack by the ingress PE. Figure 20 An MPLS LSP can transport multiple PWs, as each PW is uniquely identified by means of a PW/ VC label. For example two PWs traveling between the same PEs in the MPLS-TP network have the same outer label (LSP label), but a different inner label (PW label). The PW label is assigned by an NMS or by a targeted LDP session in the case a dynamic control plane is used. Figure 21 An optional 4-byte control word can be inserted between the PW label and the Ethernet payload. The control word is used to carry protocol specific control information and sequence numbers. For example, in environments where Frame relay or ATM would be the layer 2 payload type, the control word would be used to tunnel DE/ ECN information. However for Ethernet services the control-word is optional and may be used for OAM purposes (VCCV). 20/32

21 Figure 22 In the figure above MPLS-TP forwarding and label operation is shown (one direction). The MPLS-TP ingress PE receives Ethernet signals on its UNI-N port. An MPLS transport label and PW label is pushed onto the packet and the packet is sent to the next LSR. The P node performs a forwarding lookup based on the received outer label in the packet (label 25). The P node swaps the incoming label (label 25) with an outgoing label (label 100) and forwards the packet to next LSR. The egress PE pops the MPLS transport label and the PW label, performs a forwarding lookup based on the PW label information to determine the outgoing interface and finally forwards the client payload in its native form. Client signals received on the UNI-N are called Attachment Circuits (ACs). These signals can be Ethernet, SONET, ATM, Frame-relay, etc. In the case of Ethernet ACs, they can either be port based (raw) or VLAN based (multiplexed). With most vendor implementations, when the AC is port-based all frames, including protocol control frames like BPDU's, LACP, etc are tunneled over the PW. With multiplexed ACs this is more complex. Depending on vendor implementations Generic Bridge PDU tunneling (GBPT) is required. MPLS-TP also supports multi segment Pseudowires (MS-PWs), which provide the ability to extend EVCs over multiple Carrier Ethernet Service Provider networks or different regions within an Autonomous System and thereby creating a hierarchy. MS-PWs can be managed and monitored individually by different administrative entities. A Switching Provider Edge (S-PE) switches between the PWs, thus forming a MS-PW. Another option is to use a dedicated AC between two PE devices to glue the PWs together. In the following figure a MS-PW is shown, where PE1 and PE2 provide a PWE service between their ACs. 21/32

22 Figure 23 MPLS-TP LSPs (or PSN tunnel in PWE3 terminology) are used to provide the data path for the PWs. One MPLS-TP LSP (LSP1) between PE1 and S-PE and another LSP (LSP2) between S-PE and PE2. PW1 and PW2 are segments of the MS-PW. MS-PW can be useful when a PW must be extended between two or more CENs, for large CENs where different regions have different administrative entities or when different PWE3 signaling protocols are used in different parts of a CEN. As with Single Segment PWs, MS-PWs can be signaled dynamically or can be statically provisioned by an NMS. 22/32

23 5.3 MPLS-TP OAM A mandatory requirement for transport networks is the support for end-to-end path management and monitoring to enforce SLAs. MPLS-TP addresses these requirements with OAM fault management and performance monitoring capabilities. MPLS-TP provides OAM at the LSP, LSP Section and PW level. PWs have an Associated Channel (PW3-ACh) defined to provide OAM support. The PW3-ACh is based on the PW control word with some minor changes. MPLS-TP LSPs do not have a controlword field, which can be used for OAM capabilities (like PWE3-ACh). A requirement for MPLS-TP OAM is that it must be able to operate without IP protocol support. Furthermore it is mandatory that OAM packets follow the same path (congruency) as Ethernet services traffic. MPLS-TP addresses these OAM requirements with the support of the Generic Associated Channel (G-ACh) and the Generic Associated Label (GAL). With G-ACh LSP OAM information is fate shared with Ethernet services traffic. G-ACh is a header, which is pushed onto the MPLS-TP frame. Since OAM is provided in-band, MPLS-TP devices must differentiate between OAM and regular service traffic. The reserved GAL (value = 13) is used to make this differentiation. The GAL is always located at the bottom of MPLS label stack (S=1), thus G-ACH is only processed when MPLS-TP devices pop the GAL or if the MPLS TTL has expired. Figure 24 The G-ACh header supports multiple OAM types like Bidirectional Forwarding Detection (BFD), OAM fault management, etc. These OAM types are identified with the G-ACh Channel field. As usual, drama between the ITU-T and the IETF caused different G-ACh based OAM standards to be approved. The IETF prefers BFD, which is also known as MPLS-TP OAM. The ITU-T prefers Y.1731 to be used in G-ACh. BFD is supported by the ITU-T, although Y.1731 is not supported by the IETF. BFD is a lightweight hello protocol developed to detect (bi-directional) failures between routers. BFD runs over directly connected IP neighbors, multi hop IP neighbors and also over MPLS-TP and PWs. Currently BFD can only be used for failure detection, so it's not a full-blown OAM protocol like Y.1731, which additionally support frame delay measurements, link trace, connection verification, etc. 23/32

24 5.4 MPLS-TP Protection Switching MPLS-TP supports carrier grade service resiliency due to rich OAM support. OAM messages sent at a fast rate can achieve sub 50ms convergence. MPLS-TP protection can be linear or ring and supports 1+1 unidirectional, 1:1 uni- or bidirectional and 1:n protection when a static control plane (NMS) is used. In 1+1 unidirectional protection switching a dedicated protection LSP is reserved for each working LSP. Traffic is sent on both LSPs by the source PE, where the egress node selects the LSP (local switching mechanism) to accept traffic from. The selection mechanism is triggered based on OAM keepalives. Figure 25 With 1:1 unidirectional protection the source PE uses OAM keepalives to verify connectivity for the working and protection LSP. The Protection State Coordination Protocol is needed (except for 1+1 uni protection) as both sides/pes of the unidirectional LSP need to be in sync to determine the active path. PSC messages are sent over the G-Ach channel. LSP reversion is also supported. This feature ensures that a previously failed primary LSP becomes the primary LSP again. Figure 26 Besides protection at the LSP level, MPLS-TP also supports protection at the PW and MS-PW segment level. 24/32

25 Figure 27 MPLS-TP protection switching can be triggered by a link, node, LSP, PW failure. It is also possible to trigger protection switching manually with OAM LCK signaling. LSPs with the LCK status only transport OAM frames, but not service frames. This feature is useful in a scenario where maintenance needs to be performed on the physical path and service traffic must be switched to the backup LSP. 5.5 MPLS-TP Security MPLS-TP management plane security threats come in the form of attacks to the NMS system/ provisioning system or data plane attack. Attacks to the NMS are either directly focused on compromising the NMS or the communication channels between the NMS and the network devices. MPLS-TP attacks include: disclosure of information, taking down MPLS-TP LSPs, connecting to the wrong MPLS-TP tunnel endpoints, DoS and (in-band) OAM attacks. With MPLS-TP OAM attacks, the G-ACh can be modified or a DoS attack can be performed against G-ACh, causing MPLS-TP protection paths to switchover and might cause congestion or an outage in the MPLS-TP network. In general OAM packets that cannot be identified should be discarded and not be further processed to avoid resource exhaustion or device outages. The following defense techniques can be implemented to protect against these attacks: entity authentication, management system authentication, peer-topeer authentication, encryption and monitoring and logging. 25/32

26 6 PBB-TE and MPLS-TP comparison Both PBB-TE and MPLS-TP are carrier grade transport technologies providing more or less the same functionality. From a management plane perspective, both technologies use the concept of static provisioning and monitoring through an NMS and have the option for a dynamic control-plane. PBB-TE data-plane encapsulation and forwarding is based on Ethernet enhancements. Often referred as an advantage, is the fact that PBB-TE is interoperable with regular Ethernet switches. However there are also disadvantages. The downside in such a scenario is that the NMS might loose the ability for end-to-end path control and additional layer-2 loop prevention mechanisms are needed on the native Ethernet switches. This adds complexity and scalability issues to the PBBN. Most hardware used in native Ethernet switches cannot handle the concept of forwarding on 58bits (BVID+DMAC BEB) and forward solely on the BVID. This reduces PBB-TE scaling in terms of the maximum ESPs supported, as the BVID upper limit is 4096 per NNI in the PBBN. Therefore dedicated PBB-TE hardware is needed to maximize the full potential of PBB-TE PBBNs in terms of scalability and stability. MPLS-TP uses a concept of forwarding based on per interface specific 20 bits labels, which is inherited from the MPLS standard and highly scalable. MPLS-TP allows stacking of (in theory) unlimited labels, where eventually hardware limitations, applicability and MTU considerations would become the limiting factor. In case of PBB-TE the hierarchy is limited to only one transport identifier (B-VID) and one payload identifier (I-SID). MPLS-TP is interoperable with MPLS, although some native MPLS features cannot be used to enforce connection-oriented transport characteristics needed in MPLS-TP networks. This means that MPLS- TP equipment needs less functionality and is often referred to as being less expensive than full-blown MPLS devices. However the MPLS-TP standard also has the support for an optional dynamic control plane, which will likely increase equipment costs due to the added hardware complexity. PBB-TE and MPLS-TP (Ethernet) labeled switched paths are point-to-point. Both standards also define the functionality of creating point-to-multipoint LSPs to reduce traffic replication on NNI ports for one-to-many service flows. For both technologies the NMS is responsible for provisioning congruent bi-directional LSPs. PBB-TE does not support E-LAN services, whereas the MPLS-TP standards have support for E-LAN and E-TREE services described. From a QoS perspective both protocols support statistical multiplexing, which provides for efficient UNI/NNI bandwidth allocation within the network. PBB-TE and MPLS-TP both have rich OAM support, which is a mandatory requirement for CENs. PBB-TE uses 802.1ag and Y.1713 for OAM fault detection and performance monitoring. With MPLS- TP there are two standards; the ITU-T standard is based on 802.1ag and Y.1731, whereas the IETF standards are based on the BFD protocol. When adding a new MPLS-TP vendor to an existing MPLS/ MPLS-TP environment, care should be taken to verify if OAM interoperability is maintained. Control plane security in PBB-TE and MPLS-TP is pretty good as only the edge nodes are vulnerable to attacks initiated from the customer data-plane. In PBB-TE networks the BCBs do not need to act on the customer payload (for e.g. learning and forwarding based on the customer MAC addresses), whereas in MPLS-TP networks the label stack hides the customer payload on LSRs. 26/32

27 The following table summarizes the capabilities of the PBB-TE and MPLS-TP protocols PBB-TE MPLS-TP Point to point ESP/LSP Supported Supported Point to multipoint ESP/LSP Supported Supported E-LINE services Supported Supported E-LAN services Not supported Supported E-TREE services Not supported Supported Traffic Engineering (TE) Supported Supported Service resiliency (<50msec) Supported Supported Rich OAM Supported Supported Control plane security Good Good QoS Statistical MP Statistical MP 27/32

28 7 Conclusion PBB-TE standardized by the ITU is a transport enhancement for Ethernet and comes from the layer 2 world. MPLS-TP standardized by the IETF has its origins in the MPLS/ Layer 3 world. Both technologies are very different in terms on how they work under the hood, but provide the same functionality in terms of Carrier Ethernet transport. Over the years PBB-TEs industry acceptance turned out to be low. Although several vendors still support PBB-TE technology in their CE hardware, there haven t been any significant enhancements announced in ITU standards or by major vendors. Most CE vendors and operators, including some of the early backers of PBB-TE like Ciena and BT, have chosen sides for MPLS-TP. However the PBB encapsulation method used in PBB-TE networks is still used in many networks and also proposed in several new drafts for scaling massive (Datacenter) Ethernet networks. Furthermore the IEEE has decided to adopt PBB-TE into the SPB protocol suite, which when finished results in the SPB-TE protocol and based on IEEE 802.1Qca standard. Based on the amount of attention MPLS-TP receives from providers, the big push by MPLS vendors and many activities in the standardization bodies, it seems that MPLS-TP is the clear "winner". However for MPLS-TP there is still a long road ahead. Several vendors like Cisco and Juniper have implemented (basic) MPLS-TP functionality in their hardware, while others only have limited support or recently made MPLS-TP a roadmap item. There is still a lot of debate and many new drafts proposed on topics like, dynamic control plane functionality, OAM and enhancements to MPLS-TP protection mechanisms. 28/32

29 8 Acronyms Term Definition AC Attachment Circuit BCB Backbone Core Bridge BEB Backbone Edge Bridge BFD Bidirectional Forwarding Detection BPDU Bridge Protocol Data Unit CBP Customer Backbone Port CCM Continuity Check Message CES Carrier Ethernet Services CFM Connection Fault Management CoS Class of Service C-TAG Customer TAG ENNI External Network to Network Interface ESP Ethernet (Labeled) Switched Path EVC Ethernet Virtual Circuit G-ARCH Generic Associated Channel GAL Generic Associated Label GAP Generic Associated Channel Advertisement Protocol GMPLS Generalized Multi Protocol Label Switching IEEE Institute of Electrical and Electronics Engineers I-SID Service Instance Identifier ITU-T International Telecommunication Union - Telecommunication LACP Link Aggregation Control Protocol LAN Local Area Network LER Label Edge Router 29/32

30 LLDP Link Layer Discovery Protocol LSP Label Switched Path MAC Media Access Control MEF Metro Ethernet Forum MPLS-TP Multi Protocol Label Switching MS-PW Multi Segment Pseudowire MTU Maximum Transmission Units NMS Network Management System NNI Network to Network Interface OAM Operations Administration and Maintenance OVC Operator Virtual Circuit P2MP Point to Multi Point P2P Point to Point PBB-TE Provider Backbone Bridging Traffic Engineering PCP Priority Code Point PE Provider Edge PHP Penultimate Hop Popping PIP Provider Instance Port PSC Protection State Coordination PSN Packet Switched Network PW Pseudowire SDH Synchronous Digital Hierarchy SONET Synchronous Optical Networking SPBM Shortest Path Bridging Mac SPB-TE Shortest Path Bridging Mac Traffic Engineering S-PE Switching Provider Edge 30/32

31 S-TAG Service Provider TAG TC Traffic Class T-LDP Targeted Label Distribution Protocol TRILL Transparent Interconnection of Lots of Links UNI User to Network Interface VLAN Virtual LAN WAN Wide Area Network 31/32

32 9 References 1. draft-ietf-ccamp-gmpls-ethernet-pbb-te 2. IEEE IEEE 802.1ad 4. IEEE 802.1ah 5. IEEE 802.1q 6. IEEE 802.1Qay 7. IEEE 802.1Qca 8. MEF MEF MEF MEF MEF RFC 5317: JWT Report on MPLS Architectural Considerations for a Transport Profile 14. RFC 5586: MPLS Generic Associated Channel 15. RFC 5654: MPLS-TP Requirements 16. RFC 5960: MPLS Transport Profile Data Plane Architecture 17. RFC 6370: MPLS Transport Profile (MPLS-TP) Identifiers 18. RFC 6373: MPLS Transport Profile (MPLS-TP) Control Plane Framework 19. RFC 6378: MPLS Transport Profile (MPLS-TP) Linear Protection 20. RFC 6426: MPLS On-Demand Connectivity Verification and Route Tracing 21. RFC 6427: MPLS Fault Management Operations, Administration and Maintenance (OAM) 22. RFC 6428; Proactive Connectivity Verification, Continuity Check and RDI for the MPLS-TP 23. RFC 6435: MPLS Transport Profile Lock Instruct and Loopback Functions 24. RFC 6941: MPLS Transport Profile Security Framework 25. MPLS-TP Yaakov (J) Stein presentation - RAD Data 26. MPLS Architectural Considerations for a Transport Profile ITU-T- IETF JWT 32/32