L2 MPLS VPN (VPLS) Technology White Paper

Transcription

1 S9500 L2 MPLS VPN (VPLS) Technology White Paper L2 MPLS VPN (VPLS) Technology White Paper Keywords: MPLS, VPLS Abstract: MPLS technologies make it very easy to provide VPN services based on IP technologies and MPLS VPNs are highly scalable and easy-to-manage. There are two MPLS-based VPN services: L3 MPLS VPN and L2 MPLS VPN. L2 MPLS VPN further includes VPLS and VLL. VLL applies to point-to-point networking scenarios, while VPLS supports point-to-multipoint and multipoint-to-multipoint networking. From users point of view, the whole MPLS network is a Layer 2 switched network, through which Layer 2 connections can be established between sites. This document describes VPLS. Acronyms: Acronym Full spelling MPLS VPLS Multiprotocol Label Switching Virtual Private LAN Service Hangzhou H3C Technologies Co., Ltd. 1/27

2 S9500 L2 MPLS VPN (VPLS) Technology White Paper Table of Contents 1 Overview Basic Networking Architecture Features Terminologies Protocol Processing Mechanism Basic Transmission Components of VPLS MAC address Learning and Flooding VPLS Loop Avoidance Peer PE Discovery and PW Signaling Protocol H-VPLS Implementation Mode Packet Frame Structure Packet Encapsulation on the AC Packet Encapsulation on the PW VPLS Packet and Encapsulation Processing of User Data in the Entire Network Processing of Common L2 and L3 User Data in the Entire Network Processing of User Protocol Packet Data in the Entire Network Networking Overview Key Points of VPLS Networking Typical VPLS Networking Example Features of H3C S Features of H3C S9500 VPLS Complete H-VPLS Solution Feature Configuration and Processing of VPLS Instances H-VPLS AC Backup in the ME Mode Load Balance and Service Backup Binding of Multiple VLANs with a Single VPLS Instance Processing Flow of H3C S9500 VPLS VPLS-Relevant Features of H3C S References Hangzhou H3C Technologies Co., Ltd. 2/27

3 S9500 L2 MPLS VPN (VPLS) Technology White Paper 1 Overview As the social development moves on, the economic globalization trend becomes more obvious, more and more enterprises are more widely distributed and the mobility of company employees is increasing. This urges the telecom operators to provide link connections so that the branches of an enterprise can be incorporated to construct their own Intranet and their employees can easily access their Intranet outside the enterprise. At the very beginning, the telecom operators provided the enterprises with links via the leased lines. The major disadvantages of this method are that it does not suite the multi-branch and quick growth feature of the current enterprises and is also characteristic of high cost and difficulty in management. Then as the rise of ATM and frame relay technologies, the telecom operators turned to provide customers with point-to-point L2 connections via virtual circuits, based on which the customers constructed their own L3 networks to bear IP, IPX and other data streams. These technologies all provided point-to-point L2 connections and the configuration was complex. In particular, when a site was added, the administrator needed to make a lot of configurations. Today, IP networks are distributed all over the world and how to utilize the existing IP networks to provide low-cost private networks for enterprises gradually becomes the focus of the operators. Therefore, a kind of technology to provide VPN services on the IP network and freely set any rate with simple configurations emerges. This technology is the MPLS VPN. There are two MPLS-based VPN services: L3 MPLS VPN and L2 MPLS VPN. L2 MPLS VPN further includes VPLS and VLL. VLL only applies to the point-to-point networking application while VPLS can implement multipoint-to-multipoint VPN networking, provides a more complete solution for the operators who use point-to-point L2VPN service and can avoid the intervention in the user s internal route hierarchy as in the L3VPN. In this way, the operator may only need to manage and operate a single network and provide multiple services (e.g. best-effort IP service, L3 VPN, L2 VPN, traffic engineering, differentiated services, etc.) on this network. This reduces the operator s plenty of costs in construction, maintenance and operation. The VPLS service enables the users that are geographically isolated to connect with Hangzhou H3C Technologies Co., Ltd. 3/27

4 S9500 L2 MPLS VPN (VPLS) Technology White Paper one another via the MAN/WAN and enables the sites to be connected as if they were connected in a LAN. A series of drafts [1] of IETF have described the VPLS solution using the PWs (Pseudowires) of the MPLS as the Ethernet links and providing a transparent transmission LAN service (TLS) via the MPLS network. 2 Basic Networking Architecture In the draft [2] related to VPLS, two VPLS network architectures are proposed: The VPLS network with the fully-meshed logic connections of PWs (Pseudowires) and the hierarchical VPLS architecture, as shown in Figure 1 and Figure 2. Figure 1 Common VPLS network architecture As shown Figure 1, the PEs in various sites of the VPLS network are logically fully meshed. The VPLS network can provide point-to-multipoint connection service like L3VPN and the PEs can learn MAC addresses and switch packets between multiple points. The MPLS network provides tunnels for transparent transmission of VPN packets, and the P equipment in the network does not involve in the learning and switching of MAC addresses but only forwards the MPLS packets. Moreover, the forwarding tables between VPNs on the PEs are independently of one another and the MAC addresses can overlap between VPNs. Hangzhou H3C Technologies Co., Ltd. 4/27

5 S9500 L2 MPLS VPN (VPLS) Technology White Paper Figure 2 Hierarchical VPLS network architecture As shown Figure 2, in the hierarchical VPLS network architecture, the logic fully-meshed connections are implemented in the core equipment (NPE) while the user PE (UPE) is only connected with the nearest NPE via a PW to exchange packets with the peer site. In this way, the network topology is hierarchical and the access range is expanded. In the core network, the NPE has good performance/functions with centralized VPN service flows while the UPE has low performance/function requirement and is used for service access of VPNs. Meanwhile, link backup can be implemented between the edge access equipment and the NPE and this enhances the network robustness. The access network between the UPE and the NPE can be the MPLS edge network (connected via VPLS or VLL) or a simple Ethernet (connected via QinQ).In addition, the access mode of each UPE in the hierarchical VPLS network architecture may be in fixed use and the access types from the UPE to the NPE can be freely selected for the sites of the VPNs according to the actual access network conditions. Hangzhou H3C Technologies Co., Ltd. 5/27

6 S9500 L2 MPLS VPN (VPLS) Technology White Paper 3 Features 3.1 Terminologies MPLS L2VPN: MPLS L2VPN transparently transmits the L2 data on the MPLS network. In the point of view of the user, this MPLS network is an L2 switching network via which the L2 connections can be set up among different sites. There are two types of MPLS L2VPN: VLL and VPLS. VPLS (Virtual Private LAN Service): It is a point-to-multipoint L2VPN service provided in the public network. The VPLS service enables the users that are geographically isolated to connect with one another via the MAN/WAN and enables the sites to be connected as if they were connected in a LAN. VLL (Virtual Leased Line): It is a point-to-point L2 VPN service provided in the public network. It enables two sites to be connected as if they were connected via a line. It cannot provide switching among multiple points of the service provider. CE (Custom Edge): It is the user edge equipment directly connected to the service provider. PE (Provider Edge Router): It refers to the edge router in the backbone network and is connected to the CE for the access of VPN services. It completes the mapping and forwarding of packets from the private network to public network tunnels and from public network tunnels to the private network. PEs can be further divided into the UPE and the NPE. UPE (User facing-provider Edge): It is the PE equipment close to the user side and serves as the convergence equipment for users to access the VPN. NPE (Network Provider Edge): It is the core PE of the network and is located at the edge of the core domain of the VPLS network to provide the VPLS transparent transmission service between the core networks. VSI (Virtual Switch Instance): Through the VSI, the actual access links of the VPLS can be mapped to various PWs. PW (Pseudo Wire): It is a bidirectional virtual connection between two VSIs and is composed of a pair of unidirectional MPLS VCs. Hangzhou H3C Technologies Co., Ltd. 6/27

7 S9500 L2 MPLS VPN (VPLS) Technology White Paper AC (Attachment Circuit): It refers to the connection between the CE and PE. It may be the actual physical interface or a virtual interface. All the user packets on the AC should generally be transparently transmitted to the peer site, including the L2/L3 protocol packets of the users. QinQ (802.1Q in 802.1Q): It is a mechanism directly using the 802.1q-based tunneling protocol of the Ethernet switch to provide multipoint L2VPN services. It encapsulates the private network VLAN tag of the user into the public network VLAN tag and the packet carries both layers of tags while crossing the backbone network of the provider, thus providing the user with a simpler L2 VPN tunnel. Forwarders: It is a kind of PE. A PE receives the data frames sent over the AC while a forwarder selects the PW for forwarding the packets. A forwarder is in fact the forwarding table of VPLS. Tunnels: Used for bearing PWs. One tunnel can bear multiple PWs, generally MPLS tunnels. A tunnel is a direct-connect channel between a local PE and the peer PE to transparently transmit data between the two PEs. Encapsulation: The packet transmission over the PW uses the standard PW encapsulation format and technology. There are two modes for VPLS packet encapsulation over the PW: Tagged mode and Raw mode. PW Signaling: The PW signaling protocol is the basis for VPLS implementation, used for establishing and maintaining PWs. It can also be used for automatically discovering the peer PE of a VSI. At present, there are two PW signaling protocols: LDP and BGP. Service Quality: To map the priority information in the L2 packet header of the user into the QoS priority for transmission over the public network, generally the MPLS network that supports traffic engineering should be applied. 3.2 Protocol Processing Mechanism The VPLS-related draft describes the basic transmission components of the VPLS network. All the VPLS services are completed by the series of transmission components. The VPLS solution provided by the draft also centers on the formation and application of these basic transmission components. In addition, the draft provides the hierarchical VPLS application solution with non-fully-meshed connections of PWs. Hangzhou H3C Technologies Co., Ltd. 7/27

8 S9500 L2 MPLS VPN (VPLS) Technology White Paper Basic Transmission Components of VPLS The whole VPLS network is just like a huge switch. It establishes PWs between sites of various VPNs via MPLS tunnels and transparently transmits L2 packets of users via these PWs. The PEs will learn the source MAC address and establish an MAC forwarding table entry while forwarding a packet, so as to complete the mapping between MAC addresses and user Attachment Circuits (ACs)/PWs. The P equipment only needs to complete the MPLS data forwarding according to the MPLS label without concerning the L2 user packets internally encapsulated in the MPLS packets. The transmission components of the VPLS network and their functions are described as follows: Attachment Circuit (AC): A connection line or virtual link between the CE and the PE. All the user packets on the AC should generally be transparently transmitted to the peer site, including the L2/L3 protocol packets of the users. Pseudo wire (PW): A bidirectional virtual connection established between two VSIs of one VPN. It is composed of a pair of unidirectional MPLS VCs, borne over the LSP and established via the PW signaling protocol. For the VPLS system, the PW is just like a direct-connect channel from one local AC to the peer AC to transparently transmit L2 data of users. Forwarders: A PE receives the data frames sent over the AC while a forwarder selects the PW for forwarding the packets. A forwarder is in fact the forwarding table of VPLS. Tunnels: Used for bearing PWs. One tunnel can bear multiple PWs, generally MPLS tunnels. A tunnel is a direct-connect channel between a local PE and the peer PE to transparently transmit data between the two PEs. Encapsulation: The packet transmission over the PW uses the standard PW encapsulation format and technology. There are two modes for VPLS packet encapsulation over the PW: Tagged mode and Raw mode. Pseudo wire Signaling: The PW signaling protocol is the basis for VPLS implementation, used for establishing and maintaining PWs. It can also be used for automatically discovering the peer PE of a VSI. At present, there are two PW signaling protocols: LDP and BGP. Hangzhou H3C Technologies Co., Ltd. 8/27

9 S9500 L2 MPLS VPN (VPLS) Technology White Paper Service Quality: To map the priority information in the L2 packet header of the user into the QoS priority for transmission over the public network, generally the MPLS network that supports traffic engineering should be applied. The positions of the basic transmission components of VPLS in the network are shown in Figure 3 : Figure 3 Basic transmission components of VPLS Let s take the packet flow of VPN1 from CE3 to CE1 for example to describe the basic data flow direction. CE1 sends an L2 packet and the packet enters PE1 via the AC. After PE1 receives the packet, the forwarder selects a PW for forwarding the packet and the system then generates L2 MPLS labels according to the forwarding table entry of the PW (the private network label is used to identify the PW while the public network label is used to cross the tunnel and reach PE1). After the packet reaches PE2 through the public network tunnel, the system pops up the private network label (the public network label pops up via PHP on the P equipment). The forwarder of PE2 selects an AC for forwarding the L2 packet from CE3 to CE MAC address Learning and Flooding The control plane of VPLS does not need to advertise and distribute reachability information, but it lets the address learning of the standard bridge function in the data plane to provide reachability. (1) Source MAC address learning The MAC address learning process involves two parts: Remote MAC address learning associated with PW Because a PW is composed of a pair of unidirectional VC LSPs (the PW will be regarded as being up only when the VC LSPs in both directions are up), the PW should map the MAC address to the VC LSP in the egress direction when the VC LSP Hangzhou H3C Technologies Co., Ltd. 9/27

10 S9500 L2 MPLS VPN (VPLS) Technology White Paper in the ingress direction has learnt an MAC address originally unknown to it. Local MAC address learning of the port directly connected to the user For an L2 packet sent from the CE, the source MAC address in the packet should be learnt by the corresponding port on the VSI. The address learning and flooding process of the PE is illustrated in Figure 4. Figure 4 Address learning and flooding process of the PE (2) MAC address reclamation The MAC addresses dynamically learnt must have the refresh and re-learning mechanisms. In the draft [2] related to VPLS, a kind of address reclamation message that uses the optional MAC TLV for deleting or re-learning the specified MAC address list is provided. When the topology structure changes, the address reclamation message can be used so as to quickly remove the MAC addresses. The address message falls into two types: The address message with the MAC address list and the address message without the MAC address list. If a backup link (becoming active) receives a notification message with the re-learnt MAC address list, the PE will update the MAC address entry in the FIB table of VPLS instances and send this message to the other relevant LDP sessions to directly connect the PE. If the notification message contains a null MAC address TLV list, it indicates that the PE should remove all the MAC addresses in the specified VPLS instance (except the MAC addresses learnt from the PE that Hangzhou H3C Technologies Co., Ltd. 10/27

11 S9500 L2 MPLS VPN (VPLS) Technology White Paper sends this message). (3) MAC address aging The remote MAC addresses learnt by the PE require an aging mechanism to remove the table entries related to the VC label but no longer in use. After the packet is received, the aging timer corresponding to the source address shall be reset VPLS Loop Avoidance To avoid loop occurrence, the STP protocol should be enabled in a common L2 network but the private network STP evidently should not participate in the network of the ISP. In VPLS, fully-meshed connections and split horizon forwarding are used to avoid the running of STP on the ISP network. Each PE must create a tree for each VPLS forwarding instance to all the other PE routers in this instance. Each PE router must support the split horizon policy to avoid loop occurrence, that is, the PE router cannot forward packets between the PWs of the same VPLS instance (because all PEs are directly connected in the same VPLS instance). In this sense, split horizon forwarding means that the data packets received from the PWs on the public network side will no longer be forwarded to the other PWs but can only be forwarded to the private network side. In the point of view of the user, it is allowed to run the STP at the L2VPN private network side and all the BPDU packets of the STP are only transparently transmitted in the network of the ISP Peer PE Discovery and PW Signaling Protocol For the PEs in the same VSI, their addresses can be manually specified in a remote manner or automatically discovered by the other auto discover mechanisms. At present, the peer PE of a VSI can be automatically discovered via BGP or LDP and these two protocols can also be used as the PW signaling protocol to establish PWs. The establishment of a PW is to allocate a multiplex detachment label (VC label) and advertise the allocated VC label to the peer PE. In addition to label distribution, the PW signaling protocol is also used to advertise the parameters relevant to the VPLS system, for example, PW ID, control word, interface parameters and so on. Through the PW signaling protocol, a fully-meshed PW can be established between PEs to serve the VPLS. Hangzhou H3C Technologies Co., Ltd. 11/27

12 S9500 L2 MPLS VPN (VPLS) Technology White Paper H-VPLS Implementation Mode Because the VPLS solution described above requires that a fully-meshed tunnel LSP should be established between all the PE routers providing the VPLS service, n*(n-1)/2 PWs should be established between the PEs for each VPLS service. However, these PWs are all generated by the signaling protocol. The real disadvantage is that the above solution cannot achieve large-scale application, because the PE routers providing VCs need to duplicate data packets and each PE needs to broadcast the packet to all the peer equipment in the case of the first packet, a broadcast packet or a multicast packet. Through the hierarchical connections, we can reduce the load of the signaling protocol and data packet duplication(although the total number of broadcast packets duplicated remains unchanged, it is completed together by multiple devices in H-VPLS), so that the VPLS can be applied on a large scale. Generally, the LSP will place some small edge devices in the user inhabit and aggregate them into a PE in the central office. Therefore, it is quite necessary to extend the tunneling technology of VPLS service to the MTU (Multi-Tenant Unit). In this way, the MTU equipment can be regarded as a PE and used to provide the basic VPLS virtual connection service at each edge. The feasible technologies include the use of a PW and the Q-in-Q logical interface between the MTU and the PE. In the two-layer hierarchical VPLS, one layer is the core PW (hub) of VPLS and the other is the extended access PW (spoke). (1) Two access means of H-VPLS The two access means of H-VPLS are illustrated in the following figures: Figure 5 LSP access mode of H-VPLS As shown in Figure 5, the UPE works as the aggregation equipment MTU and it only establishes a PW with NPE1 to connect the link U-PW and does not establish any PW Hangzhou H3C Technologies Co., Ltd. 12/27

13 S9500 L2 MPLS VPN (VPLS) Technology White Paper with the rest peers. The data are forwarded in the following procedure: UPE1 sends the packet from a CE to NPE1 and adds the multiplex detachment label (MPLS label) of the U-PW to the packet. Upon receipt of the packet, NPE1 determines the VSI of the packet according to the multiplex detachment label and then adds the multiplex detachment label of the N-PW to the packet according to the destination MAC address of the user data packet before forwarding the packet. After receiving the packet from the N-PW, NPE1 adds the multiplex detachment label of the U-PW and sends the packet to the UPE. Upon receipt of the packet, the UPE then forwards it to the CE. If CE1 and CE2 exchange data for the local CEs, the UPE will directly forward the packets between CE1 and CE2 without needing to report the packet to NPE1 because of its bridge function. However, if it is the first packet or a broadcast packet whose destination MAC address is unknown, the UPE will still forward the packet via the U-PW to NPE1 while broadcasting the packet via the bridge to CE2, so that NPE1 can duplicate the packet and forward it to each peer CE. Figure 6 QinQ access mode of H-VPLS As shown in Figure 6, the MTU is a standard bridge device. QinQ is enabled on the CE access ports and the VLAN-TAG is attached as the multiplex detachment label. The packet is transparently transmitted to PE1 via the QinQ tunnel between the MTU and PE1. PE1 then determines the VSI according to the VLAN-TAG attached by the MTU and then adds the multiplex detachment label of the PW (MPLS label) to the packet according to the destination MAC address of the user data packet before forwarding the packet. After receiving the packet from the PW, PE1 determines the VSI of the packet according to the multiplex detachment label (MPLS label) and then adds the VLAN-TAG according to the destination MAC address of the user data packet for the QinQ tunnel to forward the packet to the MTU, which will then forward the packet to the CE. Hangzhou H3C Technologies Co., Ltd. 13/27

14 S9500 L2 MPLS VPN (VPLS) Technology White Paper If CE1 and CE2 exchange data for the local CEs, the MTU will directly forward the packets between CE1 and CE2 without needing to report the packet to PE1 because of its bridge function. However, if it is the first packet or a broadcast packet whose destination MAC address is unknown, the MTU will still forward the packet via the QinQ tunnel to PE1 while broadcasting the packet via the bridge to CE2, so that PE1 can duplicate the packet and forward it to each peer CE. (2) Backup of the H-VPLS AC Since there is only a single connection link between the MTU/UPE and the PE/NPE, this solution has an obvious disadvantage: Once the AC fails, all the VPNs connected to the aggregation equipment will be disconnected. Therefore, for the two access models of H-VPLS, we need to design the backup link: In normal cases, the equipment only uses one link (master) for access purposes; however, once the VPLS system detects that the master link fails, it will start the backup link to continue to provide VPN services. For the H-VPLS using LSP access, because the LDP session is run between the UPE and the NPE, the activity status of the LDP session can be used to judge the failure or not of the master PW. For the H-VPLS using QinQ access, the STP should be run between the MTU and the PE connected to the MTU, so as to ensure that the other link will be started once the master link fails. As shown in Figure 7, the UPE detects that the U-PW with NPE1 fails, so it automatically starts the backup PW to transmit data. Suppose there is a packet whose MAC address is A in CE1, it initially reaches CE3 via the master PW. Because of the MAC address learning ability of VPLS, the MAC address will be learnt by the corresponding virtual interfaces on NPE1 and NPE3. Again because NPE3 does not know the occurrence of link switchover at the peer, it still keeps the MAC address table entry, which is obvious a mistake. For this reason, the relevant MAC addresses should be reclaimed when the UPE conducts the active/standby PW switchover. The MAC address reclamation can be implemented by use of the address reclamation message of the LDP. If there are many MAC addresses to be reclaimed, then an address reclamation message whose MAC address list is null can be directly sent, so as to clear all the MAC addresses in the VPN (except for the address table entry of the link over which the MAC address reclamation message is sent). Hangzhou H3C Technologies Co., Ltd. 14/27

15 S9500 L2 MPLS VPN (VPLS) Technology White Paper CE 1 MAC A MAC A,UPW U-PW UPE NPE 1 MAC A,PW1 PW1 PW2 CE 2 U-PW (backup) MAC A, reclamation NPE 2 PW3 NPE 3 CE 3 Figure 7 MAC address update after the active/standby PW switchover The MAC address reclamation message is sent and processed as follows: The UPE sends the MAC address reclamation message to NPE2. After processing this message, NPE2 learns the address MAC A and tells it to the backup PW before sending the message to the other peers (NPE1 and NPE3). The other peers process the received message, learn MAC A and tell it to the corresponding PWs. (3) Multi-domain VPLS service The hierarchical VPLS can also be used to create the VPLS service of a larger scale and spare the need of full-meshed connections for all the VPLS equipment in the case of a single VPLS domain or crossing multiple domains. Each fully-meshed VPLS network is connected via a single LSP tunnel and each VPLS network uses one PW to connect two domains. When more than two domains are connected, the inter-domain fully-meshed PW must be established on each edge PE. In this way, a three-layer model is created: The direct connections between the MTU and the PEs; the fully-meshed connections between PEs in the domain; and the fully-meshed connections between the inter-domain edge PEs. 3.3 Packet Frame Structure Packet Encapsulation on the AC The packet encapsulation mode on the AC is decided by the user access mode. There are two user access modes: VLAN access and Ethernet access. They are defined as follows: Hangzhou H3C Technologies Co., Ltd. 15/27

16 S9500 L2 MPLS VPN (VPLS) Technology White Paper VLAN access: The header of the Ethernet frame sent upward from the CE or sent downward from the PE carries a VLAN Tag, which is a service delimiter designed by the ISP to differentiate users. We call this TAG the P-Tag. Ethernet access: The header of the Ethernet frame sent upward from the CE or sent downward from the PE does not carry any service delimiter. If the frame header contains a VLAN Tag, it indicates that the tag is only the internal VLAN Tag of the user packet and is insignificant to the PE equipment. We cal this kind of user s internal VLAN Tag the U-Tag. The VSI access means of the user can be specified through configuration Packet Encapsulation on the PW There are also two packet encapsulation modes on the PW: Raw mode and Tagged mode. In the Raw mode, the P-Tag is not transmitted on the PW: For the uplink packet on the CE side, if a packet with a service delimiter is received, the service delimiter will be removed first before the packet is sent upward, attached with two layers of MPLS labels and then forwarded; or if a packet without service delimiter is received, the packet will be directly sent upward and then attached with two layers of MPLS labels before being forwarded. For the downlink packet on the PE side, the packet will be added or not added (depending on the specific configurations) with a service delimiter before being forwarded to the CE but it is not allowed to rewrite or remove any existing tag. In the Tagged mode, the P-Tag must be carried on the frame transmitted on the PW: For the uplink packet on the CE side, if a packet with a service delimiter is received, the service delimiter will not be removed but the packet will be directly sent and then attached with two layers of MPLS labels before it is forwarded; or if a packet without service delimiter is received, the packet will be added with a null tag first before it is sent upward, attached with two layers of MPLS labels and then forwarded. For the downlink packet on the PE side, the service delimiter will be rewritten, removed or retained (depending on the specific configurations) before the packet is forwarded to the CE. The protocol [2] stipulates that the tagged mode applies by default. Hangzhou H3C Technologies Co., Ltd. 16/27

17 S9500 L2 MPLS VPN (VPLS) Technology White Paper VPLS Packet and Encapsulation As shown in Figure 8, Figure 9, Figure 10, Figure 11, the green arrows show the encapsulation of the user packets not carrying a private network VLAN tag among the devices playing different VPLS roles, while the purple arrows show the encapsulation of the user packets carrying a private network VLAN tag among the devices playing different VPLS roles. In addition, the encapsulation format between the PEs (on the PWs) shown in the figure is given without considering the outer-layer tunnel label PHP operation. If the operation is taken into account, then the packet encapsulation on the PWs may be a single-layer MPLS label (inner-layer label). Figure 8 Link packet encapsulation in the Raw mode via Ethernet access Hangzhou H3C Technologies Co., Ltd. 17/27

18 S9500 L2 MPLS VPN (VPLS) Technology White Paper Figure 9 Link packet encapsulation in the Tagged mode via Ethernet access Figure 10 Link packet encapsulation in the Raw mode via VLAN access Hangzhou H3C Technologies Co., Ltd. 18/27

19 S9500 L2 MPLS VPN (VPLS) Technology White Paper CE1 MAC P-TAG IP header Data PE1 MAC P-TAG U-TAG IP header Data MAC label1 label2 MAC P-TAG IP header Data MAC label1 label2 MAC P-TAG U-TAG IP header Data PE2 MAC P-TAG IP header Data CE2 MAC P-TAG U-TAG IP header Data Figure 11 Link packet encapsulation in the Tagged mode via VLAN access 3.4 Processing of User Data in the Entire Network Processing of Common L2 and L3 User Data in the Entire Network According to the characteristics of VPLS services, the common L2 and L3 user data will be transparently transmitted to the peer end, including the MAC header of the user packet and the private VLAN tag of the user. For the unicast packet with a known MAC address from the PE, the system will transparently transmit the packet to the corresponding CE. For an unknown unicast, multicast or broadcast packet of the user, the system will broadcast it in the entire VPLS domain, that is, all the CEs will receive the packet. For an L3 packet of the user, the VPLS system will forward it based on the L2 header of the packet without caring about the content of the L3 packet Processing of User Protocol Packet Data in the Entire Network Since the intermediate P equipment forwards packets only based on the outer-layer MPLS label without caring about whether the packet is a common packet or a protocol packet, all the L2 and L3 protocol packets of the user will be transparently transmitted by the VPLS system. The protocol packet of the private network will not interact with the protocol of the VPLS system. They are independent from each other. The private Hangzhou H3C Technologies Co., Ltd. 19/27

20 S9500 L2 MPLS VPN (VPLS) Technology White Paper network protocol data will not affect the public network protocol. For the user protocol packet whose destination MAC address is a unicast MAC address, the system will transparently transmit the packet to the corresponding CE. For the user protocol packet whose destination MAC address is a multicast or broadcast packet, the system will broadcast it in the entire VPLS domain and all the CEs will receive the protocol packet. 4 Networking Overview 4.1 Key Points of VPLS Networking (1) Logic fully-meshed connections among the PEs Fully-meshed PWs must be set up among all the PEs for the VPLS basic networking. Fully-meshed PWs must be set up among the NPEs for the H-VPLS networking. (2) Correct configuration of user access modes and access ports The access modes of all the VPLS instances at the access port must be consistent. In the VLAN access mode, the uplink packets of the user must carry P-TAG, the access port should be configured as Trunk, and the corresponding VLAN of the connected VPN shall be allowed to pass. In the Ethernet access mode, the uplink packets of the user cannot carry P-TAG but can carry the user private network tag, the access port should be configured as Access, and the QinQ function of the port should be enabled. (3) Correct UPE configuration in the H-VPLS networking Hangzhou H3C Technologies Co., Ltd. 20/27

21 S9500 L2 MPLS VPN (VPLS) Technology White Paper The roles of UPE and NPE must be made clear. Incorrect configurations will cause loops in the VPLS domain. The UPE is allowed to access one NPE only. When there are active and standby links, it can access two NPEs. The NPE can access multiple UPEs. When the UPE accesses the NPE via QinQ, the access mode of the corresponding instances on the NPE should be VLAN access. When there is link backup, the STP needs to be enabled between the UPE and two NPEs to backup the links. When the UPE accesses the NPE via LSP, the UPE can access the NPE in the VLL or VPLS mode, and it should be specified at the NPE that the access equipment be UPE. If there are the active and the standby PWs, it is necessary to specify the active/standby relation of the NPEs. The role definitions of UPE and NPE are only within a certain VPLS instance. 4.2 Typical VPLS Networking Example See Figure 12 : Figure 12 Network diagram for typical VPLS networking 5 Features of H3C S9500 S9500 series switches support the basic VPLS networking and H-VPLS networking. For the H-VPLS networking, they support multiple access modes: QinQ, VPLS and VLL. The VPLS service feature board is adopted in the S9500 for centralized Hangzhou H3C Technologies Co., Ltd. 21/27

22 S9500 L2 MPLS VPN (VPLS) Technology White Paper processing of VPLS services. The S9500 supports comprehensive feature management of VPLS instances and provides a good VPLS solution. 5.1 Features of H3C S9500 VPLS Complete H-VPLS Solution The S9500 fully supports the H-VPLS solution proposed in the draft. Its VPLS service access network can be a common Ethernet or an MPLS edge network. In addition, the access networks where multiple sites of a VPLS instance are located are independent from each other. The local Ethernet access network can interwork with the peer MPLS edge access network Feature Configuration and Processing of VPLS Instances To facilitate VSI maintenance and management, a series of features can be supported for each VPN, such as VSI traffic limit, broadcast traffic limit, MAC address quantity limit and QoS class. VSI traffic limit refers to the maximum traffic that the VPN can access on the PE. Once the user traffic exceeds this limit, the user packets will be discarded. To limit the L2 broadcast packets of the VPN, the user is allowed to specify the broadcast traffic limit. VSI broadcast traffic limit refers to the percentage of the broadcast traffic in the VPN to the maximum VPN traffic on the PE. Once the broadcast traffic exceeds the broadcast suppression percentage of the VSI traffic limit, the user s broadcast packets will be discarded. Since the forwarding table entry resources of the system are limited, it is necessary to limit the MAC address quantity of VSI. Once the number of hosts in each VSI exceeds the MAC address quantity limit of the VSI, the VPLS system will no longer learn MAC forwarding table entries. The operator can configure a proper MAC address limit value for the VPN users, so as to ensure that they can run internal private network services normally. In some exceptional cases (such as the user equipment is infected by viruses or there is MAC address attack), the system resources can be prevented from being used up by the VSI. When forwarding user packets, the VPLS system can classify the packets according their priorities to ensure that some important data are preferably forwarded to the peer in the public network. The system will map the priority information in the L2 packet Hangzhou H3C Technologies Co., Ltd. 22/27

23 S9500 L2 MPLS VPN (VPLS) Technology White Paper header of the user into the QoS priority for transmission over the public network (mapping it into a tunnel transmission priority). Generally the MPLS network that supports traffic engineering should be applied. There is a table of mapping from IEEE 802.1Q COS to tunnel EXP in the relevant protocol H-VPLS AC Backup in the ME Mode The S9500 supports the backup of the AC from the MPLS edge network to the H-VPLS. Since the UPE serves as a convergence device, all the VPLS services that access via the UPE will be affected once the link between the UPE and the NPE is faulty. To enhance the stability, the AC backup function can be enabled Load Balance and Service Backup The S9500 supports load balance between multiple VPLS service boards, improving the VPLS forwarding performance of the system. The S9500 also supports service backup. When a service board fails, the S9500 automatically switches the traffic of the board to a board working normally. This improves the reliability of VPLS services. Currently, VPLS supports eight even label ranges: Label 0 to Label 7. By establishing two tiers of mapping, VPLS redirects the services of VSIs to VPLS boards for processing: Mapping between VSIs and label ranges When you configure a VSI, the S9500 automatically selects for the VSI the label range with the greatest number of available labels. You can also use the command that the S9500 provides to specify a label range form the VSI. Mapping between label ranges and VPLS boards Mapping between label ranges and VPLS boards is implemented by configuring redirection on public network interfaces. The configuration command has two important parameters: one specifies the redirection rule, namely the VPLS label range; the other specifies the VPLS board. The command allows assigning label ranges to VPLS boards. A single label range cannot be assigned to more than one VPLS board. Using the above mechanisms, you can assign VSIs to label ranges evenly, and assign label ranges to VPLS boards evenly, implementing load balance between multiple VPLS service boards. When a board fails or is pulled out, the S9500 immediately selects the VPLS board Hangzhou H3C Technologies Co., Ltd. 23/27

24 S9500 L2 MPLS VPN (VPLS) Technology White Paper which is servicing the least number of VSIs to take over all the VSI services on the original board. After the original board comes back into service or a normal VPLS is substituted, the S9500 waits to see whether the board can work normally for a period of time. If so, it switches the VSI services back to the board Binding of Multiple VLANs with a Single VPLS Instance The S9500 supports binding a single VPLS instance with up to 64 local VLANs, allowing the local VLANs to communicate with each other and to communicate with the remote VLANs bound to the same VPLS instance at Layer 2. This expands the user access scope greatly. Note that the number of VLANs that can be bound with a VPLS instance may vary. Refer to the relevant specification description. Figure 13 Bind multiple VLANs with a single VPLS In the example shown in Figure 13, VLAN 10 and VLAN 11 are bound with the same VPLS instance. The following describes the data forwarding process: When an ARP request of VLAN 10 arrives at the VPLS service board, the board looks up its ARP table based on the destination IP address. As no match is found, the board broadcasts the ARP request to all remote user side networks of the same VPLS instance and to all the other local VLANs bound with the same VPLS instance (VLAN 11 in this example). When the local PE receives the ARP response from VLAN 11, it forwards it to VLAN 10. Hangzhou H3C Technologies Co., Ltd. 24/27

25 S9500 L2 MPLS VPN (VPLS) Technology White Paper 5.2 Processing Flow of H3C S9500 VPLS The user-side packets access via the common interface board of S9500. The system will send upward all the data in the private VLAN to which the VPLS service is bound to the VPLS service board for centralized processing. After learning the MAC address and searching for the forwarding table entry, the system will add two layers of MPLS labels to the original user packet and then forward it to the next-hop device of the peer PE. In S9500, since the VPLS packets at the PW side cannot be processed by common interface boards, the user needs to configure redirection rules on the corresponding port of the public network, so as to direct the packets to the VPLS service board for processing. At present, S9500 supports the establishment of up to 128K PWs, that is, the system can allocate 128K private network labels for PW establishment. During the VPLS service configuration, the user is required to configure a rule to redirect the MPLS packets within the private network label range corresponding to the PW on the public network port, so that the VPLS services from the PW side will be directed to the service board for centralized processing. After learning the MAC address and searching for the forwarding table entry, the system forwards the packets to the CE. When the S9500 runs VPLS, it requires a VPLS service board for centralized processing. Figure 14 illustrates the processing model. Figure 14 Diagram for VPLS processing on the S9500 Hangzhou H3C Technologies Co., Ltd. 25/27

26 S9500 L2 MPLS VPN (VPLS) Technology White Paper 5.3 VPLS-Relevant Features of H3C S9500 Supports both Martini (LDP) and Kompella VPLS. Refer to the specific specification description. Supports up to four VPLS service boards for load balance and service backup. The VPLS boards use NP boards for centralized processing and upgrade of VPLS boards can be implemented by upgrading the software. No interface is provided; inputting and outputting of traffic depends on LPUs. The VPLS service boards take responsibility for MPLS label encapsulation and MPLS forwarding, and therefore the LPUs can be standard ones or enhanced ones and do not necessarily support MPLS. Supports MAC address aging and reclamation. Supports MPLS network loop avoidance based on full mesh and split horizon. Allows private networks to run STP for loop avoidance on private networks and supports transparent transport of STP protocol messages between private networks. Supports H-VPLS and provides QinQ and LSP access between UPE and NPE. Supports VSI traffic bandwidth limiting. Supports VSI broadcast traffic limiting. Supports VSI MAC address limiting and helps in preventing MAC address attacks. Supports QoS and mapping from CoS priorities to EXP priorities. For other relevant information, refer to the related documents. 6 References Draft [1] : Draft: draft-ietf-pwe3-control-protocol-11.txt Draft [2] : RFC: RFC 4761 (Virtual Private LAN Service (VPLS) Using BGP for Auto-Discovery and Signaling.txt) RFC: RFC 4762 (Virtual Private LAN Service (VPLS) Using Label Distribution Protocol (LDP) Signaling.txt) Hangzhou H3C Technologies Co., Ltd. 26/27

27 S9500 L2 MPLS VPN (VPLS) Technology White Paper Draft: draft-ietf-l2vpn-vpls-ldp-03.txt Draft: draft-ietf-l2vpn-vpls-bgp-02.txt Copyright 2007 Hangzhou H3C Technologies Co., Ltd. All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd. The information in this document is subject to change without notice. Hangzhou H3C Technologies Co., Ltd. 27/27

28 S9500 NAT Technology White Paper H3C S9500 NAT Technology White Paper Keywords: NAT Abstract: Network Address Translation (NAT) provides a way of translating the source IP address in an IP packet header to another IP address. In practice, NAT is primarily used to allow users using private IP addresses to access the Internet. With NAT, a few public IP addresses are used by a larger number of private network hosts to solve the problem of IP addresses depletion. Acronyms: Acronym Full spelling NAT Network Address Translation Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 1/20

29 S9500 NAT Technology White Paper Table of Contents 1 Overview Introduction to NAT Related Terms Operation of NAT Single Instance Multi-Instance Application Scenarios Common POP Network Multi-ISP Network Using Policy-Based Routing Multi-Instance VPN-Public NAT Multi-Instance VPN-VPN NAT H3C S9500 Characteristics Overview Use of Network Processor Large Capacity, High Performance Support for Access to Internal Servers Support for Static Address Translation Rich ALG Features Blacklist Function Logging function Support for VPN Users Limit to the Numbers of Users and Connections Within a VPN NAT for Inter-VPN Communication NAT Operation Process of the H3C S NAT Single Instance NAT Multi-Instance...19 Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 2/20

30 S9500 NAT Technology White Paper 1 Overview As the Internet is faced with IPv4 address depletion, NAT and IPv6 were introduced to solve this problem. NAT is used based on the following fact: In private networks (such as an enterprise network), only a small number of hosts access the Internet at a specific time, and 80% traffic are limited within the network. Therefore, the hosts are assigned private IP addresses (The IANA reserves the addresses of network segments /8, /12, and /16). A private address does not need to be globally unique, and it can be used in different private networks and translated into a public address when the host using it accesses the Internet. The MPLS L3 VPN technology is widely used, especially in large enterprise networks, for it inherits the advantage of IP routing and integrates the fast forwarding and flexible networking characteristics of MPLS. MPLS L3 VPN features network structure simplification, easy maintenance, stable performance, and secure network access. Integrating NAT with MPLS L3 VPN can make a private network invisible to the outside to enhance network security, and help save operating costs by providing reusable IP addresses. NAT multiple-instance enables perfect integration of NAT and MPLS L3 VPN by allowing for access to the Internet and between VPNs through NAT, and address reuse in different VPNs. 2 Introduction to NAT 2.1 Related Terms NAT: Provides a way of translating private IP addresses into public IP addresses, allowing hosts in a private network (or a public network) to access the public network (or the private network). NAPT(Network Address and Port Translation): Network Address and Port Translation (NAPT) identifies each internal host by TCP/UDP port number or by the identifier field of ICMP packets. Unless otherwise stated, the port numbers of IP packets refer to the TCP/UDP port numbers or the identifier of ICMP packets. NAPT can better utilize IP address resources by allowing more internal hosts to access the Internet simultaneously. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 3/20

31 S9500 NAT Technology White Paper VPN: Virtual Private Network (VPN) enables construction of private networks over a shared public network by using multiple technologies, such as MPLS, tunneling and encryption. Unless otherwise stated, the term VPN refers to Layer 3 VPN (BGP/MPLS VPN) in this document. ALG: Application Layer Gateway (ALG) provides address translation for some special application layer protocol packets (such as ICMP destination unreachable packets, FTP packets, and ILS packets). These application layer protocols need to negotiate port numbers between client and server, and thus the corresponding NAT entries are created based on the negotiation results; the private IP addresses or port numbers are contained in the payload of such protocol packets. EASY IP: Uses the IP address of an interface on the router as the public IP address for translation through NAPT, to save IP address resources. FTP: The File Transfer Protocol (FTP) is used to transfer a file from a file system to another. DNS: Domain Name System (DNS) is a distributed database used by TCP/IP applications to translate domain names into IP addresses and provide related routing information. ILS: Internet Location Service (ILS) is a dynamic directory service function provided by Microsoft. Users can store and search dynamic information (such as IP address) through ILS. FIB: Forwarding Information Base (FIB) stores the core data for Layer 3 packet (IP packet) forwarding. ARP: The Address Resolution Protocol (ARP) is used to resolve an IP address into a MAC address. NP: A Network Processor (NP) is a programmable, high-performance network processor for handling packets. 2.2 Operation of NAT Single Instance 1. NAT NAT only translates IP addresses, as shown in 错误! 未找到引用源. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 4/20

32 S9500 NAT Technology White Paper Figure 1 NAT operation NAT operates as follows: (1) The NAT device receives a packet from the private host to the public host. (2) The NAT device selects an unused public address from its address pool and establishes corresponding NAT entries (both inbound and outbound). (3) The NAT device uses the outbound NAT entry to translate the source private IP address to the public address and sends the packet to the public host. (4) After receiving a response packet from the public host, the NAT device uses the inbound NAT entry to translate the destination public IP address to the private address and sends the packet to the private host. Note that NAT cannot solve IP address depletion effectively, and is not commonly adopted in practice. 2. NAPT NAPT translates both IP addresses and port numbers (or the identifier field of ICMP messages) and can better utilize IP address resources, allowing more internal hosts to access the Internet simultaneously. NAPT does not support non-tcp/udp/icmp packets. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 5/20

33 S9500 NAT Technology White Paper Figure 2 NAPT process As shown in the above figure, NAPT operates as follows: (1) The NAT device receives a packet from the private host to the public host. (2) If the connection is new, the NAT device selects an unused IP address and a port number from its address pool, and then creates corresponding NAT entries (both outbound and inbound). (3) The NAT device uses the outbound entry to translate the source private IP address and port number to the public ones and sends the packet to the public host. (4) After receiving a response packet from the public host, the NAT device uses the inbound NAPT entry to translate the destination IP address and port number to the private ones and sends the packet to the private host. 3. NAPT internal server Normally, public hosts have no permission to access most private hosts, but they may need to access some internal servers. The problem is that NAPT entries cannot be dynamically generated when public hosts initiate connections to internal servers. To solve this problem, you can configure NAT internal servers on the NAPT device, that is, to configure mappings between public IP addresses/port numbers and private IP addresses/port numbers. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 6/20

34 S9500 NAT Technology White Paper Figure 3 Operation of NAT internal server As shown in the above figure, NAPT with an internal server configured operates as follows: (1) The NAT device receives a packet from the public host to the internal server. (2) The NAT device uses the outbound NAPT entry to translate the destination public IP address and port number to the private ones and sends the packet to the internal server. (3) After receiving a response from the internal server, the NAT device uses the inbound NAPT entry to translate the source private IP address and port number to the public ones, and sends the packet to the public host. 4. NAPT ALG Some application layer protocols need to negotiate port numbers between client and server, so that the server can initiate connections to the client using the negotiated port numbers (such as the establishment of an FTP data channel). If the NAT device knows nothing about the negotiation process, it cannot perform translation between private IP address/port number and public IP address/port number, and thus the server and client cannot access each other. NAT ALG can solve this problem. The following takes FTP as an example to describe ALG operation. There are two FTP modes, Common FTP and Passive FTP. In Common FTP mode, the client specifies a port for the server to establish a connection. If the client resides in a private network, the NAT device needs to use ALG to generate a NAT/NAPT entry through which the server can access the client. In Passive FTP mode, the server specifies a port for the client to establish a connection. If the server resides in a private network, the NAT device also needs to use ALG for the client to access the server. When a private client wants to access a public server in Passive FTP mode, or when a public client wants to access a private server in Common FTP mode, the connection is initiated from the private network and thus ALG need not be used. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 7/20

35 S9500 NAT Technology White Paper Common FTP If a private FTP client wants to access a public FTP server, two TCP connections need to be established. One is the control connection (TCP port number 21 on the server) which is used to forward control information, such as commands and parameters; the other one is the data connection (TCP port number 20 on the server) which is used to transmit files. Figure 4 Common FTP mode The client notifies its port number and IP address through the PORT command to the FTP server over a control connection, and then the server initiates a TCP data connection at port 20 to the specified IP address and port. To allow the public server to access the private client, the corresponding NAT entries need to be created on the NAT device. To do so, the NAT device monitors the control flow between the client and the server. It uses the private IP address and port number in the received PORT command to create NAT/NAPT entries, and replaces them with the corresponding public ones in the PORT command. Passive FTP In Passive FTP mode, both the control and data connections are initiated by the client. The client sends a PASV request through the control channel to tell the server that it will use the Passive FTP mode. Then, the client uses a port above 1023 to transmit data to the server at a port dynamically assigned, which may not be 20. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 8/20

36 S9500 NAT Technology White Paper Figure 5 Passive FTP mode In the PASV request, the client notifies the server to use a specified data port (not the default data port). Then, the server sends a response containing the port number and its IP address, and waits for the client to initiate a connection. The PASV response sent from the server is: 227 Entering Passive Mode. A1,A2,A3,A4,a1,a2 In this message, 227 represents the PASV response code; A1,A2,A3,A4 represents the server IP address; (a1*256 +a2) is the port number of the server, which has the same format as that of the PORT command. If the server resides in a private network, the NAPT entries need to be created on the NAT device for the public client to access the private server. To do so, the NAT device uses the private IP address and port number in the PASV response received from the server to create the corresponding NAT/NAPT entries, and replace the private ones in the PASV response with the public ones through the entry Multi-Instance 1. NAT NAT multi-instance extends NAT single-instance to support VPN address translation, ensuring the same private IP addresses used in different VPNs are translated into different public IP addresses. See Figure 6 for the translation process of NAT multiinstance. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 9/20

37 S9500 NAT Technology White Paper Figure 6 NAT process NAT multi-instance operates as follows: (1) As shown in the above figure, the NAT device receives a packet from a private host to a public host. (2) If it is the first time that the private host accesses the public network, the NAT device selects an unused public IP address from its address pool and establishes corresponding NAT entries (both inbound and outbound), containing the VPN name, source private IP address, and assigned public IP address. (3) The NAT device uses the outbound NAT entry to translate the source private IP address into the public one and sends the packet to the public host. (4) After receiving a response from the public host, the NAT device uses the inbound NAT entry to translate the destination public IP address into the private one and forwards the packet to the private host in the corresponding VPN. 2. NAPT NAPT multi-instance extends NAPT single-instance to support VPN address translation. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 10/20

38 S9500 NAT Technology White Paper Figure 7 NAPT NAPT multi-instance operates as follows: (1) As shown in the above figure, the NAT device receives a packet from a private host to a public host. (2) If it is a new connection from the private network, the NAT device selects an unused IP address and a port number from its address pool, and then creates corresponding NAT entries (both outbound and inbound), containing the VPN name, private IP address/port number, and assigned public IP address/port number. (3) The NAT device uses the outbound NAPT entry to translate the private IP address and port number into public ones and sends the packet to the public host. (4) After receiving a response packet from the public host, the NAT device uses the inbound NAPT entry to translate the destination public IP address and port number into private ones and forwards the packet to the private host in the corresponding VPN. 3. NAPT multi-instance internal server The internal servers in NAPT multi-instance provide VPN support at the private network side. The operation process is similar to that of the single instance. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 11/20

39 S9500 NAT Technology White Paper Figure 8 Operation of NAT internal server NAPT multi-instance with a internal server configured operates as follows: (1) As shown in the above figure, the NAT device receives a packet from the public host to the internal server. (2) The NAT device uses the outbound NAPT entry to translate the destination public IP address and port number to the private ones and sends the packet to the internal server in the corresponding VPN. (3) After receiving a response from the internal server, the NAT device uses the inbound NAPT entry to translate the source private IP address and port number to the public ones, and sends the packet to the public host. 4. NAPT multi-instance ALG NAT multi-instance ALG extends NAT single-instance ALG to support VPN. 3 Application Scenarios 3.1 Common POP Network The fast expansion of the Internet results in shortage of IPv4 addresses. Therefore, NAT is used on high-end routers and core switches in large-sized enterprise and metropolitan-area networks, to facilitate network maintenance and management. 3.1 Figure 9 shows a common point of presence (POP) network. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 12/20

40 S9500 NAT Technology White Paper Figure 9 Common POP network 3.2 Multi-ISP Network Using Policy-Based Routing A private network may connect to multiple ISPs, as shown in 错误! 未找到引用源. With policy-based routing (PBR) configured on the NAT device, hosts in network /24 can access the Internet through ISP 1, and hosts in network /24 can access the Internet through ISP 2. Configure address pool 1 ( through ) and address pool 2 ( through ) on the NAT device. Address pool 1 belongs to ISP 1 and address pool 2 belongs to ISP 2. When accessing the Internet, hosts in network /24 use the IP addresses in address pool 1, and hosts in network /24 use the IP addresses in address pool 2. Thus, hosts in different private network segments can access the Internet through different ISPs, and can be provided with differentiated services. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 13/20

41 S9500 NAT Technology White Paper Internat ISP 1 ISP 2 NAT 1 Address Group ~ NAT 2 Address Group ~ / /24 Figure 10 Multi-ISP network using policy-based routing 3.3 Multi-Instance VPN-Public NAT As shown in the following figure, each PE has its own address pool for NAT translation and supports MPLS encapsulation. This networking mode mainly applies to enterprises, where users can assign IP addresses independently. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 14/20

42 S9500 NAT Technology White Paper Figure 11 Multi-instance NAT The hosts of each VPN can access the Internet through NAT configured on the local PE. When receiving a packet from a CE, the corresponding PE matches it against the configured ACL to determine whether it is destined to the Internet. If so, the PE translates the source IP address, adds a public MPLS tag, and sends the packet out. If the packet is destined to a host in another site of the same VPN, the PE encapsulates the corresponding private and public tags and sends it out. In this way, hosts in different sites of a VPN can access each other and the Internet over a common link without any interference. 3.4 Multi-Instance VPN-VPN NAT The same private IP addresses can be used in different branches of a government network or an enterprise network. Besides accessing the Internet, VPN users may need to access an authorized server that is usually placed in a VPN for security. Other VPNs use RT to control the access to the server. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 15/20

43 S9500 NAT Technology White Paper Figure 12 Multi-instance NAT As shown in Figure 12, hosts in VPN 1 and VPN 2 need to access the Internet, as well as a public server in the VPN named Server. To implement this application, configure NAT on the PEs connected to VPN 1 and VPN 2 respectively, and configure ACL rules to achieve address translation for packets from VPN 1 and VPN 2 to the Internet and VPN Server. Communication between different sites of VPN 1 is enabled through Layer 3 forwarding. 4 H3C S9500 Characteristics 4.1 Overview Use of Network Processor The S9500 uses NAT boards to implement NAT functions. Because one S9500 can have multiple NAT boards, you need to specify the NAT board number when configuring a NAT entry or an internal server. A NP is used as the core packet processing chip on a NAT board. The NP is programmable and scalable to provide flexible services. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 16/20

44 S9500 NAT Technology White Paper Large Capacity, High Performance Since the high-performance NP is used to process data packets, NAT of the S9500 features large NAT table capacity and powerful processing capabilities. The NAT table can accommodate a maximum of 1.2 M NAT entries, the rate of link setup can reach 150 Kpps, and the bidirectional packet forwarding rate through NAT can reach 3.0 Mpps. Suppose each packet is 64 bytes long, and the bidirectional translation rate can reach 1.5 Gbps. Because NP is on a per packet basis, longer packets can promote forwarding performance Support for Access to Internal Servers Through configuring an internal server (a mapping between private IP address/port number and public IP address/port number) on the NAT device, you can allow public hosts to access the internal server in a private network. Additionally, the H3C S9500 supports the AnyServer feature, which enables public hosts to access any port of a protocol on the internal server (ICMP does not use ports). This helps simplify internal server configuration on the NAT device Support for Static Address Translation Through static address translation, a private address can be mapped to a fixed public address. Thus, hosts in the private network can access public networks using a fixed public address. In addition, static address translation supports point-to-point applications by enabling a public host to directly access a private host Rich ALG Features S9500 uses software to implement NAT ALG for packets. The S9500 NAT ALG functions support FTP, TFTP, DNS, ICMP time exceeded/unreachable messages, LDAP, MSN Messenger 7.0 voice/video, and other commonly used application software Blacklist Function To prevent a private host from excessively occupying public network bandwidth, you can limit its total network connections using a NAT blacklist based on link setup rate, Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 17/20

45 S9500 NAT Technology White Paper number of connections, or both. To limit the number of connections, you can set a threshold value. Then, if the number of connections established from a user exceeds this value, the user is added into the blacklist and cannot establish new connections. When the existing NAT entries of the user have aged out, the NAT device waits 30 seconds and then removes the user from the blacklist to allow the user to establish new connections. To limit the link setup rate, the token bucket in the standard single-rate color-blind mode is adopted. If a private host s link setup rate exceeds the CIR, it is added into the blacklist and cannot establish new connections. The user is removed from the blacklist and can establish new connections when the link setup rate decreases to a value that makes enough tokens available in the token bucket Logging function NAT entries can be logged to a server when they are established and aged out and when they exceeds the specified active time. NAT logging configuration items include enabling of logging, log version, source and destination IP addresses, source and destination port numbers, and logging mode (flow-begin, logs sending interval for active flows). When enabling logging, you can specify a configured ACL to determine which packets needs to be logged Support for VPN Users Traditionally, when two VPNs use the same public IP address to access the Internet through NAT, address conflicts will occur and packets returned from the public network cannot be sent to the correct VPN. The S9500 NAT multi-instance feature adds VPN information into NAT entries, allowing multiple VPNs to access the Internet through a common NAT device without affecting each other. MPLS and IP networks are also supported to provide various networking modes for ISPs Limit to the Numbers of Users and Connections Within a VPN If multiple enterprises (VPNs) want to access the Internet through a common NAT device, you need to specify the maximum numbers of users and connections of each Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 18/20

46 S9500 NAT Technology White Paper VPN to prevent any enterprise from excessively using address resources NAT for Inter-VPN Communication In traditional MPLS VPNs, one VPN can access another VPN through RT. However, if the two VPNs use the same private IP addresses, address conflicts will occur, and therefore inter-vpn communication cannot be implemented only by using RT. To satisfy this requirement, the H3C S9500 can translate VPN private IP addresses into the IP addresses in the NAT address pool. 4.2 NAT Operation Process of the H3C S NAT Single Instance 1. Outbound operation process (1) Look up the NAT entries. If a match is found, go to Step 3. (2) Match the packet against the configured ACL to determine whether to perform NAT. If not, the packet is forwarded; if yes, select an address from the address pool. A port is also selected for NAPT translation. (3) Translate the source IP address and port number of the packet. If ALG processing is needed, the packet is processed by NAT ALG. (4) Look up the FIB table to forward the packet. 2. Inbound operation process (1) Look up the NAT entries. If a match is found, go to the next step; otherwise, the packet is discarded (2) Translate the destination IP address and port number of the packet. If ALG processing is needed, the packet is processed by NAT ALG. (3) Look up the FIB table based on the translated private IP address and forward the packet NAT Multi-Instance NAT multi-instance extends NAT single instance by supporting VPNs. When a private host accesses a public host, NAT multi-instance creates a NAT/NAPT entry, which includes the VPN information. Thus, hosts in different VPNs can use the same private IP addresses. A packet returned from the public network matches the corresponding NAT entry and is forwarded to the VPN specified in the entry. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 19/20

47 S9500 NAT Technology White Paper Copyright 2007 Hangzhou H3C Technologies Co., Ltd. All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd. The information in this document is subject to change without notice. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 20/20

48 S9500 Network Security Technology White Paper H3C S9500 Network Security Technology White Paper Keywords: Network security, threat Abstract: With the emergence of more and more network based critical services, security problems are drawing more and more attention, making network security research a hotspot in both the computer and telecommunications fields. This document describes commonly known network attacks and introduces the network security features of the H3C S9500 series. Acronyms: Acronym Full spelling DoS Denial of Service Hangzhou H3C Technologies Co., Ltd. 1/10

49 S9500 Network Security Technology White Paper Table of Contents 1 Overview Network Security Threats Definition of Network Security Threat Classification of Network Security Threats Security Threats to Network Devices Threats at Data Transport Level Threats at Signaling Level Threats at Device Management Level Security Capabilities of H3C S Security Features at Data Transport Level Defense Against Address Scanning Defense Against DoS/DDoS Attacks Broadcast/Multicast Rate Limit Defense Against MAC Address Table Capacity Attacks Support for Static MAC Address Entries and ARP Entries Powerful ACL Capabilities Security Features at Signaling Level Defense Against ARP Attacks Address Conflicts Detection Defense Against TC/TCN Attacks Defense Against Address Embezzlement Defense Against Routing Protocol Attacks Security Features at Device Management Level Support for User Levels Secure Remote Management Security Auditing Access Control SFTP Service...10 Hangzhou H3C Technologies Co., Ltd. 2/10

50 S9500 Network Security Technology White Paper 1 Overview With the evolvement of Internet technologies and the explosive growth of the Internet scale, Internet applications, starting from science research fields, have now reached every walk of life. More and more network based critical services are emerging and networks have become the new drive for improvement of productivity and life quality. However, the Internet is based on IP, and therefore has inherent problems such as security, quality of service, and operation mode, with security the most outstanding and important problem. In addition, the openness of IP networks makes the security problem even more complicated. While the simplicity and openness of IP networks boost the rapid development of the Internet, they also result in security vulnerabilities. Meanwhile, with the development of technologies and the acceleration of information delivery, technical difficulty in launching attacks to IP networks is falling and attack tools are becoming more automatic, enabling more people to launch attacks. The number of network attack events increases every year and the resulting economic cost is becoming higher and higher. Network security threats are not only disturbing corporations, but endangering the national information security, casting a shadow on the development of the Internet. 2 Network Security Threats 2.1 Definition of Network Security Threat Network security threat refers to destruction and unauthorized access and modification of data that is saved or transferred on networks, servers, and desktops. Network security threats are usually implemented by specific technologies or tools and are challenges to network security. 2.2 Classification of Network Security Threats Security threats on IP networks fall into two categories: those to the security of hosts (including user s hosts and application servers) and those to the security of networks, mainly network devices such as routers and switches. The former generally attack specific operating systems, primarily the Windows systems. Examples include viruses Hangzhou H3C Technologies Co., Ltd. 3/10

51 S9500 Network Security Technology White Paper and Trojan horses. The latter mainly attack TCP/IP protocols. This white paper discusses the latter, namely security threats to network devices. 2.3 Security Threats to Network Devices Network devices provide three levels of functions: data transport level, signaling level, and network device management level. According, this section describes security threats to network devices in these three levels Threats at Data Transport Level The network data transport level is responsible for processing and forwarding of data entering a network device. Functions of this level may be affected by two types of attacks: Attacks based on high traffic or abnormal packets, which are intended to consume large quantity of CPU resources so that normal traffic cannot be serviced. User data targeted attacks, which compromise the confidentiality and integrity of user data by sniffing, tampering, or deleting user data Threats at Signaling Level The signaling level maintains operation of network protocols to control routing and switching of packets. Routing information sniffing and IP address forging are the main threats at this level. These threats may cause routing information leakage and abuse Threats at Device Management Level The device management level supports remote management of network devices. Threats at this level come from two aspects: one is the vulnerabilities of the protocols (such as Telnet and HTTP) for device management, and the other is management defects such as the leakage of a management account. 3 Security Capabilities of H3C S9500 The H3C S9500 series are high-end routing switches that are based on the Comware software platform. They not only hold all the security features of the Comware platform, but also incorporate some other security features. Hangzhou H3C Technologies Co., Ltd. 4/10

52 S9500 Network Security Technology White Paper 3.1 Security Features at Data Transport Level Defense Against Address Scanning When launching an address scanning attack, an attacker sends to a target network a large quantity of IP packets with different destination IP addresses. In this case, the network device connecting the target network has to send a great deal of ARP packets for delivering of the attack packets. If no host is present with the destination address of an attack packet, the network device has to send destination unreachable notifications as well. When the target network has many hosts and the attack packets are in great quantities, the CPU and memory resources of the network device may be depleted, resulting in network service interruption. The H3C S9500 series support defense against address scanning attacks. When an H3C S9500 routing switch receives a packet destined for one of its directly connected network segment, it checks whether an ARP entry is present for the destination address. If not, it sends an ARP request and adds a drop entry for the destination address to prevent subsequent packets to the address from impacting the CPU. If it receives a response to the ARP request later, it removes the dropping entry and adds an ARP entry. A drop entry expires after a specified period of time. This mechanism can effectively block attack packets while allowing normal traffic. The H3C S9500 series provide some configuration commands to enable/disable defense against address scanning attacks Defense Against DoS/DDoS Attacks During a denial of service (DoS) attack, an attacker sends large amounts of connection requests to the target device to deplete the resources of the device, making the device unable to function normally or even go down. DoS attacks usually aim at servers, preventing servers from providing services for legal users. That is why the attacks are called DoS attacks. Distributed denial of service (DDoS) is an upgraded version of DoS. A DDoS attack can compromise multiple devices at the same time and is more destructive in a greater range. The H3C S9500 series can better defend themselves against common DoS and Hangzhou H3C Technologies Co., Ltd. 5/10

53 S9500 Network Security Technology White Paper DDoS attacks such as Spoofing, Land, and Smurf, ensuring that when some protocol is compromised, the others can function normally. Besides, when a server behind an H3C S9500 routing switch is targeted by a DoS attack, the switch can assign specific ACL rules to filter attack packets, so as to ensure that the connected server and hosts can work normally Broadcast/Multicast Rate Limit Broadcast and multicast packets in great quantities can consume a great deal of network bandwidth and therefore degrade the forwarding performance of network devices. When a loop exists on a network, broadcast and multicast packets may even bring the network down. The H3C S9500 series have powerful broadcast/multicast packet filtering functions. Using these functions, you can set an absolute broadcast/multicast rate limit for a port or a limit on the broadcast/multicast rate percentage. You can also configure ACL rules to limit the rates at which broadcast packets, multicast packets, and unknown unicast packets can pass a port Defense Against MAC Address Table Capacity Attacks A MAC address table capacity attack sends a great deal of frames with different, forged source MAC addresses to a target device, making the device learn a lot of useless MAC addresses. As the capacity of a MAC address table is limited, the device may not be able to learn MAC addresses of legal users normally. During Layer 2 forwarding, the attack packets may be broadcasted in the VLAN, wasting a lot of bandwidth and impacting the hosts connected to the network device. With the H3C S9500 series, you can set the maximum number of MAC addresses that a port or VLAN can learn based on the number of hosts connected to the port or VLAN, preventing a port or VLAN from using up all the MAC address table resources. When setting the MAC address limit, you can also specify whether the device should forward packets with unknown source MAC addresses when the limit is reached. This allows you to prevent too much broadcast traffic in a VLAN from impacting other devices Support for Static MAC Address Entries and ARP Entries The H3C S9500 series support static MAC address entries and static ARP entries. By Hangzhou H3C Technologies Co., Ltd. 6/10

54 S9500 Network Security Technology White Paper configuring static MAC address entries, you can ensure the correct forwarding of Layer 2 frames. By configuring static ARP entries, you can bind MAC addresses with IP addresses, preventing IP addresses being embezzled Powerful ACL Capabilities In complicated network environments, there may be kinds of attack packets compromising network devices or the attached hosts. The H3C S9500 series provide powerful ACL capabilities, allowing identification, limit and filtering of packets based on the fields at data link layer, network layer, and transport layer. The ACL rules can not only be based on common fields such as ICMP, IGMP, TCP port number, UDP port number, IP address, and MAC address, but can also be based on TTL, VLAN_ID, and EXP fields. In addition, you can configure ACL rules for a device, a port, or a VLAN as required. 3.2 Security Features at Signaling Level Defense Against ARP Attacks The ARP protocol supports no authentication methods, although it is very important in data forwarding. Attackers often use forged ARP packets to launch attacks. The H3C S9500 series support defense against this kind of attack. After an H3C S9500 routing switch receives an ARP packet, it hashes the source MAC address of the packet. Besides, it counts the received ARP packets. When it detects that the CPU is dropping packets and the number of ARP packets from a MAC address exceeds the limit, it considers the host an ARP attacker and will log the event, give an alert message, and add a source MAC address drop entry to filter packets from the host Address Conflicts Detection If the interface of a network device is using the same IP address as that of a host or another network device which is connected with the interface, an address conflict exits. In this case, if the network device cannot detect the address conflict, the ARP entry for the network device on the other connected hosts may be updated to have a wrong MAC address, disabling the hosts from communicating with the network device Hangzhou H3C Technologies Co., Ltd. 7/10

55 S9500 Network Security Technology White Paper normally. The H3C S9500 series support address conflicts detection. When an H3C S9500 routing switch receives an ARP packet, it checks whether the source IP address of the packet is the same as that of the interface connecting the network segment. If yes, it sends an address conflict notification packet to tell the ARP packet sender that the IP address has been used. At the same time, it sends a gratuitous ARP broadcast packet, notifying all hosts and network devices on the segment to use the correct ARP entry for the IP address. An address conflict alert message may also be generated and logged, so that network administrators know the situation Defense Against TC/TCN Attacks With Spanning Tree Protocol (STP) enabled, if a port of a device on the network detects an STP state change, it generates a topology change (TC) or topology change notification (TCN) message. When another device on the network receives such a TC or TCN message and finds that the network topology has changed, it needs to remove the MAC address and ARP entries to avoid using the entries for data forwarding. If there are a lot of TC or TCN messages on a network, MAC address and ARP entry flushing will occur frequently and large amounts of ARP requests will then be broadcasted in the VLAN. In this case, Layer 3 packets may be dropped and the network may not be able to function normally. The H3C S9500 series can protect the network against TC/TCN attacks. Upon receiving a TC/TCN packet, an H3C S9500 routing switch removes the MAC address entries but does not remove the ARP entries. When relearning a MAC address, it checks whether there is an ARP entry for the MAC address. If so, it directly modifies the outbound port of the ARP entry. Modifying ARP entries based on MAC addresses can avoid packet dropping during Layer 3 forwarding. Frequent topology change may affect the operation stability of all devices on the network. The H3C S9500 series can deal with this situation. After receiving the first TC/TCN message, an H3C S9500 routing switch executes a series of processes accordingly and starts a timer. Before the timer expires, it does not respond to any more TC/TCN messages. Once the timer expires, it checks whether it has received any TC/TCN messages during the period. If so, it performs the flushing operation. This mechanism helps keep the devices working stably. Hangzhou H3C Technologies Co., Ltd. 8/10

56 S9500 Network Security Technology White Paper Defense Against Address Embezzlement Address embezzlement refers to the situation where an illegal user exploits the IP address of a legal user. In this case, the network device will learn a wrong ARP entry and the legal user will not be able to get online normally. The H3C S9500 series can protect users against address embezzlement attacks. With MAC address and IP address bindings configured, an H3C S9500 routing switch performs address validation when learning ARP entries and learns only legal ARP entries Defense Against Routing Protocol Attacks Routing protocol attacks send forged routing update packets to routers that do not perform routing protocol authentication, populating the routing tables with forged routes. This may even cause the networks to crash. Experienced attackers may further launch more severe attacks. The H3C S9500 series support routing protocol authentication: (1) OSPF: Plaintext/MD5 authentication between neighboring routers and plaintext/md5 authentication within an OSPF area. (2) IS-IS: Level-1 plaintext/md5 authentication between interfaces, Level-2 plaintext/md5 authentication of interfaces, plaintext/md5 authentication within an IS-IS area, plaintext/md5 authentication in an IS-IS routing domain. (3) BGP: MD5 authentication between neighboring routers and within a BGP area. (4) RIPv2: Plaintext/MD5 authentication between neighboring routers. 3.3 Security Features at Device Management Level Support for User Levels The H3C S9500 series provide four user levels (visit, monitor, system, and manage) and support encryption of user passwords and limit of password attempts. If a user cannot enter the correct password before the limit is reached, the device will give an alert message Secure Remote Management The H3C S9500 series support the SSH protocol. Network administrators can log in Hangzhou H3C Technologies Co., Ltd. 9/10

57 S9500 Network Security Technology White Paper to a network device by SSH securely Security Auditing The H3C S9500 series provide basic security auditing functions including security alarm logging and user operation logging Access Control The H3C S9500 series support 802.1x authentication in port-based mode and MACbased mode, guaranteeing secure LAN access SFTP Service Secure FTP (SFTP) allows users to log in to devices and perform remote file management securely. An H3C S9500 routing switch can function as an SFTP server or client. When it functions as a client, you can log in to a remote device from it to perform file management. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd. The information in this document is subject to change without notice. Hangzhou H3C Technologies Co., Ltd. 10/10

58 S9500 OSPF/IS-IS/BGP GR Technology White H3C S9500 OSPF/IS-IS/BGP GR Technology White Paper V1.00 Keywords: GR Abstract: Graceful Restart (GR) ensures continuity of packet forwarding and hence key services when the routing protocol restarts. GR is a highly reliable technology widely used in active-standby switchover and system upgrade. Acronyms: Acronym Full spelling OSPF ISIS BGP GR Open Shortest Path First Intermediate System-to-Intermediate System Border Gateway Protocol Graceful Restart

59 S9500 OSPF/IS-IS/BGP GR Technology White Table of Contents 1 Introduction Typical Networking Analysis Features Terms How GR Works OSPF GR Standard OSPF GR... 错误! 未定义书签 Compatibility mode ISIS GR BGP GR H3C S9500 Features...18

60 S9500 OSPF/IS-IS/BGP GR Technology White 1 Introduction The control plane and forwarding plane of a high-end router/switch are separate from each other. The control plan controls and manages the whole device, discovering routes and delivering routes to the interface boards. The forwarding plane is dedicated to data forwarding. The respective processors of these two planes are functionally independent. Each time the control plane restarts, all the routing protocols have to restart, the neighbor relationships between the device and the adjacent devices have to be rebuilt, and all the routing information databases have to be re-synchronized. Neighbor relationship interruption triggers route recalculation on neighbors, causing routing flaps and communication failures. To solve this problem, IETF proposed a series of enhanced protocols for different routing protocols, such as IS-IS, OSPF, BGP, and LDP respectively. With these enhanced protocols, the original protocol operating flows are improved. When the control plane restarts on a device, the device will notify its neighbors to temporarily preserve its routing information and adjacency relationships with it. After the protocol restarts, the neighbors will help the restarting device restore routing information in a very short time. During the restart, no routing flaps occur, and packet forwarding on the network remains normal. These enhanced protocols are so called Graceful Restart. GR ensures the continuity of packet forwarding and hence key services when the routing protocol restarts. GR is widely used in active-standby switchover and system upgrade. 2 Typical Networking Analysis GR generally works between neighbors, as shown in the following figure:

61 S9500 OSPF/IS-IS/BGP GR Technology White Switch B GR Helper Switch C GR Helper Switch A GR Restarter Switch D GR Helper Switch E GR Helper Figure 1 Typical GR network application When its control software restarts, Switch A starts GR and notifies its neighbors Switch B, Switch C, Switch D, and Switch E to start GR. During the GR process, Switch A finishes synchronizing routing information with its neighbors and the forwarding services remain uninterrupted. 3 Features 3.1 Terms GR Restarter A GR Restarter is a device whose control plane restarts. GR Helper A GR Helper is a neighbor device that assists the GR Restarter in synchronizing routing information during the GR process. The GR Restarter and GR Helpers must be GR-capable and perform GR capability negotiation in advance, including GR capabilities and GR time. If the negotiation succeeds, when the control plane of a GR-capable device restarts, the neighbor devices is notified to become GR Helpers and the routes of the GR Restarter remain

62 S9500 OSPF/IS-IS/BGP GR Technology White unchanged within the GR time. 3.2 How GR Works GR works with different routing protocols in the similar way, though the GR flows for respective protocols vary. In the following figure, the solid lines indicate that adjacencies are formed between Switch A and Switch B, and between Switch A and Switch C, while the dotted lines indicate that Switch A, Switch B, and Switch C are GR-capable and GR capability negotiation has been complete among them. When its control plane restarts, Switch A begins to work as a GR Restarter and its forwarding plane remains normal. Switch B and Switch C begin to work as GR Helpers, with the routes of the GR Restarter unchanged. Then, the GR Restarter (Switch A) reestablishes neighbor relationships with the two GR Helpers (Switch B and Switch C) and receives routing information from them. When the GR Restarter finishes receiving all the routing information, it calculates the routes and synchronizes the calculation results to the forwarding plane. After that, the GR process is complete. Switch B GR Helper Switch A GR Restarter Switch C GR Helper Figure 2 Typical GR networking This GR process is generic to routing protocols. GR processing details vary with routing protocols. The following sections describe the GR processing mechanisms for OSPF, IS-IS, and BGP respectively. 3.3 OSPF GR OSPF GR has two modes, standard mode and compatible modes.

63 S9500 OSPF/IS-IS/BGP GR Technology White Standard OSPF GR 1. Packet format Standard OSPF GR uses an Opaque-LSA (Type 9) to notify a neighbor device to start the GR process. Known as the Grace-LSA, the Opaque-LSA has an Opaque type of 3 and Opaque ID of 0. The following figure depicts the Grace LSA format. Figure 3 Grace LSA format The following figure shows its TLV format: 2 Bytes 2 Bytes Type Length Value Figure 4 TLV format The RFC defines three types of TLVs: Grace Period TLV The Grace Period TLV has a Type value of 1 and Length value of 4, and indicates the maximum time during which a neighbor acts as a GR Helper. If the GR Restarter has not completed the GR process before this period expires, the neighbor device stops

64 S9500 OSPF/IS-IS/BGP GR Technology White working as a GR Helper. Grace-LSAs must contain a Grace Period TLV. Graceful restart reason TLV A Graceful Restart Reason TLV has a Type value of 2 and Length value of 1, and describes the graceful restart reason. Possible values of the Value field are 0 for unknown reason, 1 for software restart, and 2 for software reloading (upgrade). Grace-LSAs must contain a Graceful Restart Reason TLV. IP interface address TLV An IP interface Address TLV has a Type value of 3 and Length value of 4, and indicates the IP address of the interface sending the Grace-LSA. This IP address uniquely identifies the restarting device on a broadcast, NBMA, or P2MP network. 2. Protocol processing flow Standard OSPF GR works as follows:

65 S9500 OSPF/IS-IS/BGP GR Technology White GR Restarter GR Helper 1 Grace-LSA ACK 2 3 HELLO 4 DD LSU ACK 5 6 Grace-LSA ACK 7 LSU Figure 5 RFC 3623 protocol processing flow 1) Once brought up again, an OSPF interface on the GR Restarter sends a Grace- LSA. 2) Upon receiving the Grace-LSA, the neighbor starts to act as a GR Helper and send an ACK to the GR Restarter. 3) Hello packets are exchanged on the broadcast or NBMA network to elect a DR and BDR. 4) The GR Restarter begins normal LSDB synchronization. The neighbor state transits from Exstart, through Exchange and Loading to Full. During this process, the GR Restarter stores received self-originated LSAs, and labels them as Stale.

66 S9500 OSPF/IS-IS/BGP GR Technology White 5) The GR Helper also begins normal LSDB synchronization. The neighbor state transits from Exstart, through Exchange and Loading to Full. During this transition process, the GR Helper operates as in the FULL state, without generating any new Router LSAs or Network LSAs. 6) When all the neighbor relationships become FULL, namely, restored, Grace-LSA flushing is initiated. 7) The GR process is complete, and new LSAs are generated and flooded. The LSAs labeled as Stale but not regenerated are flushed Compatible OSPF GR 1. Packet format Link-local Signaling (LLS) Block Compatible OSPF GR extends the OSPF packet format to carry different types of application data. The following figure shows the extended OSPF packet format: IP Header Length = HL+X+Y+Z OSPF Header Length = X OSPF Data Header Length X Authentication Data Length = Y LLS Block Length =Z Y Z Figure 6 Extended OSPF packet format The authentication data and LLS block fields are not included in the OSPF packet length. Currently, only two types of OSPF packets, Type 1 (OSPF Hello) and Type 2 (OSPF DD), contain LLS Block, which is identified by the L Bit (0x10) in the Option field. * * D C L N / P M C E * Figure 7 Option field of OSPF packet

67 S9500 OSPF/IS-IS/BGP GR Technology White The LLS Block field adopts an extensible TLV structure, defining two types of TLVs: Extended Options TLV (EO_TLV) and Cryptographic Authentication TLV (CA_TLV), as show in the following figures. 2 Bytes 2 Bytes Type 1 Length 4 Extended Options Figure 8 EO_TLV format 2 Bytes 2 Bytes Type 2 Length 20 Sequence Number Auth Data Figure 9 CA_TLV format An EO_TLV has a Type value of 1 and a Value field with 4-byte Extended Options for Option extension in OSPF packets. OOB In traditional OSPF, LSDB resynchronization is performed only when neighbor relationships are reestablished. Normal LSDB synchronization is carried out through flooding after neighbor relationships are established. OOB (out-of-band) LSDB resynchronization is carried out in a network where neighbor relationships have been established and the network topology is stable.

68 S9500 OSPF/IS-IS/BGP GR Technology White In traditional OSPF, LSDB resynchronization requires the neighbor state machine to be in the Exstart state. This causes OSPF to generate new Type-1 LSAs (Router LSAs), triggering route recalculation. LR_Bit is introduced in the OOB flow for the OOB capability negotiation between neighbors. LR_Bit is contained in the Extended Option in an EO_TLV. If the device is OOB-capable, when sending OSPF Hello packets and DD packets, the device sets the LR_Bit in the Extended Option of the EO_TLV to 0x * * * * * * * * * * LR Figure 10 LR_Bit In addition, R_Bit is introduced in the OOB flow to notify neighbor devices to perform OOB resynchronization. R_Bit is contained in the DD packets sent to neighbors. In the DD packets, R_Bit and I/M/MS are set. This means the sender wants to start OOB resynchronization. In this case, if the neighbor state machine is FULL, the device sets the neighbor state to ExStart to start LSDB resynchronization. During OOB resynchronization, the neighbor state is treated as FULL regardless of whether the state is ExStart, Exchange, or Loading, this is, the device operates as if the neighbor is in the FULL state, and therefore Router LSAs and Network LSAs do not change, keeping the network stable. RS bit In the compatible mode, an RS_Bit is added to the Extended Option of the EO_TLV to notify the neighbor to start the GR process. The value of the RS_Bit is 0x * * * * * * * * * RS LR Figure 11 RS_Bit 2. Protocol processing flow The following figure shows the processing flow of compatible OSPF GR:

69 S9500 OSPF/IS-IS/BGP GR Technology White GR Restarter GR Helper 1 Hello LR = 1 RS = 1 Hello LR = 1 RS = DD R = 1 DD R = DD R = 1 LSU LSACK 6 Figure 12 Draft GR flow 1) Once brought up again, an OSPF interface of the GR Restarter sends a hello packet containing LLS Block. The RS bit and LR bit in the Extended Options field of the EO_TLV in the LLS Block are set. 2) Upon receiving the hello packet, the neighbor skips the two-way state, that is, it keeps the neighbor state unchanged, enters the GR Helper process flow, and sends back a hello packet with the LR bit on and RS bit off. 3) After receiving the hello packet with LR bit on, the GR Restarter sets the neighbor state to 2-way and the subsequent flow is the same as that of the traditional OSPF protocol. Once the DR election is complete, the first DD packet (with R_bit on) is sent to start the OOB flow. In the hello packet sent after the DR election, the RS_bit will not be set. 4) After receiving the DD packet with R_bit set and then setting the corresponding neighbor state to Exstart, the GR Helper also enters the OOB flow.

70 S9500 OSPF/IS-IS/BGP GR Technology White 3.4 ISIS GR 5) During LSDB resynchronization, the neighbor state transits from Exchange to Loading and to Full. During this process, the GR Restarter stores the received self-originated LSAs, and labels them as Stale. 6) When all the neighbor relationships become FULL and all the routing information is restored, the GR process is complete. LSAs are regenerated and flooded, and the LSAs labeled as Stale are not regenerated and are flushed directly. 1. Packet format In IS-IS GR, a new TLV, namely, Restart TLV, is added to IIH packets to notify the neighbor device to enter the GR flow. This new TLV has a Type value of 211. The following figure illustrates its Value field: Flags Remaining Time Restarting Neighbor ID 1 2 ID Length Figure 13 Value field of a Restart TLV The one-byte Flags field records necessary state flags during the restart. The following figure shows the Flags format: * * * * * SA RA R RRR Figure 14 Flags format Currently, only the last three bits (SA, RA, and RR) are used. When the control software restarts, the RR (Restart Request) bit of the first IIH packet sent through each interface must be set. Upon receiving the IIH packet, the neighbor device must acknowledge the receipt by sending back an IIH packet with the RA (Restart Acknowledgement) bit set. The SA (Suppress adjacency advertisement) bit is optional and used to avoid blackhole routes.

71 S9500 OSPF/IS-IS/BGP GR Technology White The 2-byte Remaining Time field indicates the time in seconds before the neighbor ages out. This field and the RA bit must be present at the same time. Upon receiving an IIH packet with RR bit set from the restarting device, the neighbor device must immediately acknowledges the receipt by sending back an IIH packet whose RA bit is set to 1. In this acknowledge packet, the time in seconds before the corresponding neighbor (restarting device) ages out is filled in the Remaining Time field. The System ID of the restarting device is filled in the Restarting Neighbor System ID field. 2. Protocol processing flow In IS-IS GR, three timers, namely, T1, T2, and T3 are defined. T1 timer: Like the IIH timer, the T1 timer is defined on each interface. It defines the interval for sending IIH packets with the RR bit set and defaults to three seconds. When the device restarts, a T1 timer is created on each interface and an interface periodically sends IIH packets with RR bit set. The T1 timer on the interface is not removed until the interface receives the IIH acknowledge packet with RA bit set and the complete CSNP packet. T2 timer: The T2 timer defines the maximum wait time of LSDB resynchronization and defaults to 60 seconds. Each LSDB has such a timer. T3 timer: The T3 timer defines the maximum restart time in IS-IS. Once the T3 timer expires, the GR process ends regardless of whether the LSDB resynchronization is complete and the normal IS-IS processing flow begins. Upon initialization, the T3 timer is set to seconds. After all interfaces receive the IIH acknowledge packets with the RA bit set, the T3 timer is reset based on the minimum among the Remaining Time values of these packets. The following figure depicts the IS-IS GR working flow:

72 S9500 OSPF/IS-IS/BGP GR Technology White GR Restarter GR Helper 1 IIH RR = 1 RA = 0 T3 Timer T2 Timer T1 Timer IIH RR = 0 RA = 1 CSN P 2 3 LSP 4 5 IIH RR = 0 RA = 0 Figure 15 IS-IS GR flow 1) When IS-IS is re-enabled on the GR Restarter, T2 and T3 timers are enabled globally. When an interface is brought up again, the T1 timer is started on the interface (Different from the original protocol flow, when the interface is up, the T1 timer, instead of the IIH timer, is started), and an IIH packet with the RR bit set is sent. 2) After receiving the IIH packet, the neighbor leaves the neighbor state of the sender unchanged, and sends back an IIH packet with the RA bit set. The IIH packet is filled with the GR Restarter s age remaining time and System ID in the Remaining Time and Restarting Neighbor System ID fields of the Restart TLV respectively. If the interface is a broadcast interface, a DIS election is performed, which is different from traditional IS-IS DIS election. If it is elected as the DIS, the interface sends CSNP packets and all LSPs. If the interface is a P2P interface, it directly sends CSNP packets and all the LSPs. 3) After receiving the IIH packet with RA bit set and all the CSNP packets, the GR Restarter removes the T1 timer. Otherwise, the GR Restarter periodically sends IIH packets with RR bit set and does not remove the T1 timer until it has received the IIH packet with RA bit set and complete CSNP packets or when the maximum number of T1 timer timeouts is reached.

73 S9500 OSPF/IS-IS/BGP GR Technology White 3.5 BGP GR 4) Once the GR Restarter finds that the LSDB resynchronization at a level is complete, it removes the T2 timer of the level. 5) After removing all the T2 timers, the GR Restarter removes the T3 timer and enters the normal IS-IS flow. 1. Packet format Graceful Restart Capability BGP GR defines a new BGP capability which is known as the Graceful Restart capability and has a capability value of 64. The following figure shows is Value field. Figure 16 BGP GR Capability Value The R bit identifies Restart State. When it is set to 1, it means the sender is restarting and the receiver can send routing information without needing to receive the End-of- RIB marker from the sender. This prevents locking when multiple BGP speakers await the End-of-RIB marker. The Restart Time means the maximum route holdtime after the peer is found down. The <AFI, SAFI, Flags for address family> fields indicate which network address

74 S9500 OSPF/IS-IS/BGP GR Technology White families the GR feature supports. GR can support IPv4 and IPv6 at the same time. In BGP GR, an End-of-RIB marker is defined to speed up the BGP GR process. The Update message with both reachable NLRI and withdrawn NLRI as null is designated as the End-of-RIB marker. After a BGP connection is established, this marker can notify the peer that its initial notification is complete Protocol processing flow The following figure depicts the BGP GR working flow: Switch A Switch B BGP Open,GR64,AF IPv4 BGP Open,GR64,AF IPv4 BGP Open,GR64,AF IPv4 R=1, Time=180 BGP session OK BGP Update message End of RIB(IPv4) BGP Update message Figure 17 BGP GR flow

75 S9500 OSPF/IS-IS/BGP GR Technology White 1) Switch A sends an Open message containing the IPv4 GR capability to the neighbor. 2) The Open message sent by Switch B also contains the IPv4 GR capability. 3) Switch A restarts, and sends an Open message with R_bit set to request Switch B to start the GR Helper processing flow; the maximum route holdtime in the message is 180 seconds. 4) Upon receiving the Open message, Switch B starts the GR Helper processing flow, labels all the IPv4 routes received from Switch A before as Stale, and holds the routes for 180 seconds before deletion. Other flows are the same as those of traditional BGP. 5) Optimal route selection is not performed during this process. 6) After sending all the Update messages, Switch B sends an End-of-RIB to notify update completion. 7) After receiving all the routing information, Switch A performs optimal route selection and resends Update messages to notify neighbor devise to update routes. 8) Switch B deletes the routes labeled as stale. 4 H3C S9500 GR Characteristics The routing protocols running on the S9500 series switches have rich GR features that allow excellent fault tolerance and compatibility. Each protocol can interoperate with devices of other vendors. OSPF GR, in particular, supports both standard and compatible modes and therefore is scalable in GR networking. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. All rights reserved. No part of this manual may be reproduced or transmitted in any form or by any means without prior written consent of Hangzhou H3C Technologies Co., Ltd. The information in this document is subject to change without notice.

76 S9500 QoS Technology White Paper H3C S9500 QoS Technology White Paper Key words: QoS, quality of service Abstract: The Ethernet technology is widely applied currently. At present, Ethernet is the leading technology in various independent local area networks (LANs), and many Ethernet LANs have been part of the Internet. With the development of the Ethernet technology, most common Internet users access the Internet through Ethernet. To implement end-to-end QoS throughout the network, you must guarantee QoS for Ethernet. To do this, Ethernet switching devices must use the QoS technology to provide different QoS guarantees for different types of traffic flows, especially those traffic flows with higher demand for delay and jitter guarantees. Acronyms: Acronym Full spelling QoS Quality of Service Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 1/15

77 S9500 QoS Technology White Paper Table of Contents 1 Overview Basic Networking Structure Features Service Model Traffic Classification Traffic Policing Priority Marking Queue Scheduling Congestion Avoidance Traffic Shaping Policy Routing QoS Processing Procedure on the S9500 Series...14 Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 2/15

78 S9500 QoS Technology White Paper 1 Overview On traditional packet switching networks, switches and routers treat all packets equally and handle them using the first in first out (FIFO) policy. This service is called best-effort. It delivers packets to their destinations as possibly as it can, without any guarantee for delay and jitter. With the development of computer networks, more and more traffic such as voice, video, and critical data which is sensitive to bandwidth, delay, and jitter is transmitted over networks. This enriches the services resources on a network greatly. On the other hand, there is a higher demand for the Quality of Service (QoS) of network transmission. The Ethernet technology is widely applied currently. At present, Ethernet is the leading technology in various dependent local area networks (LANs), and many Ethernet LANs have been part of the Internet. With the development of the Ethernet technology, most common Internet users access the Internet through Ethernet. To implement end-to-end QoS throughout the network, you must guarantee QoS for Ethernet. To do this, Ethernet switching devices must use the QoS technology to provide different QoS guarantees for different types of traffic flows, especially those traffic flows with higher demand for delay and jitter guarantees. 2 Basic Networking Structure Figure 1 Basic networking structure Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 3/15

79 S9500 QoS Technology White Paper 3 Features 3.1 Service Model A service model refers to a set of end-to-end QoS functions. The simplest service model is the Best-Effort model adopting the FIFO policy. It delivers packets to their destinations as possibly as it can, without any guarantee for delay and jitter. The Diff- Serv model was introduced to implement QoS for network transmission. The Diff-Serv model is a multi-service model. It provides QoS services for each packet according to the QoS parameters specified for the packet, thus satisfying differentiated QoS demands. The Diff-Serv model is used to implement end-to-end QoS for some critical services. The S9500 series support the Diff-Serv model. 3.2 Traffic Classification To specify different QoS parameters for packets of different levels, the Diff-Serv model must classify the network traffic first. Traffic classification organizes packets with different characteristics into different classes using classification rules. A classification rule is a filter rule configured to meet your management requirements. It can be very simple. For example, you can use a classification rule to identify traffic with different priorities according to the ToS field in the IP packet header. It can be very complicated too. For example, you can use a classification rule to identify the packets according to the combination of link layer (Layer 2), network layer (Layer 3), and transport layer (Layer 4) information including MAC addresses, IP protocol, source addresses, destination addresses, port numbers of applications, and so on. Generally, the traffic classification criterion is limited in the header of an encapsulated packet. Contents of packets are rarely adopted for traffic classification. The S9500 series support Layer 2, Layer 3, and Layer 4 ACL rules for traffic classification. Such ACL rules can classify packets based on source MAC addresses, destination MAC addresses, VLAN IDs, source IP addresses, destination IP addresses, source TCP/UDP port numbers, destination TCP/UDP port numbers, protocol types, IP precedence, ToS precedence, DSCP precedence, and whether packets are fragmented. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 4/15

80 S9500 QoS Technology White Paper 3.3 Traffic Policing To use limited network resources to provide customers with better services, you can enable traffic policing on the incoming port for the traffic of the specified customers, thus making the traffic adapt to the network resources assigned to it. Traffic policing uses token buckets for traffic control. Figure 2 Traffic policing Figure 2 depicts the processing procedure of traffic policing. First, packets are classified and the packets with the specified characteristics enter the token bucket for processing. If the token bucket has enough tokens for sending the packets, the packets can pass through; otherwise, the packets are dropped. In this way, you can control the traffic of a certain class of packets. The system puts tokens into the bucket at the set rate. You can set the capacity of the token bucket. When the token bucket is full, the extra tokens will overflow and the number of tokens in the bucket stops increasing. When the token bucket processes packets, if it has enough tokens for sending these packets, the packets are sent, and at the same time, the corresponding number of tokens are taken out of the bucket. If the token bucket does not have enough tokens for sending these packets, these packets are dropped. Therefore, the traffic rate is restricted under the rate of generating tokens, thus implementing traffic control. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 5/15

81 S9500 QoS Technology White Paper The S9500 series support traffic policing with the granularity of 8 kbps. 3.4 Priority Marking Through marking different priorities for packets, you can identify the service levels of different packets. The S9500 series can perform priority marking for specific packets. ToS precedence, differentiated services codepoint (DSCP) precedence, and 802.1p precedence can be marked. These priority types apply to different QoS models and are defined in different models. The following part introduces IP precedence, ToS precedence, DSCP precedence, 802.1p precedence, and EXP precedence. I. IP precedence, ToS precedence, and DSCP precedence Figure 3 IP precedence, ToS precedence, and DSCP precedence As shown in Figure 3, the ToS field of the IP header contains 8 bits: the first three bits (0 to 2) represent IP precedence from 0 to 7; the following 4 bits (3 to 6) represent a ToS value from 0 to 15. In RFC2474, the ToS field of the IP header is redefined as the DS field, where a DiffServ code point (DSCP) precedence is represented by the first 6 bits (0 to 5) and is in the range 0 to 63. The remaining 2 bits (6 and 7) are reserved. II p precedence 802.1p precedence lies in Layer 2 packet headers and is applicable to occasions where the Layer 3 packet header does not need analysis but QoS must be guaranteed at Layer 2. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 6/15

82 S9500 QoS Technology White Paper Figure Q Ethernet frame format As shown in the figure above, each host supporting the 802.1Q protocol adds a 4- byte 802.1Q tag header after the source address of the former Ethernet frame header when sending the packet. The 4-byte 802.1Q tag header contains a 2-byte Tag Protocol Identifier (TPID) whose value is 8100 and a 2-byte Tag Control Information (TCI). TPID is a new class field defined by IEEE to indicate that the current packet is 802.1Q-tagged. Figure 5 describes the detailed contents of an 802.1Q tag header. Figure p precedence In the figure above, the 3-bit priority field in the TCI field is 802.1p priority in the range of 0 to 7. The three bits specify the precedence of the frame. Eight precedence values are used to determine which packets are sent preferentially when congestion occurs. The precedence is called 802.1p precedence because applications related to the precedence are defined in detail in the 802.1p specifications. To provide differentiated services for VLAN VPN or QinQ frames, you must classify frames by VLANs or 802.1p precedence in their inner VLAN tags. The inner VLAN and 802.1p precedence of a packet determines its queue scheduling priority and drop precedence. The 802.1p precedence of the inner VLAN tag of a packet determines the scheduling priority and drop precedence of a packet at the egress. Figure p precedence mapping Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 7/15

83 S9500 QoS Technology White Paper III. EXP precedence Figure 7 MPLS label In an Ethernet MPLS packet, there is a shim between the Layer 2 header and Layer 3 data. You can use the reserved fields in the shim, a 3-bit EXP to determine the scheduling priority and drop precedence of the packet. You can classify MPLS packets by their EXP precedence and determine the scheduling priority and drop precedence of MPLS packets at the egress. You can map the DSCP precedence of IP packets to the EXP precedence and use the EXP precedence to determine the scheduling priority and drop precedence of MPLS packets at the egress. Figure 8 EXP precedence marking 3.5 Queue Scheduling When the network is congested, the problem that many packets compete for resources must be solved, usually through queue scheduling. The S9500 series support two queue scheduling algorithms: strict priority (SP), and weighted round robin (WRR). Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 8/15

84 S9500 QoS Technology White Paper I. SP queue scheduling algorithm High priority Queue 7 Packets sent via this interface Queue 6 Packets sent Queue 5~2 Classify dequeue Sending queue Queue 1 Low priority Queue 0 Figure 9 Diagram for SP queueing SP queue scheduling algorithm is dedicated to critical service applications. The key feature of mission-critical applications is that they require preferential service to reduce the response delay when congestion occurs. Assume that there are eight output queues on a port and the SP queueing classifies the eight output queues on the port into eight classes, which are queue 7, queue 6, queue 5, queue 4, queue 3, queue 2, queue 1, and queue 0 in the descending order of priority. SP schedules the packets in a strict priority order. It sends the packets in the queue of the highest priority first, and sends packets in a queue of a lower priority only when the queue of a higher priority is empty. You can put critical service packets into the queues with higher priority and put non-critical service (such as ) packets into the queues with lower priority. In this case, critical service packets are sent preferentially and non-critical service packets are sent when critical service groups are not sent. The SP mechanism has its disadvantage. When congestion occurs and if high-priority queues are occupied for a long time, the packets in the lower-priority queues are starved before obtaining services. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 9/15

85 S9500 QoS Technology White Paper II. WRR queue scheduling algorithm A switch port supports eight output queues. WRR queue-scheduling algorithm schedules all the queues in turn and every queue can be assured of a certain service time. Assume there are eight priority queues on a port. WRR configures a weight value for each queue, which is w7, w6, w5, w4, w3, w2, w1, and w0. The weight value indicates the proportion of obtaining bandwidth. On a 100 M port, configure the weight value of WRR queue-scheduling algorithm as 50, 30, 10, 10, 50, 30, 10, and 10 (corresponding to w7, w6, w5, w4, w3, w2, w1, and w0 in order). In this way, the queue with the lowest priority can get 5 Mbps bandwidth at least, thus avoiding the disadvantage of SP queue-scheduling that the packets in queues with lower priority may not get service for a long time. Another advantage of WRR queuing is that: though the queues are scheduled in order, the service time for each queue is not fixed; that is to say, if a queue is empty, the next queue will be scheduled. In this way, the bandwidth resources are made full use. 3.6 Congestion Avoidance When the network is congested, common network devices adopt tail drop to avoid congestion. That is, when the queue length reaches the upper threshold, all the newly arriving packets are dropped. However, if plenty of TCP traffic is dropped, which will cause TCP timeout, the slow start and congestion avoidance mechanisms of TCP will be triggered, thus reducing TCP traffic. If a queue drops packets of multiple TCP sessions at the same time, slow start and congestion avoidance mechanisms will be triggered for these TCP sessions at the same time. This is called global TCP synchronization. In this case, these TCP sessions reduce the size of traffic sent to the queue at the same time, so that the traffic sent to the queue is less than the bandwidth of the queue, thus reducing the utilization of the line. On the other hand, the size of the traffic sent to the queue is not stable but fluctuates between the maximum bandwidth and a very small traffic size. The S9500 series adopt the Weighted Random Early Detection (WRED) mechanism to avoid global TCP synchronization. You can set the upper threshold and lower threshold for a queue. When the queue length is smaller than the lower threshold, no Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 10/15

86 S9500 QoS Technology White Paper packet is dropped; when the queue length is between the lower threshold and the lower threshold, WRED begins to drop packets randomly, and the drop probability increases as the queue length increases; when the queue length is bigger than the upper threshold, all newly arriving packets are dropped. WRED drops packets randomly, thus avoiding global TCP synchronization. When the sending rate of a TCP session slows down after its packets are dropped, the other TCP sessions remain in high packet sending rates. In this way, some TCP sessions remain in high packet sending rates in any case, and the link bandwidth can be fully utilized. If the current queue length is compared with the upper threshold and lower threshold to determine the drop policy, bursty traffic is not fairly treated and proper data transmission is affected. To solve this problem, WRED compares the average queue size with the lower threshold and upper threshold to determine the drop policy. The average queue size reflects the queue size change trend but is not sensitive to bursty queue size changes, and thus bursty traffic can be fairly treated. On a S9500 switch, you can set the exponential factor for average queue length calculation, upper threshold, lower threshold, and drop probability for packets with different precedence values respectively to provide differentiated drop policies. When congestion occurs, the S9500 switch drops packets as soon as possible to release queue resources and try not to assign packets to high-delay queues in order to eliminate congestion. A S9500 switch can assign drop levels to packets according to their 802.1p precedence, that is, color the packets, or assign drop levels through priority marking. The drop level can be 0, 1, or 2, which represent green, yellow, and red respectively. When congestion occurs, red packets are the first to be dropped, while green packets are the last to be dropped. You can set congestion avoidance parameters and thresholds for each queue and each drop level. The S9500 series support two drop algorithms: Tail drop: when packets are dropped, the drop policy for packets in a color (red, yellow, or green, assigned according to drop levels) is determined by the Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 11/15

87 S9500 QoS Technology White Paper threshold set for the color. When the size of packets in a color (red, yellow, or green) exceeds the corresponding upper threshold, the system beings to drop newly arriving packets in this color. WRED drop algorithm: the drop levels are taken into account when packets are dropped by queue. When the size of packets in a color (red, green, or yellow) exceeds the lower threshold set for the color, the system begins to drop the packets in the color between the upper threshold and lower threshold according to a certain slope. When the size of packets in a color exceeds the upper threshold set for the color, the system begins to drop all packets in the color exceeding the upper threshold. 3.7 Traffic Shaping Traffic shaping controls the rate of output traffic, so that the traffic can be sent out at an even rate. Normally, traffic shaping is applied on a device to adapt its output rate to the input rate of its connected downstream device so as to avoid unnecessary packet drop and congestion. It differs from traffic policing mainly in that traffic shaping buffers packets exceeding the rate limit so that packets are sent out at an even rate, while traffic policing drops packets exceeding the rate limit. However, traffic shaping introduces additional delay while traffic policing does not. The S9500 series support port-based traffic shaping, that is, traffic shaping can be implemented to all traffic on a port. It also supports queue-based traffic shaping on a port. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 12/15

88 S9500 QoS Technology White Paper 3.8 Policy Routing Figure 10 Policy routing application scenario The S9500 series can classify packets first and then configure traffic redirecting for a certain class of packets to implement policy routing. As shown in Figure 10, the S9500 switch first classifies packets based on source IP addresses and destination IP addresses to identify packets whose source IP addresses are private address while whose destination IP address are public addresses. Then you can use policy routing to redirect such packets to the NAT device for address translation and then to the Internet. Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 13/15

89 S9500 QoS Technology White Paper 4 QoS Processing Procedure on the S9500 Series Figure 11 QoS processing procedure on the S9500 series The S9500 series use traffic classification to classify traffic based on source MAC addresses, destination MAC addresses, Ethernet types, VLANs, 802.1p priority, IP protocol, source IP addresses, destination IP addresses, application port numbers, ICMP packet types, IP precedence, ToS, DSCP, EXP, and VLAN IDs and 802.p priorities in the inner VLAN tags of QinQ frames. After classifying traffic into different classes, besides simply permitting a class of packets to pass through or dropping a class of packets, the S9500 series provide a policy control list (PCL) to perform the following actions for the traffic flows: traffic policing, traffic accounting, marking QoS parameters (including 802.1p priority, DSCP, EXP, and drop precedence), traffic mirroring, traffic redirecting, and specifying the output queue. After packets are marked with different drop levels through priority mapping, the congestion avoidance module determines the drop policies for packets based on the user-defined drop mode and the upper threshold and lower threshold set for each color. With tail drop adopted, when the size of packets in a color (red, yellow, or green) exceeds the upper threshold set for the color, the system begins to drop newly arriving packets in the color. With WRED drop mode adopted, when the size of packets in a color (red, green, or yellow) exceeds the lower threshold set for the color, Copyright 2007 Hangzhou H3C Technologies Co., Ltd. Page 14/15