Fragmenting and Interleaving Real-Time and Nonreal-Time Packets

CHAPTER 16 Fragmenting and Interleaving Real-Time and Nonreal-Time Packets Integrating delay-sensitive real-time traffic with nonreal-time data packets on low-speed links can cause the real-time packets to experience long queuing delays while waiting for the larger nonreal-time packets to transmit. Real-time traffic, however, cannot tolerate delay. The challenge becomes how to integrate real-time and nonreal-time packets while reducing latency for the real-time packets. The Cisco 10000 series router addresses this by breaking larger data packets into fragments and interleaving the smaller real-time packets between the fragments. In this way, time-sensitive real-time traffic remains intact and does not experience excessive delay. This chapter describes fragmentation and interleaving on the Cisco 10000 series router. It includes the following topics: Link Fragmentation and Interleaving, page 16-1 Multilink PPP-Based Link Fragmentation and Interleaving, page 16-11 FRF.12 Fragmentation, page 16-37 Configuration Examples for Link Fragmentation and Interleaving, page 16-59 Verifying and Monitoring Link Fragmentation and Interleaving, page 16-65 Related Documentation, page 16-69 Link Fragmentation and Interleaving Link fragmentation and interleaving (LFI) is a method that allows long nonreal-time data packets to be fragmented into smaller frames and shorter real-time packets to be interleaved between the fragments. In this way, real-time delay-sensitive packets, such as voice over IP (VoIP), and nonreal-time delay-insensitive packets, such as data transfer, can be carried together on low-speed links without causing excessive delay to the real-time traffic. Real-time delay-sensitive traffic becomes susceptible to increased latency when the network processes nonreal-time delay-insensitive packets. Long queuing delays can occur while real-time traffic waits for the nonreal-time packet to be transmitted. Therefore, controlling the maximum one-way end-to-end delay for time-sensitive traffic becomes challenging when integrating voice and data traffic. 16-1

Link Fragmentation and Interleaving Chapter 16 An important part of delay is the time it takes to actually place the bits onto an interface, referred to as serialization delay. We recommend that serialization delay not exceed 20 ms. Serialization delay is calculated using the following formula: Serialization Delay = Frame Size (bits) / Link Bandwidth (bps) As shown in Figure 16-1, a nonreal-time data packet of 1500 bytes takes 214 ms to leave the router over a 56-kbps link. While waiting for the large data packet to transmit, the router queues real-time packets. However, real-time traffic cannot tolerate delay. For example, good voice quality requires delay to be less than 150 ms. By fragmenting the nonreal-time large data packet into smaller frames and interleaving real-time packets between the fragments, both real-time packets and data frames can be carried together on low-speed links, without causing excessive delay to the real-time traffic. Figure 16-1 Integrating Voice and Data Packets on Low-Speed Links Before Voice Data After 214-ms serialization delay for 1500-byte frame at 56 kbps Data Data Voice Data 40822 The Cisco 10000 series router supports the following types of link fragmentation and interleaving (LFI): MLP over Serial-based LFI Uses the fragmentation and interleaving capability of MLP to integrate real-time packets (such as voice packets) and nonreal-time packets (such as data transfers) on the same link while reducing real-time packet latency. MLP defines the mechanisms to fragment, reassemble, and sequence large datagrams across multiple logical data links. MLP over serial-based LFI supports up to 10 member links per MLP bundle, one of which is LFI-enabled. You can terminate the serial links on multiple line cards in the router chassis if all of the links are the same type, such as T1 or E1. The router supports subrate T1 interfaces as member links. The link speeds must be the same for all of the links in the bundle. Single-VC MLP over ATM-based LFI Uses the fragmentation and interleaving capability of MLP to integrate real-time and nonreal-time packets together on the same link. MLP defines the mechanisms to fragment, reassemble, and sequence large datagrams across multiple logical data links. MLP uses the fragmentation and packet sequencing specifications defined in RFC 1990 to implement link fragmentation and interleaving at the bundle level. Single-VC MLP over ATM-based LFI supports only one member link per MLP bundle and the link is LFI-enabled. Multi-VC MLP over ATM-based LFI Uses the fragmentation and interleaving capability of MLP to integrate real-time packets and nonreal-time packets on the same link while reducing real-time packet latency. MLP implements link fragmentation and interleaving at the bundle level. Multi-VC MLP over ATM-based LFI supports up to 10 member links, one of which is LFI-enabled. 16-2

Chapter 16 Link Fragmentation and Interleaving MLP over Frame Relay-based LFI Uses the fragmentation and interleaving capability of MLP to transport real-time traffic (for example, voice) and nonreal-time traffic (for example, data transfers) together on low-speed Frame Relay permanent virtual circuits (PVCs) without causing excessive delay to the real-time traffic. MLP uses the fragmentation and packet sequencing specifications defined in RFC 1990 to implement link fragmentation and interleaving at the bundle level. MLP over Frame Relay-based LFI supports only one member link per MLP bundle and the link is LFI-enabled. FRF.12 Fragmentation Uses Frame Relay Forum FRF.12-based fragmentation on Frame Relay permanent virtual circuits (PVCs) to allow long, nonreal-time data packets to be broken into smaller fragments and shorter real-time packets to be interleaved between the fragments. In this way, real-time and nonreal-time packets can be carried together on low-speed links without causing excessive delay to the real-time traffic. The real-time packets remain intact and are less likely to experience long queuing delays. Feature History for Link Fragmentation and Interleaving Cisco IOS Release Description Required PRE Release 12.0(23)SX The PVC-based FRF.12 Fragmentation feature and the MLP over Serial-based LFI feature were introduced on the router. PRE1 Release 12.0(27)S Release 12.2(27)SBB Release 12.2(31)SB2 The FRF.12 Fragmentation feature was enhanced to enable interface-based FRF.12 fragmentation. The following features were introduced on the PRE2: MLP over Serial-based LFI, MLP over Frame Relay-based LFI, Single-VC and Multi-VC MLP over ATM-based LFI, and PVC-based and Interface-based FRF.12 Fragmentation. The following features were introduced on the PRE3: MLP over Serial-based LFI, MLP over Frame Relay-based LFI, Single-VC and Multi-VC MLP over ATM-based LFI, and PVC-based and Interface-based FRF.12 Fragmentation. PRE1 PRE2 PRE3 16-3

Link Fragmentation and Interleaving Chapter 16 System Limits for Link Fragmentation and Interleaving Table 16-1 lists the system limits for link fragmentation and interleaving (LFI). Table 16-1 System Limits for Link Fragmentation and Interleaving Feature MLP over Serial-based LFI Single-VC MLP over ATM-based LFI Multi-VC MLP over ATM-based LFI Maximum No. of Members Per Bundle Maximum No. of Bundles Per System Maximum No. of Member Links Per System Multilink Interface Range 10 1250 2500 1 to 9999 (Release12.2(28)SB and later) 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later) LFI Supported Yes Interleaving on 1 member link 1 8192 8192 10,000 and higher Yes Interleaving on 1 member link 10 1250 2500 1 to 9999 (Release12.2(28)SB and later) 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later) Yes Interleaving on 1 member link MLP over Frame Relay-based LFI 1 2048 2048 10,000 and higher Yes Interleaving on 1 member link FRF.12 Fragmentation NA NA 4096 NA Yes Note The multilink interface ranges described in Table 16-1 require Cisco IOS Release 12.2(28)SB and later releases. For releases prior to Cisco IOS Release 12.2(28)SB, the valid multilink interface range is 1 to 2,147,483,647. 16-4

Chapter 16 Link Fragmentation and Interleaving Configuration Commands for MLP-Based Fragmentation and Interleaving The following commands are used to configure Multilink PPP (MLP)-based fragmentation and interleaving: interface multilink Command, page 16-5 ppp multilink Command, page 16-6 ppp multilink fragment-delay Command, page 16-7 ppp multilink interleave Command, page 16-8 ppp multilink fragmentation Command, page 16-8 ppp multilink fragment disable Command, page 16-9 ppp multilink group Command, page 16-10 interface multilink Command To create and configure a multilink bundle, use the interface multilink command in global configuration mode. To remove a multilink bundle, use the no form of the command. By default, no multilink interfaces are configured. interface multilink multilink-bundle-number no interface multilink multilink-bundle-number Syntax Description multilink-bundle-number A nonzero number that identifies the multilink bundle. interface multilink Command History Cisco IOS Release Release 12.0 Release 12.2(16)BX Release 12.2(28)SB Release 12.2(31)SB2 Description The interface multilink command was introduced on the PRE1. This command was introduced on the PRE2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. The range of valid values for multilink interfaces was changed from 1 to 9999 (Release 12.2(28)SB and later) to from 1 to 9999 and 65,536 to 2,147,483,647 for MLP over serial and multi-vc MLP over ATM. Usage Guidelines for the interface multilink Command The following describes the range of valid values for multilink interfaces: MLP over Serial-based LFI 1 to 9999 (Cisco IOS Release 12.2(28)SB and later releases) 1 to 9999 and 65,536 to 2,147,483,647 (Cisco IOS Release 12.2(31)SB2 and later releases) Single-VC MLP over ATM-based LFI 10,000 and higher (Cisco IOS Release 12.2(28)SB and later releases) 16-5

Link Fragmentation and Interleaving Chapter 16 Multi-VC MLP over ATM-based LFI 1 to 9999 (Cisco IOS Release 12.2(28)SB and later releases) 1 to 9999 and 65,536 to 2,147,483,647 (Cisco IOS Release 12.2(31)SB2 and later releases) MLP over Frame Relay-based LFI 10,000 and higher For releases prior to Cisco IOS Release 12.2(28)SB, the valid multilink interface range is 1 to 2,147,483,647. ppp multilink Command To enable Multilink PPP (MLP) on an interface, use the ppp multilink command in interface configuration mode. To disable MLP, use the no form of the command. By default, the command is disabled. ppp multilink no ppp multilink ppp multilink Command History Cisco IOS Release Release 12.0(23)SX Release 12.2(16)BX Release 12.2(28)SB Release 12.2(31)SB2 Description The ppp multilink command was introduced on the PRE1. This command was introduced on the PRE2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. Usage Guidelines for the ppp multilink Command The ppp multilink command applies only to interfaces that use Point-to-Point Protocol (PPP) encapsulation. When you use the ppp multilink command, the first channel negotiates the appropriate Network Control Protocol (NCP) layers (such as the IP Control Protocol and IPX Control Protocol), but subsequent links negotiate only the Link Control Protocol (LCP) and MLP. 16-6

Chapter 16 Link Fragmentation and Interleaving ppp multilink fragment-delay Command To specify a maximum size in units of time for packet fragments on a Multilink PPP (MLP) bundle, use the ppp multilink fragment-delay command in interface configuration mode. To reset the maximum delay to the default value, use the no form of the command. By default, if fragmentation is enabled, the fragment delay is 30 milliseconds. ppp multilink fragment-delay delay-max no ppp multilink fragment-delay delay-max Syntax Description delay-max Specifies the maximum amount of time, in milliseconds, that is required to transmit a fragment. Valid values are from 1 to 1000 milliseconds. ppp multilink fragment-delay Command History Cisco IOS Release Release 12.0(23)SX Release 12.2(16)BX Release 12.2(28)SB Release 12.2(31)SB2 Description The ppp multilink fragment-delay command was introduced on the PRE1. This command was introduced on the PRE2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. Usage Guidelines for the ppp multilink fragment-delay Command The ppp multilink fragment-delay command is useful when packets are interleaved and traffic characteristics such as delay, jitter, and load balancing must be tightly controlled. MLP chooses a fragment size on the basis of the maximum delay allowed. If real-time traffic requires a certain maximum boundary on delay, using the ppp multilink fragment-delay command to set that maximum time can ensure that a real-time packet gets interleaved within the fragments of a large packet. By default, MLP has no fragment size constraint, but the maximum number of fragments is constrained by the number of links. If interleaving is enabled, or if the bundle contains links that have differing bandwidths, or if a fragment delay is explicitly configured with the ppp multilink fragment-delay command, then MLP uses a different fragmentation algorithm. In this mode, the number of fragments is unconstrained, but the size of each fragment is limited to the fragment-delay value, or 30 milliseconds if the fragment delay has not been configured. The ppp multilink fragment-delay command is configured under the multilink interface. The value assigned to the delay-max argument is scaled by the speed at which a link can convert the time value into a byte value. 16-7

Link Fragmentation and Interleaving Chapter 16 ppp multilink interleave Command To enable interleaving of real-time packets among the fragments of larger nonreal-time packets on a Multilink PPP (MLP) bundle, use the ppp multilink interleave command in interface configuration mode. To disable interleaving, use the no form of the command. By default, interleaving is disabled. ppp multilink interleave no ppp multilink interleave ppp multilink interleave Command History Cisco IOS Release Release 12.0(23)SX Release 12.2(16)BX Release 12.2(28)SB Release 12.2(31)SB2 Description The ppp multilink interleave command was introduced on the PRE1. This command was introduced on the PRE2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. Usage Guidelines for the ppp multilink interleave Command The ppp multilink interleave command applies to multilink interfaces, which are used to configure a bundle. Interleaving works only when the queuing mode on the bundle is set to fair queuing. If interleaving is enabled when fragment delay is not configured, the default delay is 30 milliseconds. The fragment size is derived from that delay, depending on the bandwidths of the links. ppp multilink fragmentation Command To enable packet fragmentation, use the ppp multilink fragmentation command in interface configuration mode. To disable fragmentation, use the no form of the command. By default, fragmentation is enabled. ppp multilink fragmentation no ppp multilink fragmentation ppp multilink fragmentation Command History Cisco IOS Release Release 12.0(23)SX Release 12.2(16)BX Release 12.2 Release 12.2(28)SB Release 12.2(31)SB2 Description The ppp multilink fragmentation command was introduced on the PRE1. This command was introduced on the PRE2. The no ppp multilink fragmentation command was changed to ppp multilink fragment disable. The no ppp multilink fragmentation command is recognized and accepted through Cisco IOS Release 12.2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. 16-8

Chapter 16 Link Fragmentation and Interleaving ppp multilink fragment disable Command To disable packet fragmentation, use the ppp multilink fragment disable command in interface configuration mode. To enable fragmentation, use the no form of this command. ppp multilink fragment disable no ppp multilink fragment disable ppp multilink fragment disable Command History Cisco IOS Release Release 11.3 Release 12.2 Release 12.2(28)SB Release 12.2(31)SB2 Description This command was introduced as ppp multilink fragmentation. The no ppp multilink fragmentation command was changed to ppp multilink fragment disable. The no ppp multilink fragmentation command was recognized and accepted through Cisco IOS Release 12.2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. Usage Guidelines for the ppp multilink fragment disable Command The ppp multilink fragment delay and ppp multilink interleave commands have precedence over the ppp multilink fragment disable command. Therefore, the ppp multilink fragment disable command has no effect if these commands are configured for a multilink interface and the following message displays: Warning: 'ppp multilink fragment disable' or 'ppp multilink fragment maximum' will be ignored, since multilink interleaving or fragment delay has been configured and have higher precedence. To completely disable fragmentation, you must do the following: Router(config-if)# no ppp multilink fragment delay Router(config-if)# no ppp multilink interleave Router(config-if)# ppp multilink fragment disable 16-9

Link Fragmentation and Interleaving Chapter 16 ppp multilink group Command To restrict a physical link to joining only a designated multilink group interface, use the ppp multilink group command in interface configuration mode. To remove the restriction, use the no form of the command. By default, this command is disabled. ppp multilink group group-number no ppp multilink group group-number Syntax Description group-number Identifies the multilink group. This number must be identical to the multilink-bundle-number you assigned to the multilink interface. Valid values are: MLP over Serial-based LFI 1 to 9999 (Cisco IOS Release 12.2(28)SB and later releases) 1 to 9999 and 65,536 to 2,147,483,647 (Cisco IOS Release 12.2(31)SB2 and later releases) Single-VC MLP over ATM-based LFI 10,000 and higher Multi-VC MLP over ATM-based LFI 1 to 9999 (Cisco IOS Release 12.2(28)SB and later releases) 1 to 9999 and 65,536 to 2,147,483,647 (Cisco IOS Release 12.2(31)SB2 and later releases) MLP over Frame Relay-based LFI 10,000 and higher ppp multilink group Command History Cisco IOS Release Release 12.0 Release 12.2 Release 12.2(28)SB Release 12.2(31)SB2 Description The ppp multilink group command was introduced on the PRE1 as multilink-group command. The multilink-group command was changed on the PRE2 to ppp multilink group. The multilink-group command was accepted by the command line interpreter through Cisco IOS Release 12.2. This command was integrated in Cisco IOS Release 12.2(28)SB for the PRE2. This command was introduced on the PRE3. 16-10

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving Usage Guidelines for the ppp multilink group Command By default the ppp multilink group command is disabled, which means the link can negotiate to join any bundle in the system. When the ppp multilink group command is configured, the physical link is restricted from joining any but the designated multilink group interface. If a peer at the other end of the link tries to join a different bundle, the connection is severed. This restriction applies when MLP is negotiated between the local end and the peer system. The link can still come up as a regular PPP interface. Multilink PPP-Based Link Fragmentation and Interleaving Interactive traffic such as Telnet and Voice over IP (VoIP) is susceptible to increased latency when the network processes large packets such as LAN-to-LAN FTP transfers traversing a WAN. Packet delay is especially significant when the FTP packets are queued on slow links within the WAN. To solve delay problems on slow bandwidth links, the router supports link fragmentation and interleaving (LFI) based on the Cisco implementation of Multilink PPP (MLP), which supports the fragmentation and packet-sequencing specifications in RFC 1990. LFI allows reserve queues to be set up so that Real-Time Transport Protocol (RTP) streams can be mapped into a higher priority queue. As shown in Figure 16-2, without fragmentation and interleaving nonreal-time data packets can overwhelm real-time packets such as voice. However, when fragmentation and interleaving is enabled, bandwidth is shared equitably between nonreal-time packets and real-time packets. Figure 16-2 Fragmenting and Interleaving Packets Voice Data Voice Data Without LFI, data can overwhelm voice With LFI, bandwidth is equitably shared to reduce voice jitter Voice Data 132419 MLP fragmentation allows large packets to be multilink encapsulated and fragmented into a small enough size to satisfy the delay requirements of real-time traffic. MLP fragmentation is enabled by default. To disable fragmentation, use the no ppp multilink fragmentation or ppp multilink fragment disable command. Small real-time packets are not multilink encapsulated. MLP interleaving provides a special transmit queue (priority queue) for delay-sensitive packets to allow the packets to be sent earlier than other packet flows. Real-time packets remain intact and are sent (interleaved) between the fragments of the larger packets. 16-11

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 The router supports MLP-based fragmentation and interleaving on serial, Frame Relay, and ATM links. For information on how MLP works and MLP-based LFI, see the following sections: How MLP-Based LFI Works, page 16-12 MLP Over Serial-Based LFI, page 16-13 Single-VC MLP Over ATM-Based LFI, page 16-14 Multi-VC MLP Over ATM-Based LFI, page 16-19 MLP Over Frame Relay-Based LFI, page 16-21 How MLP-Based LFI Works To understand how MLP-based LFI works, it helps to understand the problem it addresses. The complete end-to-end delay target for real-time packets, especially voice packets, is 150 to 200 milliseconds (ms). The IP-based datagram transmission techniques for audio transmission do not adequately address the problems posed by limited bandwidth and the very stringent telephony delay bound of 150 ms. Unacceptable queuing delays for small real-time packets exist regardless of the use of QoS features such as weighted fair queuing (WFQ), and the use of voice compression algorithms such as code excited linear prediction (CELP) compression, which reduces the inherent bit rate from 64 kbps to as low as 8 kbps. Despite these measures, real-time delay continues to exist because per-packet header overhead is too large and large maximum transmission units (MTUs) are needed to produce acceptable bulk transmission efficiency. A large MTU of 1500 bytes takes 215 ms to traverse a 56-kbps line, which exceeds the delay target. Therefore, to limit the delay of real-time packets on relatively slow bandwidth links links such as 56-kbps Frame Relay or 64-kbps ISDN B channels a method for fragmenting larger packets and queuing smaller packets between fragments of the large packet is needed. MLP helps to solve this problem through LFI. MLP provides a method of splitting, recombining, and sequencing datagrams across multiple logical data links. The LFI scheme is relatively simple: large datagrams are multilink encapsulated and fragmented to packets of a size small enough to satisfy the delay requirements of the delay-sensitive traffic; small delay-sensitive packets are not multilink encapsulated, but are interleaved between fragments of the large datagram. MLP allows the fragmented packets to be sent at the same time over multiple point-to-point links to the same remote address. The multiple links come up in response to a dialer load threshold that you define. The load can be calculated on inbound traffic, outbound traffic, or on either, as needed for the traffic between the specific sites. MLP provides bandwidth on demand and reduces transmission latency across WAN links. To ensure correct order of transmission and reassembly, LFI adds multilink headers to the datagram fragments after the packets are dequeued and ready to be sent. 16-12

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving MLP Over Serial-Based LFI MLP over serial-based LFI uses the fragmentation and interleaving capability of MLP to integrate real-time packets (such as voice packets) and nonreal-time packets (such as data transfers) on the same link while reducing real-time packet latency. MLP allows you to bundle T1 interfaces into logical groups. As indicated in Figure 16-3, you can configure a MLP bundle with up to 10 T1 links. Using MLP, you can create a degree of redundancy by configuring a MLP bundle that is made up of T1 links from more than one line card. If one line card stops operating, the part of the bundle on other line cards continues to operate. Figure 16-3 shows a MLP bundle that consists of T1 interfaces from three T3 interfaces. Figure 16-3 MLP Bundle You can combine up to ten T1s to create a Multilink bundle. The bundle can include T1 channels assigned to different T3s. Multilink bundle T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 T1 38545 Feature History for MLP Over Serial-Based LFI Cisco IOS Release Description Required PRE Release 12.0(23)SX The MLP over serial-based LFI feature was introduced on PRE1 the PRE1. Release 12.2(27)SBB This feature was introduced on the PRE2. PRE2 Release 12.2(31)SB2 This feature was introduced on the PRE3 and the valid multilink interface values changed from 1 to 9999 (Release 12.2(28)SB and later) to from 1 to 9999 and 65,536 to 2,147,483,647. PRE3 Performance and Scalability for MLP Over Serial-Based LFI To enhance performance and scalability for MLP over serial-based LFI, configure the hold-queue command in interface configuration mode for all physical interfaces, except when configuring the ATM OC-12 line card. The OC-12 does not require the hold-queue command. For example: Router(config-if)# hold-queue 4096 in For more information, see the Scalability and Performance chapter in the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. 16-13

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 Restrictions and Limitations for MLP Over Serial-Based LFI A multilink bundle can have up to 10 member links. The router supports subrate T1 interfaces as member links. The link speeds must be the same for all of the links in the bundle. The router supports a maximum of 1250 bundles per system and a maximum of 2500 member links per system. The valid multilink interface values are from 1 to 9999 (Release 12.2(28)SB and later), or from 1 to 9999 and 65,536 to 2,147,483,647 (Release 12.2(31)SB2 and later). For example: Router(config)# interface multilink 8 Interleaving is supported on one member link. MLP over Serial-based LFI must be enabled on an interface that has interleaving turned on. All member links in a MLP bundle must have the same encapsulation type and bandwidth. Performance is not guaranteed when member links have different bandwidths. We strongly recommend that you use only strict priority queues when configuring MLP over Serial-based LFI. For more information, see Chapter 8, Prioritizing Services. Line Card Support for MLP Over Serial-Based LFI The following line cards support MLP over serial-based LFI for the Cisco 10000 series router: 24-port Channelized T1/E1 6-port Channelized T3 4-port Channelized OC-3/STM-1 1-port Channelized OC-12/STM-4 Single-VC MLP Over ATM-Based LFI Single-VC MLP over ATM-based LFI uses the fragmentation and interleaving capability of MLP to integrate real-time and nonreal-time packets together on the same link. MLP defines the mechanisms to fragment, reassemble, and sequence large datagrams across multiple logical data links. MLP uses the fragmentation and packet sequencing specifications defined in RFC 1990 to implement link fragmentation and interleaving at the bundle level, supporting only one LFI-enabled member link per MLP bundle. 16-14

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving The following describes how MLP implements fragmentation and interleaving: Delay-Sensitive, Real-Time Packets On transmit, MLP encapsulates the packets as PPP over ATM (PPPoA) and sends the packets to a special transmit queue to enable the router to transmit the real-time packets earlier than other packet flows. The router interleaves the real-time packets between the fragments of the larger, nonreal-time packet over a single point-to-point link to the remote address. Upon receipt, the receiving fragmentation peer processes the real-time packets as PPPoA packets. Delay-Insensitive, Nonreal-Time Packets On transmit, MLP fragments the large data packets to a size small enough to satisfy the delay requirements of real-time traffic. MLP encapsulates the packets as MLP packets and sends the packets to a transmit queue to enable the router to transmit the fragments at the same time over multiple point-to-point links to the same remote address. The receiving fragmentation peer reassembles the fragments to the original packet and then processes it as Point-to-Point Protocol over ATM (PPPoA). The underlying PPP encapsulation conforms to RFC 1661. All outbound MLP packets with a payload larger than the specified fragment size are fragmented. The minimum fragment size depends on the AAL5 encapsulation type and whether or not protocol compression is enabled (see Table 16-2 on page 16-15). When configuring single-vc MLP over ATM-based LFI, you must configure a virtual template interface for the MLP bundle. However, the virtual template does not need to be unique for each bundle multiple MLP bundles can share the same virtual template. For more information about MLP, see the Multilink PPP-Based Link Fragmentation and Interleaving section on page 16-11 and the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. Feature History for Single-VC MLP Over ATM-Based LFI Cisco IOS Release Description Required PRE Release 12.2(27)SBB The single-vc MLP over ATM-based LFI feature was PRE2 introduced on the router for the PRE2. Release 12.2(31)SB2 This feature was introduced on the PRE3. PRE3 Fragment Size Calculation for MLP Over ATM-Based LFI For MLP over ATM-based LFI, the ideal fragment size should allow the fragments to fit into an exact multiple of ATM cells. Table 16-2 lists the minimum fragment sizes for Single-VC and Multi-VC MLP over ATM-based LFI. As shown in the table, the minimum fragment size depends on the AAL5 encapsulation type used and whether or not protocol compression is enabled. Table 16-2 ATM Minimum Fragment Size AAL5 Encapsulation Type Protocol Compression Minimum Fragment Size AAL5 MUX OFF 82 Bytes AAL5 SNAP OFF 78 Bytes AAL5 Cisco OFF 80 Bytes AAL5 MUX ON 83 Bytes AAL5 SNAP ON 79 Bytes AAL5 Cisco ON 81 Bytes 16-15

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 To calculate the fragment size, do the following: Step 1 Step 2 Step 3 Step 4 Calculate the nominal fragment size (link weight) by using the following formula: (Link Bandwidth * Fragment-Delay) / 8 Determine the number of whole ATM cells the nominal fragment size represents. If the number of ATM cells is less than two, then use two ATM cells in Step 3. The minimum number of ATM cells you can have is 2. Calculate the total bytes per fragment, including the MLP header bytes and AAL5 trailer bytes, by multiplying the number of ATM cells you calculated in Step 2 by 48: Number of Cells * 48 Subtract the MLP header bytes and AAL5 trailer bytes. The AAL trailer is 8 bytes. The number of MLP header bytes depends on the AAL5 encapsulation and whether or not protocol compression is enabled (see Table 16-3). Table 16-3 lists the number of bytes in the MLP header, depending on the AAL5 encapsulation type and whether or not protocol compression is used. Table 16-3 MLP Header Bytes and AAL5 Trailer Bytes AAL5 Encapsulation Type MLP Bundle Interface and Service Policies Protocol Compression No. of MLP Header Bytes AAL5 MUX OFF 6 Bytes 8 Bytes AAL5 SNAP OFF 10 Bytes 8 Bytes AAL5 Cisco OFF 8 Bytes 8 Bytes AAL5 MUX ON 5 Bytes 8 Bytes AAL5 SNAP ON 9 Bytes 8 Bytes AAL5 Cisco ON 7 Bytes 8 Bytes No. of AAL5 Trailer Bytes The router applies a service policy, attached to a multilink interface, to only the MLP bundle interface. The QoS actions defined by the service policy are applied to the outbound nonreal-time packets before the packets reach the bundle first-in first-out (FIFO) queue. The nonreal-time packets are fragmented in the FIFO queue and then the real-time packets are interleaved between the fragments as the real-time packets exit their priority queue. 16-16

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving Transmit Processing The purpose of MLP over ATM-based LFI transmit processing is to fragment large nonreal-time delay-insensitive packets and interleave smaller real-time delay-sensitive packets between the fragments. Each MLP bundle has multiple transmit packet queues. MLP does not interleave packet fragments from different packet queues associated with a given MLP bundle. Instead, MLP transmits all of the fragments associated with a nonreal-time packet in order before transmitting fragments from another nonreal-time packet. MLP posts all of the packets from the various nonreal-time packet queues to a single bundle first-in first-out (FIFO) queue. It is from this single bundle queue that MLP does the following: Fragments nonreal-time traffic Encapsulates the fragments with MLP Transmits the fragments Real-time traffic, such as voice, are queued intact to a priority (low-latency) queue. It is from this queue that MLP transmits the real-time packets and interleaves them between the nonreal-time fragments. Because real-time packets are not MLP encapsulated or fragmented, MLP can safely interleave these packets as needed. Traffic transmitted from the priority queue takes precedence over the MLP encapsulated traffic that is transmitted from the related bundle queue. Figure 16-4 shows an example of the packet flow of real-time and nonreal-time packets. Figure 16-4 Delay Insensitive Packet Queues MLP Over ATM-Based LFI Packet Queue Flow Bundle FIFO Queue Link Delay Sensitive Packet Queues 132278 Receive Processing The purpose of MLP over ATM-based LFI receive processing is to reassemble MLP over ATM encapsulated packet fragments into PPP over ATM packets. During receive processing, the fragments that arrive out of order and the packets with missing fragments are discarded. Valid fragments are merged in memory until the entire packet is reassembled. 16-17

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 Performance and Scalability for Single-VC MLP Over ATM-Based LFI The following list describes how to enhance performance and scalability for Single-VC MLP over ATM-based LFI: Configure the following commands and recommended values on the virtual template interface: ppp max-configure 110 ppp max-failure 100 ppp timeout retry 5 keepalive 30 Configure the hold-queue command in interface configuration mode for all physical interfaces, except when configuring the ATM OC-12 line card. The OC-12 does not require the hold-queue command. For more information, see the Scalability and Performance chapter in the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. Restrictions and Limitations for Single-VC MLP Over ATM-Based LFI Single-VC MLP over ATM member links are restricted to non-aggregated PVCs (for example, variable bit rate-nonreal-time (VBR-nrt) and constant bit rate (CBR) ATM traffic classes only). The multilink interface can have only one PPP link that is between 64 and 2048 kbps. The router supports a maximum of one member link per bundle. The router supports a maximum of 8192 bundles per system and a maximum of 8192 member links per system. The valid multilink interface values are 10,000 and higher. Interleaving occurs on one member link. MLP VCs cannot be on-demand VCs that are automatically provisioned. Associating MLP over ATM VCs with ATM virtual paths (VPs) is discouraged, though not prevented. Cisco IOS software supports a maximum of 4096 total virtual template interfaces. We strongly recommend that you use only strict priority queues when configuring MLP over ATM-based LFI. For more information, see Chapter 8, Prioritizing Services. Line Card Support for MLP Over ATM-Based LFI The following line cards support MLP over ATM-based LFI for the Cisco 10000 series router: 8-Port E3/DS3 ATM 4-Port OC-3/STM-1 ATM 1-Port OC-12 ATM 16-18

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving Multi-VC MLP Over ATM-Based LFI Multi-VC MLP over ATM-based LFI uses the fragmentation and interleaving capability of MLP to integrate real-time and nonreal-time packets together on the same link. MLP defines the mechanisms to fragment, reassemble, and sequence large datagrams across multiple logical data links. MLP uses the fragmentation and packet sequencing specifications defined in RFC 1990 to implement link fragmentation and interleaving at the bundle level. Multi-VC MLP over ATM-based LFI supports up to 10 member links per MLP bundle, one of which is LFI-enabled. MLP encapsulates real-time packets as PPP over ATM (PPPoA) and sends the packets to a priority transmit queue to enable the router to transmit the real-time packets earlier than other packet flows. The router interleaves the real-time packets between the fragments of the larger, nonreal-time packet over a single point-to-point link to the remote address. Upon receipt, the receiving fragmentation peer processes the real-time packets as PPPoA packets. MLP fragments the large data packets to a size small enough to satisfy the delay requirements of real-time traffic. MLP encapsulates the packets as MLP packets and sends the packets to a first-in first-out (FIFO) queue to enable the router to transmit the fragments at the same time over multiple point-to-point links to the same remote address. Upon receipt, the receiving fragmentation peer reassembles the fragments to the original packet and then processes it as PPPoA. The underlying PPP encapsulation conforms to RFC 1661. MLP fragments all outbound MLP packets with a payload that is larger than the specified fragment size. The smallest fragment size depends on the AAL5 encapsulation type and whether or not protocol compression is enabled. For more information, see the Fragment Size Calculation for MLP Over ATM-Based LFI section on page 16-15. When configuring Multi-VC MLP over ATM-based LFI, you must configure a virtual template interface for the MLP bundle. However, the virtual template does not need to be unique for each bundle multiple MLP bundles can share the same virtual template. Feature History for Multi-VC MLP Over ATM-Based LFI Cisco IOS Release Description Required PRE Release 12.2(27)SBB The Multi-VC MLP over ATM-based LFI feature was PRE2 introduced on the router for the PRE2. Release 12.2(31)SB2 This feature was introduced on the PRE3 and the valid multilink interface values changed from 1 to 9999 (Release 12.2(28)SB and later) to from 1 to 9999 and 65,536 to 2,147,483,647. PRE3 Fragment Size Calculation for Multi-VC MLP Over ATM-Based LFI For Multi-VC MLP over ATM-based LFI, the ideal fragment size should allow the fragments to fit into an exact multiple of ATM cells. Table 16-2 on page 16-15 lists the minimum fragment sizes for Single-VC and Multi-VC MLP over ATM-based LFI. As shown in the table, the minimum fragment size depends on the AAL5 encapsulation type used and whether or not protocol compression is enabled. For more information, see the Fragment Size Calculation for MLP Over ATM-Based LFI section on page 16-15. 16-19

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 MLP Bundle Interface and Service Policies The router applies a service policy, attached to a multilink interface, to only the MLP bundle interface. The QoS actions defined by the service policy are applied to the outbound nonreal-time packets before the packets reach the bundle first-in first-out (FIFO) queue. The nonreal-time packets are fragmented in the FIFO queue and then the real-time packets are interleaved between the fragments as the real-time packets exit their priority queue. Performance and Scalability for Multi-VC MLP Over ATM-Based LFI The following list describes how to enhance performance and scalability for Multi-VC MLP over ATM-based LFI: Configure the following commands and recommended values on the virtual template interface: ppp max-configure 110 ppp max-failure 100 ppp timeout retry 5 keepalive 30 Configure the hold-queue command in interface configuration mode for all physical interfaces, except when configuring the ATM OC-12 line card. The OC-12 does not require the hold-queue command. For more information, see the Scalability and Performance chapter in the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. Restrictions and Limitations for Multi-VC MLP Over ATM-Based LFI Multi-VC MLP over ATM member links are restricted to non-aggregated PVCs (for example, variable bit rate-nonreal-time (VBR-nrt) and constant bit rate (CBR) ATM traffic classes only). Each MLP over ATM member link can be up to 2048 kbps. The router supports a maximum of 10 member links (ATM PVCs) per bundle. The router supports a maximum of 1250 bundles per system and a maximum of 2500 member links per system. For Cisco IOS Release 12.2(28)SB and later releases, the valid multilink interface values are from 1 to 9999. For Cisco IOS Release 12.2(31)SB2 and later releases, valid values are from 1 to 9999 and 65,536 to 2,147,483,647. All member links in a MLP bundle must have the same encapsulation type and bandwidth. Performance is not guaranteed when member links have different bandwidths. Interleaving occurs on one member link. MLP VCs cannot be on-demand VCs that are automatically provisioned. Associating MLP over ATM VCs with ATM virtual paths (VPs) is discouraged, though not prevented. Cisco IOS software supports a maximum of 4096 total virtual template interfaces. We strongly recommend that you use only strict priority queues when configuring Multi-VC MLP over ATM-based LFI. For more information, see Chapter 8, Prioritizing Services. 16-20

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving Line Card Support for MLP Over ATM-Based LFI The following line cards support MLP over ATM-based LFI for the Cisco 10000 series router: 8-Port E3/DS3 ATM 4-Port OC-3/STM-1 ATM 1-Port OC-12 ATM MLP Over Frame Relay-Based LFI MLP over Frame Relay-based LFI uses the fragmentation and interleaving capability of MLP to transport real-time traffic (for example, voice) and nonreal-time traffic (for example, data transfers) together on low-speed Frame Relay permanent virtual circuits (PVCs) without causing excessive delay to the real-time traffic. MLP over Frame Relay-based LFI supports RFC 1990, The PPP Multilink Protocol. MLP over Frame Relay-based LFI makes it possible for delay-sensitive packets and delay-insensitive packets to share the same link by fragmenting the long data packets into a sequence of smaller data packets referred to as fragments. The fragments are interleaved with the real-time packets. On the receiving side of the link, the fragments are reassembled and the packet is reconstructed. This method of fragmenting and interleaving helps guarantee the appropriate QoS for the real-time traffic. When configuring MLP over Frame Relay-based LFI, you must configure a virtual template interface for the MLP bundle. The virtual template must be unique to only that bundle multiple MLP bundles cannot share the same virtual template. For more information about MLP, see the Multilink PPP-Based Link Fragmentation and Interleaving section on page 16-11 and the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. Feature History for MLP Over Frame Relay-Based LFI Cisco IOS Release Description Required PRE Release 12.2(27)SBB The MLP over Frame Relay-based LFI feature was PRE2 introduced on the router for the PRE2. Release 12.2(31)SB2 This feature was introduced on the PRE3. PRE3 Multilink Group Interfaces and Virtual Template Interfaces You can configure MLP by assigning a multilink group to a virtual template interface configuration. Virtual templates allow a virtual access interface (VAI) to dynamically clone interface parameters from the specified virtual template. If you assign a multilink group to a virtual template and you assign the virtual template to a physical interface, all of the links that pass through the physical interface belong to the same multilink bundle. 16-21

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 MLP Bundle Interface and Service Policies The router applies a service policy, attached to a multilink interface, to only the MLP bundle interface. The QoS actions defined by the service policy are applied to the outbound nonreal-time packets before the packets reach the bundle first-in first-out (FIFO) queue. The nonreal-time packets are fragmented in the FIFO queue and then the real-time packets are interleaved between the fragments as the real-time packets exit their priority queue. Transmit Processing The purpose of MLP over Frame Relay-based LFI transmit processing is to fragment large nonreal-time delay-insensitive packets and interleave smaller real-time delay-sensitive packets between the fragments. Each MLP bundle has multiple transmit packet queues. MLP does not interleave packet fragments from different packet queues associated with a given MLP bundle. Instead, MLP transmits all of the fragments associated with a nonreal-time packet in order before transmitting fragments from another nonreal-time packet. MLP posts all of the packets from the various nonreal-time packet queues to a single bundle first-in first-out (FIFO) queue. It is from this single bundle queue that MLP does the following: Fragments nonreal-time traffic Encapsulates the fragments with MLP Transmits the fragments Real-time traffic, such as voice, are queued intact to a priority (low-latency) queue. It is from this queue that MLP transmits the real-time packets and interleaves them between the nonreal-time fragments. Because real-time packets are not MLP encapsulated or fragmented, MLP can safely interleave these packets as needed. Traffic transmitted from the priority queue takes precedence over the MLP encapsulated traffic that is transmitted from the related bundle queue. Figure 16-4 shows an example of the packet flow of real-time and nonreal-time packets. Figure 16-5 Delay Insensitive Packet Queues MLP Over Frame Relay-Based LFI Packet Queue Flow Bundle FIFO Queue Link Delay Sensitive Packet Queues 132278 Receive Processing The purpose of MLP over Frame Relay-based LFI receive processing is to reassemble MLP over Frame Relay encapsulated packet fragments into PPP over ATM packets. During receive processing, the fragments that arrive out of order and the packets with missing fragments are discarded. Valid fragments are merged in memory until the entire packet is reassembled. 16-22

Chapter 16 Multilink PPP-Based Link Fragmentation and Interleaving Fragment Size Calculation for MLP Over Frame Relay-Based LFI To calculate the minimum fragment size for MLP over Frame Relay-based LFI, do the following: Step 1 Step 2 Step 3 Calculate the nominal fragment size (link weight) by using the following formula: (Link Bandwidth * Fragment-Delay) / 8 Subtract the Frame Relay encapsulation bytes and the MLP header bytes by using the following formula: Nominal Fragment Size (Frame Relay Encap. Bytes + MLP Header Bytes + Cells Checksum) where: Frame Relay Encapsulation Bytes is 4. MLP Header Bytes is 4. Cells Checksum is 2. If PPP protocol compression is on, subtract 1 byte. For no protocol compression, subtract 2 bytes. For MLP over Frame Relay-based LFI, the minimum fragment size is 56, calculated as follows: (MLP Min. Weight) (PPP Encapsulation Bytes) (MLP Header Bytes) = Min. Fragment Size where: MLP Minimum Weight is 64 PPP Encapsulation Bytes is 4. MLP Header Bytes is 4. Performance and Scalability for MLP Over Frame Relay-Based LFI The following list describes how to enhance performance and scalability for MLP over Frame Relay-based LFI: Configure the following commands and recommended values on the virtual template interface: ppp max-configure 110 ppp max-failure 100 ppp timeout retry 5 keepalive 30 Configure the hold-queue command in interface configuration mode for all Frame Relay physical interfaces. For more information, see the Performance and Scalability chapter in the Cisco 10000 Series Router Broadband Aggregation, Leased-Line, and MPLS Configuration Guide. 16-23

Multilink PPP-Based Link Fragmentation and Interleaving Chapter 16 Restrictions and Limitations for MLP Over Frame Relay-Based LFI The router supports a maximum of one member link per bundle. The member link can be up to 2048 kbps. The router supports a maximum of 2048 bundles per system and a maximum of 2048 member links per system. The valid multilink interface values are 10,000 and higher. Interleaving occurs on only one member link. Interface fragmentation and Frame Relay traffic shaping cannot be configured at the same time on an interface. Do not configure the frame-relay traffic-shaping command on an interface with Frame Relay interface fragmentation configured. The frame-relay fair-dlci queuing command cannot be configured on an interface with Frame Relay interface fragmentation configured. To specify QoS on LFI-enabled interfaces, use service policies (see Chapter 13, Defining QoS for Multiple Policy Levels ). Local Management Interface (LMI) traffic is not fragmented. Cisco IOS software supports a maximum of 4096 total virtual template interfaces. We strongly recommend that you use only strict priority queues when configuring MLP over Frame Relay-based LFI. For more information, see Chapter 8, Prioritizing Services. Configuring MLP-Based LFI Table 16-4 lists the components you must configure for MLP-based LFI. Table 16-4 Configuration Requirements for MLP-Based LFI LFI Type MLP Bundle Member Links Virtual Template Service Policy MLP over Serial-Based Required Required Not Required Not Required LFI Single-VC MLP over ATM-Based LFI Required Required Required Required 1 Multi-VC MLP over Required Required Required Required ATM-Based LFI MLP over Frame Required Required Required Required 2 Relay-Based LFI 1. A service policy with a priority queue defined must be attached to the multilink interface. The VC does not require a service policy. 2. A service policy with the shape command defined must be attached to the VC. A service policy with a priority queue defined must be attached to the multilink interface. 16-24