Design and implementation of a high performance metropolitan multicasting infrastructure

Design and implementation of a high performance metropolitan multicasting infrastructure FRANCESCO PALMIERI Centro Servizi Didattico Scientifico Università degli studi di Napoli Federico II Complesso Universitario di Monte S. Angelo,Via Cinthia 5 NAPOLI - ITALY Abstract: - This paper describes recent implementation efforts and experimental activities regarding the delivery of IP Multicast Services (live video and audio distribution, multiparty conferencing) on the Federico II University metropolitan area network backbone, in Naples, Italy. These services are being used for interactive, live distribution of conferences, meetings, seminars, and lessons, as well as an opportunity for collaborative work via the sharing of applications. Architecture, implementation choices and motivations are discussed in detail to present a successful approach and provide the basic guidelines and implementation framework for building an intranet which supports full multimedia services with scalability. Key-Words: - Multicast routing, Mbone, Multimedia Applications, Protocol Independent Multicasting, RealTime Protocol 1 Introduction Traditional network computing applications involve communication between two computers. However, important emerging applications, such as LAN/IP TV, desktop conferencing, multiparty videoconferencing, shared whiteboard applications, corporate broadcasts, and collaborative computing, require simultaneous communication between groups of computers. This process is known as multipoint communication and it is often obtained by multicast IP. Multicast IP saves bandwidth by forcing the network to do packet replication only when necessary. It offers an attractive alternative to unicast transmission for the delivery of wide interesting network information. Thus one of the major advantages of using multicasting is the decrease of the network load. Today, the majority of Internet applications still rely on point-to-point transmission and the usage of multipoint transmission has traditionally been limited to local area network applications; nevertheless, the number of new applications of Internet that rely on multicast transmission has seen a huge increase over the last few years and the importance of multicast transmission is constantly growing. Multicast IP can also play an important role for the distribution of commercial information and publicity, so that these technologies and applications have the potential to significantly improve productivity. In order to fully support them it will be necessary to build network infrastructures that can implement multicast efficiently. This paper describes recent implementation efforts and experimental activities involving the delivery of IP Multicast Services, including live video and audio distribution, and multiparty conferencing, on the Federico II University metropolitan area network backbone, in Naples, Italy. These services are being used for live distribution, with interactive facilities, of conferences, meetings, seminars and lessons, as well as an opportunity for collaborative work through the sharing of applications. The paper discusses the implementation choices and details and focuses on monitoring and deployment of Quality of Service (QoS) for multimedia applications. Section 2 is devoted to introduce basic concepts that will be used in the rest of the paper. In Section 3 we overview the standards and the state of the art in multipoint networking. In Section 4 we describe the IP multicast

infrastructure we have implemented and discuss how the existing network has been adapted for the co-existence of multicast and classic unicast traffic. At the end of Section 4 we also discuss the issue of QoS delivery, implementation and management. All these aspects and issues are combined to present a successful approach and they provide the basic guidelines and implementation to build an intranet that supports full multimedia services with scalability. Section 5 presents our conclusions. 2 Basic Concepts There are three ways to design multipoint networking applications: unicast, broadcast, and multicast. With a unicast design, applications can send one copy of each packet to each member of the multicast group. This technique is simple to implement, but it has significant scaling restrictions if the group is large. In addition, it requires extra bandwidth, because the same information has to be transmitted multiple times. In a broadcast design, applications send one copy of each packet to a broadcast address. This technique is even simpler to implement than unicast; however, when this technique is used, the network must either stop broadcasts at the LAN boundary (a technique that is frequently used to prevent broadcast storms) or it has to send the broadcast everywhere. Sending the broadcast everywhere is a significant usage of network resources when only a small group of multicast members needs to see the packets. With a multicast design, applications can send one copy of each packet and address it to the group of computers that need to receive it. This technique addresses packets to a group of receivers rather than to a single receiver. It depends on the network to forward the packets only to the sub-networks that need to receive them. Multicasting is not connection-oriented. A multicast datagram is delivered to the members of the destination group with the same "besteffort" reliability as a standard unicast IP datagram. This means that multicast datagrams are not guaranteed to reach all members of a group, nor to arrive in the same order in which they were transmitted. The only difference between a multicast IP packet and a unicast IP packet is the presence of a 'group address' in the Destination Address field of the IP header. Instead of a Class A, B, or C IP destination address, multicasting employs a Class D address format, which ranges from 224.0.0.0 to 239.255.255.255. The notion of group is essential to the concept of multicasting. By definition a multicast message is sent from a source to a group of destination hosts. In IP multicasting, multicast groups have an ID called multicast group ID. Whenever a multicast message is sent out, a multicast group ID specifies the destination group. These group ID's are essentially a set of IP addresses called "Class D". Therefore, if a host (a process in a host) wants to receive a multicast message sent to a particular group, it needs to somehow listens to all messages sent to that particular group. The Internet Group Management Protocol (IGMP) is used to build dynamic associations (membership) between hosts and groups. Whenever a multicast router receive a multicast packet it checks the group ID of the message and forwards the packet only if there is a member of that group in networks connected to it. There are many different algorithms such as "flooding", "spanning tree", "reverse path forwarding", and "reverse path multicasting" for exchanging the routing information among the routers. Some of these algorithms have been used in dynamic multicast routing protocols such as Distance Vector Multicast Routing Protocol (DVMRP), Multicast extension to Open Shortest Path First (MOSPF), and Protocol Independent Multicast (PIM). Based on the routing information obtained through one these protocols, whenever a multicast packet is sent out to a multicast group, multicast routers will decide whether to forward that packet to their network(s) or not. There are several parameters that the network layer must define in order to support multicast communications: Addressing. There must be a network-layer address that is used to communicate with a group of receivers rather than a single receiver. In addition, there must be a mechanism for mapping this address onto data-link layer multicast addresses where they exist.

Dynamic registration. There must be a mechanism for the computer to communicate to the network that it is a member of a particular group. Without this ability, the network cannot know which networks need to receive traffic for each group. Multicast routing. The network must be able to build packet distribution trees that allow sources to send packets to all receivers. A primary goal of these packet distribution trees is to ensure that each packet exists only one time on any given network. 3 Standards Taxonomy The Internet Engineering Task Force has been developing standards that address each of the issues described above. 3.1 Addressing The IP address space is divided into four pieces: Class A, Class B, Class C, and Class D. Classes A, B, and C are used for unicast traffic. Class D is reserved for multicast traffic. Class D addresses are allocated dynamically. router can decide whether to forward multicast messages it receives to its subnetwork(s) or not. After receiving a multicast packet sent to a certain multicast group, the router will check and see if there is at least one member of that particular group on its subnetwork. If that is the case the router will forward the message to that subnetwork. Otherwise, it will discard the multicast packet. Obviously this will be the last phase of delivering a multicast packet. 3.3 Multicast Routing 3.3.1 Algorithms Several algorithms have been proposed for building multicast trees through which the multicast packets can be delivered to the destination nodes. All these algorithms can be potentially used in implementing the multicast routing protocols. The simpler algorithms are Flooding and Spanning Trees. More sophisticated algorithms are Reverse Path Forwarding (RPF), Truncated Reverse Path Forwarding (TRPF), and Core-Based Trees (CBT). 3.3.2 Standards Figure 1: Class D multicast address format 3.2 Dynamic registration RFC 1112 defines the Internet Group Membership Protocol (IGMP) [8] that specifies how the host should inform the network that it is a member of a particular multicast group. A group membership protocol is employed by routers to learn about the presence of group members on their directly attached subnetworks. There are no restrictions on the physical location or the number of members in a multicast group. A host may be a member of more than one multicast group at any given time and does not have to belong to a group to send packets to members of a group. The IGMP is also used by the routers to periodically check whether the known group members are still active. Based on the information obtained from the IGMP the There are several standards available for routing IP Multicast traffic: - RFC 1075 defines the Distance Vector Multicast Routing Protocol (DVMRP) [6]. - RFC 1584 defines the Multicast Open Shortest Path First (MOSPF) protocol [7] -- an extension to OSPF that allows it to support IP Multicast. - Three Internet standards-track drafts describe PIM -- a multicast protocol that can be used in conjunction with all unicast IP routing protocols. These documents are entitled Protocol- Independent Multicast (PIM): Motivation and Architecture [3], Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol Specification [4] and Protocol-Independent Multicast (PIM), Dense-Mode Protocol Specification [5]. DVMRP (RFC 1075). DVMRP uses the Reverse Path Forwarding technique. When a router receives a packet, it floods the packet out of all paths except the one that leads back to the

packet's source. Doing so allows a data stream to reach all LANs (possibly multiple times). If a router is attached to a set of LANs that do not want to receive a particular multicast group, the router can send a "prune" message back up the distribution tree to stop subsequent packets from traveling where there are no members. DVMRP implements its own unicast routing protocol in order to determine which interface leads back to the source of the data stream. This unicast routing protocol is very like RIP and is based purely on hop counts. As a result, the path that the multicast traffic follows may not be the same as the path that the unicast traffic follows. DVMRP has been used to build the MBONE -- a multicast backbone across the public Internet -- by building tunnels between DVMRP-capable machines. The MBONE is used widely in the research community to transmit the proceedings of various conferences and to permit desktop conferencing. Multicast Extended OSPF (RFC 1584). Multicast OSPF (MOSPF) was defined as an extension to the OSPF unicast routing protocol. OSPF works by having each router in a network understand all of the available links in the network. Each OSPF router calculates routes from itself to all possible destinations. MOSPF works by including multicast information in OSPF link state advertisements. An MOSPF router learns which multicast groups are active on which LANs. MOSPF works only in internetworks that are using OSPF. PIM (Draft [3]). Protocol-Independent Multicast (PIM) works with all existing unicast routing protocols. PIM supports two different types of multipoint traffic distribution patterns: dense and sparse. Dense mode is most useful when: Senders and receivers are in close proximity to one another. There are few senders and many receivers. The volume of multicast traffic is high. The stream of multicast traffic is constant. Dense-mode PIM uses Reverse Path Forwarding and looks a lot like DVMRP. The most significant difference between DVMRP and dense-mode PIM is that PIM works with whatever unicast protocol is being used; PIM does not require any particular unicast protocol. Sparse multicast is most useful when: There are few receivers in a group. Senders and receivers are separated by WAN links. The type of traffic is intermittent. Sparse-mode PIM is optimized for environments where there are many multipoint data streams. Each data stream goes to a relatively small number of the LANs in the internetwork. For these types of groups, Reverse Path Forwarding techniques waste bandwidth. Sparse-mode PIM works by defining a Rendezvous Point. When a sender wants to send data, it first sends to the Rendezvous Point. When a receiver wants to receive data, it registers with the Rendezvous Point. Once the data stream begins to flow from sender to Rendezvous Point to receiver, the routers in the path will optimize the path automatically to remove any unnecessary hops. Figure 2: PIM-SM rendezvous Sparse-mode PIM assumes that no hosts want the multicast traffic unless they specifically ask for it. PIM is able to simultaneously support dense mode for some multipoint groups and sparse mode for others. 3.3.3 The Mbone The Internet Multicast Backbone (MBone) is an interconnected set of subnetworks and routers that support the delivery of IP multicast traffic. The goal of the MBone is to construct a semipermanent IP multicast testbed to enable the deployment of multicast applications without

waiting for the ubiquitous deployment of multicast-capable routers in the Internet. The MBone is a virtual network that is layered on top of sections of the physical Internet. It is composed of islands of multicast routing capability connected to other islands, or "regions," by virtual point-to-point links called "tunnels." The tunnels allow multicast traffic to pass through the non-multicast-capable parts of the Internet. Since the MBone and the Internet have different topologies, multicast routers execute a separate routing protocol to decide how to forward multicast packets The majority of the MBone regions are currently interconnected by the Distance Vector Multicast Routing Protocol (DVMRP). Internally, the regions may execute any routing protocol they choose, i.e., Multicast extensions to OSPF (MOSPF), or the Protocol-Independent Multicast (PIM) routing protocol(s), or the DVMRP. As multicast routing software features become more widely available on the routers of the Internet, providers may gradually decide to use "native" multicast as an alternative to using lots of tunnels. 4 Design Issues Multicasting has been implemented on a live network that currently serves over 10.000 users, for a wide range of scientific and production applications. 4.1 Network Layout A metropolitan star-shaped network based on 2/4 Mbps leased-line connections (will be soon upgraded to 34Mbps ATM on E3 leased lines) has been deployed as illustrated in Figure 4. Multicast routing has been enabled on the central router C7507 (mbone-gw) equipped with the experimental IOS version 12.0S and connected to the MBONE via the national Mbone-IT gateway and on the other area aggregation gateways C7507, C4500, (will be upgraded to 7206VXR) and C14xx, deploying multicast services in their respective areas, corresponding to the most important and large campuses in the metropolitan area. Figure 4: network layout In detail, the networking platform incorporates broadband IP and ATM technology, covering four very large campus aggregations separated by up to 10 km and five medium-sized campuses, variously scattered around the town. These medium-sized sites are connected via ABR PVCS on ATM 25Mbps connections to the central multicast router (mbone-gw) using the metropolitan ATM public backbone, reachable via the C8540 central ATM Switch. 4.2 Mbone Connection The connection to the Mbone is achieved via the Italian Academic and Research Network (GARR) Mbone Gateway (MBONE-IT), with a GRE (Generic route Encapsulation) tunnel passing through the local C7500 (garr-gw), and ATM Switch (C8540) and the GARR backbone routers. For compatibility sake the DVMRP multicast routing protocol is used on this connection. 4.3 Multicast Routing Choices Traditional multicast routing mechanisms (e.g. DVMRP and MOSPF) were intended for use within regions where groups are widely represented or bandwidth is universally plentiful. When group members, and senders to those group members, are distributed sparsely across a wide area, these schemes are not efficient; data packets or membership report information are periodically sent over many links that do not lead to receivers or senders, respectively. This characteristic has brought the Internet community to investigate multicast routing

architectures that efficiently establish distribution trees across wide-area internets, where many groups are sparsely represented and where bandwidth is not uniform due to the distances and multiple administrations traversed. Efficiency is evaluated in terms of the state, control message processing, and data packet processing required across the entire network in order to deliver data packets to the members of the group. For this and other reasons we use PIM-SM as it uses the unicast routing paths to deliver traffic with no dependency on which unicast routing protocol is used. It is an explicit join protocol and supports both shared and source-specific trees, with the switch being determined by the traffic level (packets per second) transmitted on a group basis. To provide a contiguous multicast network, the Rendezvous Point was located on the same router as the MBONE connection. However, due to issues with PIM-SM, the join period was slow (up to 60 seconds). In detail, the Protocol Independent Multicast-- Sparse Mode (PIM-SM) architecture: maintains the traditional IP multicast service model of receiver-initiated membership; uses explicit joins that propagate hop-byhop from members' directly connected routers toward the distribution tree; builds a shared multicast distribution tree centered at a Rendezvous Point, and then builds source-specific trees for those sources whose data traffic warrants it; is not dependent on a specific unicast routing protocol; and uses soft-state mechanisms to adapt to underlying network conditions and group dynamics; The robustness, flexibility, and scaling properties of this architecture make it well suited to large heterogeneous inter-networks. The DVMRP protocol is only used in our connection to the Mbone because, as stated above, it is a multicast network over-laid on the public Internet. The Internet does not natively support multicast so the Mbone provides connectivity by tunneling the multicast data within unicast packets. 4.4 Real Time Performance 4.4.1 Measurements Irregular delivery of multimedia data has effects that are easily detected by users, such as jerky movements or blockiness as video data is presented on their screen. Measuring performance parameters is a way of distinguishing these effects and is also important in developing a network that can perform adequately. Compressed Real Time Protocol (CRTP), that is a link-by-link compression feature of the RTP protocol defined in RFC 1889 [2], is one means of providing real-time feedback on performance and QoS parameters (for example, packets received/lost), reception quality, user names, etc. In fact, it is intended to provide end-to-end network transport functions for applications that support audio, video, or simulation data over multicast or unicast network services and provides support for real-time conferencing of groups of any size within the Internet. This support includes source identification and support for gateways such as audio and video bridges as well as multicast-to-unicast translators. CRTP offers QoS feedback from receivers to the multicast group, and support for the synchronization of different media streams. Figure 5: Compressed Real Time Protocol This enables participating users to be logged, along with their session quality statistics. CRTP sends feedback to the sender and to other recipients of a multicast stream. They can then compare their reception quality with that of others to help isolate problems. 4.4.2 Performance

To provide acceptable performance and service levels for multimedia real-time traffic some QoS tools have been extensively used in the network. Moreover, extended resource control must be used, particularly for bandwidth surveillance to make certain that not only bandwidth and minimum delays required by time-sensitive multimedia and voice applications are available, but also that your WAN is used efficiently by mission-critical applications; and that other applications using the link get their fair service without interfering or being influenced by multimedia and mission-critical traffic. Under these assumptions: multicast capacity has been rate-limited via Committed Acces Rate (CAR) facility to 1/3 of available link bandwidth, multicast traffic has been assigned different classes of service and priorities (IP precedence) by partitioning global network traffic according to the packet classification facility. Traffic has been prioritized according to bandwidth requirement and protocol type using a Weighted Fair Queuing (WFQ) schema, that dynamically divides bandwidth across queues of traffic based on weights calculated on IP precedence, and traffic pattern characteristics. WFQ also provides strict priority queueing available through use of IP RTP Priority. References: [1] S. E. Deering, "Host extensions for IP multicasting," IETF RFC 1112, August 1989. [2] S. Casner, R. Frederick, V. Jacobson, "RTP: A Transport for Real-Time Applications," IETF RFC 1889, January 1996. [3] S. Deering, D. Estrin, D. Farinacci, V. Jacobson, C. Liu, and L. Wei, Protocol- Independent Multicast (PIM): Motivation and Architecture, <draft-ietf-idmr-pim-arch-01.ps>, January 11, 1995. [4] D. Estrin, D. Farinacci, A. Helmy, D. Thaler, S. Deering, M. Handley, V. Jacobson, C. Liu, P. Sharma, L. Wei, "Protocol Independent Multicast-Sparse Mode (PIM-SM): Protocol Specification", IETF RFC 2117, June 1997. [5] D. Estrin, D. Farinacci, V. Jacobson, C. Liu, L. Wei, P. Sharma, and A. Helmy Protocol- Independent Multicast (PIM), Dense-Mode Protocol Specification, <draft-ietf-idmr-pim- DM-spec-01.ps>, January 17, 1996. [6] D. Waitzman, C. Partridge, and S. Deering, Distance Vector Multicast Routing Protocol, IETF RFC 1075, November 1988. [7] J. Moy, "Multicast Extensions to OSPF", IETF RFC 1584, March 1994 [8] W. Fenner, Internet Group Management Protocol, Version 2, <draft-ietf-idmr-igmp-v2-01.txt>, Expires April 1996. [9] J. Reynolds and J. Postel, "Assigned Numbers", IETF RFC 1700, October 1994. (STD 2) Figure 6: Weighted Fair Queueing schema 5 Conclusion The described implementation has been succesfully adopted on a live network that currently serves over 10.000 users, for a wide range of scientific and production applications. [10] V. Jacobson, "Multimedia Conferencing for real-time applications and protocols", Tutorial Notes - ACM Sig-comm 94, London, Sept. 1994. [11] S. Paul, K. Sabnani, D. Kristol, "Multicast Transport Protocols for High-Speed Networks", Proc. 1994 IEEE Int. Conf. Network Protocols, Boston, Oct. 1994.