A method for managing distributed IP packet forwarding in ATM/LANE based networks

Size: px

Start display at page:

Download "A method for managing distributed IP packet forwarding in ATM/LANE based networks"

Piers Randall
5 years ago
Views:

1 A method for managing distributed IP packet forwarding in ATM/LANE based networks Nikola Musa ICT Department, Networking Services Financial Agency (FINA) Ul. Grada Vukovara 70, Zagreb, HR-10000, Croatia Phone: (385)(1) Marko Zec, Mladen Kos Department for Telecommunications Faculty of Electrical Engineering and Computing Unska 3, Zagreb, HR-10000, Croatia Phone: (385)(1) Abstract - Modern Layer-3 LAN switches, that are often installed as edge devices in ATM/LANE based network backbones, posses significant IP packet forwarding capabilities, and thereby have the potential to offload the traditional one-armed inter-elan routers. However, in environments with more than just a few of such LAN switches installed, an efficient utilization of available aggregate IP processing power, distributed on edge devices, can become difficult and inconvenient to achieve. The sub-optimal distribution of CPU intensive IP packet forwarding tasks also results in poor utilization of available aggregate bandwidth, unnecessary packet traversing through the ATM backbone, and introduction of single points of failure on the IP layer. In this paper we present a new method for managing distributed IP packet forwarding in ATM/LANE networks, based on introduction of virtual IP default gateway model and custom filters for each ELAN/IP subnet. We discuss how the proposed method accomplishes an even distribution of IP forwarding tasks on all edge LAN switches, and describe our referent implementation on IBM ATM / LAN networking equipment. I. INTRODUCTION A common enterprise ATM LAN Emulation (LANE) [1] based campus/lan environment consists of an ATM switched backbone, one or more LAN Emulation Configuration Server (S), LAN Emulation Server () and Broadcast and Unknown Server () entities, and a number of legacy (usually ) edge LAN switches, connected to the ATM backbone. Driven primarily by the enterprise internal organizational structure, an overlay of numerous logical virtual LANs (VLANs) is usually built upon the underlying hardware infrastructure. The LANE technology allows a single VLAN to be conveniently and transparently spread around the whole ATM backbone, utilizing the possibility of direct bridging of VLANs to appropriate ELANs. Each VLAN/ELAN pair will usually form a unique IP subnet. For accessing the hosts outside its own subnet, each station relies on an IP packet forwarding device, which it references as its default gateway, to perform functions of IP packet routing and forwarding to the other portions of the network [2]. In this article we briefly discuss the two most common approaches for implementing inter-vlan IP packet forwarding in LAN Emulated environments, and outline their benefits and weaknesses. We introduce a new model for managing IP packet forwarding in such environments, based on the concept of virtual IP default gateway, and describe how filters should be designed to support the virtual IP default gateway operation. Following a short description of our referent implementation, we conclude our report by outlining the main benefits in application of our method in complex LANE environments. II. CENTRALIZED ROUTING MODEL In a sample ATM/LANE based network model, as shown in Fig. 1., end nodes are distributed in a number of separated logical VLANs, which spread over the whole physical network span. Each VLAN segment is bridged through the appropriate ATM ELAN, so that the end nodes in a single VLAN/ELAN can communicate transparently, no matter weather they are connected to the same LAN switch, or bridged through the ATM/LANE backbone. In the model shown in figure 1., all inter-vlan IP packet forwarding is accomplished using one or more central one-armed routers, which have LAN Emulation Client () instances active in each ELAN. Advantages Simple management and maintenance, as all IP packet forwarding, routing and filtering functions are concentrated in a single point;

2 IP routing central IP router ELAN ELAN S ELAN L2 switch #1 L2 switch #2 L2 switch #3 LAN LAN LAN LAN LAN LAN LAN LAN LAN Fig. 1. Centralized IP packet forwarding among ELANs/VLANs using one-armed router. Various clustering options available for increased availability and reliability, such as Virtual Router Redundancy Protocol (VRRP) [3] and LANE redundant IP gateway [4]. Weak points The central router or cluster of routers represents a single point of failure for all IP traffic to non-local subnet destinations; In environments with large number of edge LAN switches, this model doesn t scale well: Router s single link to the ATM backbone has a finite capacity, and can therefore present a bandwidth bottleneck; Under high packet rates and/or complex IP processing tasks, such as packet filtering, the CPU on a single IP forwarding device can become overloaded, leading to overall network throughput degradation. III. DISTRIBUTED ROUTING ON EDGE SWITCHES A different approach, shifting the IP packet forwarding functions to edge LAN switches, is shown in Fig. 2. Switches capable of traffic processing on the third OSI layer, such as IP packet forwarding, are offered today by various vendors at a relatively affordable cost. On recent hardware, vendors usually implement many of the Layer- 3 functions in Application Specific Integrated Circuit (ASIC) modules, thus offering very high overall throughput and performance [5]. Benefits and downsides of deploying networks with IP packet forwarding tasks distributed on edge LAN switches can be summarized as follows: Advantages IP processing load is distributed, so aggregate packet forwarding throughput scales well with the number of edge switches; S ELAN ELAN ELAN IP routing L3 switch #1 L3 switch #2 L3 switch #3 LAN LAN LAN LAN LAN LAN LAN LAN LAN Fig. 2. Distributed IP packet forwarding on Layer 3 edge LAN switches

3 There are no bandwidth-related bottlenecks in the backbone, as opposed to the centralized one-armed router model; In a properly configured network, single points of failure on the IP layer can be avoided, as in case of failure of a single edge LAN switch, the remainder of the network will continue to operate uninterrupted; Weak points Management and maintenance becomes more demanding and complex as the number of the LAN switches grow, requiring the introduction of specialized management and automation methods and tools. Although shifting of IP packet forwarding functions from central routers to the edge LAN switches may seem to be as straight-forward as are the mentioned benefits, a couple of problems have to be addressed first, before this concept can be safely deployed in non-forgiving large scale production environments. IV. ISSUES IN DISTRIBUTED IP FORWARDING Let us consider the possible options for IP packet forwarding configuration between VLANs/ELANs in the sample network configuration shown on figure 2. Each station that attempts to send IP packets outside its own subnet, has to pass these packets to its default gateway, which has to be one of the Layer-3 LAN switches. Basically, this can be organized in two different ways: A. For each VLAN/ELAN/IP subnet, only one LAN switch will be chosen and referenced as the default gateway, no matter where the end stations are physically located - directly connected to the LAN ports of that particular switch, or bridged through the ATM/LANE backbone; B. Stations physically connected to the LAN ports of each switch will use unique IP address of that particular switch as its default gateway, ensuring that IP forwarding is always performed as close as possible to the end stations. Method A is simple to configure, deploy and maintain, but it has virtually no advantages compared to the onearmed central router model, described in section II. As all non-local subnet IP traffic for the whole VLAN/ELAN still have to be processed by only one LAN switch, bandwidth and CPU constraints remain unresolved. Additionally, further analysis would show that this approach could raise various reliability and availability issues. Method B enables significantly more even distribution of IP forwarding tasks on all edge LAN switches. This approach will therefore scale well with the large number of LAN switches installed, concerning bandwidth and CPU constraints. Another advantage of method B is avoidance of suboptimal, and sometimes completely unnecessary, packet traversing through the ATM/LANE backbone. This brings a couple of positive effects: preservation of bandwidth, and mitigation of additional segmentation and reassembly (SAR) delays and CPU utilization. Keeping overall end-toend delay at minimum levels can be of critical importance in networks with high bandwidth-delay products. This can be especially true for end stations with IP stacks that do not support or do not have window scale TCP extensions [6] enabled, where excessive delays can result in significant TCP throughput degradation. However, new problems arise with the introduction of large number of different default gateway IP addresses, which are required by implementation of method B. If we consider a network configuration consisting of only N=10 edge LAN switches, and M=10 unique VLANs/ELANs/IP subnets, we will have to manage N*M=100 different default gateway IP addresses. Furthermore, having a different default gateway IP address required for each LAN switch, usage of some convenient management services, such as Dynamic Host Configuration Protocol (DHCP) [7] becomes questionable or even impossible. And even if the absence of DHCP service would be an acceptable option, which is rarely the case in today s large and complex LAN environments, the possibility of unintentional end station IP misconfiguration would still be exceptionally high. Having in mind the problems mentioned above, it is obvious that model B does not scale well in terms of end station IP configuration management. As a solution for overcoming these scalability issues, we propose a new model for managing distributed IP packet forwarding. The new approach is in its concept still closely tied to the described model B, and inherits its main positive aspects good scaling in bandwidth and CPU load distribution. V. VIRTUAL IP DEFAULT GATEWAY MODEL In the previous section we have shown that usage of multiple IP default gateway addresses on a single VLAN/ELAN presents the major obstacle for scaling in network size, as well as for application of additional management services such as DHCP. We therefore introduce a model of single virtual IP default gateway address, which will be unique for the whole VLAN/ELAN, and independent of physical location of end stations. The new method will ensure that although all stations will reference the same default gateway IP address for sending packets to non-local IP subnet destinations, the edge LAN switch, that the end station is physically connected to, will always perform IP packet forwarding for the transmitting station. In order to accomplish this goal, we must implement mechanisms that will enable and enforce the following behavior on each VLAN/ELAN/IP subnet:

4 "Legacy" ATM ELAN LAN switch #1 LAN switch #2 LAN switch #3 Workstation who has DG IP? time ARP reply - sw #1 ARP reply - sw #2 ARP reply - sw #3 who has DG IP? who has DG IP? ARP reply - sw #2 ARP reply - sw #3 who has DG IP? Fig. 3. ARP traffic flow for virtual IP default gateway address without filtering A. Allow all edge LAN switches that have a proxy- active in a specific VLAN/ELAN, to be able to directly reply to an IP Address Resolution Protocol (ARP) [8] request for the virtual default gateway IP address, with their own MAC addresses; B. Ensure that the only LAN switch that will be allowed to reply to uest for the default gateway IP address, will be the switch that the end station, which issued this uest, is physically connected to. The simple solution for the requirement A is making permanent or static proxy-arp entries on each edge LAN switch, which will advertise that switch s own MAC address associated with the desired virtual IP default gateway address, for each VLAN/ELAN/IP subnet. The basic traffic flow in the described proxy-arp scenario is shown in fig. 3., where it can be seen that we have accomplished the fulfillment of requirement A. However, as a byproduct we receive not only the desired IP ARP reply from the LAN switch the transmitting end station is directly connected to, but also unwanted replies from all other LAN switches that have s active in the same ELAN. A solution for mitigation of unwanted excessive IP ARP replies (requirement B) can be found in the nature of LANE implementation of broadcast traffic propagation through the Non-Broadcast Multiple Access (NBMA) ATM network. As the ATM technology doesn t provide the native means for transporting broadcasts, the LANE system relies on the for collecting frames with broadcast destination address, and forwarding them to all s via multicast distribute VCCs, thus effectively emulating broadcast behavior. Some (but not all) of / implementations allow custom filters to be applied on frames received by the, before forwarding them to all s in the ELAN. Depending on capabilities and granularity of filtering criteria, it may be possible to stop selected broadcast frames from being distributed to all s, while allowing all other broadcast traffic to be forwarded in the usual way. IP ARP reply packets are normally sent as unicasts. Therefore we have to focus on IP uest frames, as these are always being sent as broadcasts, and as such can be subject to analysis and filtering at the. Fig. 4. shows the basic IP ARP traffic flow that can be enforced by filtering. A simple filtering rule, also "Legacy" ATM ELAN LAN switch #1 LAN switch #2 LAN switch #3 Workstation who has DG IP? who has DG IP? time ARP reply - sw #1 filtering for DG IP? Yes No Forward frame on all s Drop frame Fig. 4. Filtering uests on results in a single ARP reply, only from the local LAN switch

5 hard size prot size destination addr source addr frame type hard type prot type op sender addr sender IP addr target addr tarfet IP addr header 14 bytes uest / reply 28 bytes FFFFFFFFFFFF * * * * W.X.Y.Z Fig. 5 a. IP ARP packet format for IPv4/ (top); b. IP uest packet for IP address W.X.Y.Z (bottom) outlined in the same figure, must ensure that only ARP requests for the virtual default gateway IP address will be rejected, and other uests, as well as all other broadcast traffic, will be processed and distributed as usual. It is necessary to properly design the filtering rule, which has to test several portions of every frame received by the. An IP ARP packet consists of various fields, as shown in fig. 5a. The fields that are relevant for IP uest packets, that we are interested in, are outlined in fig. 5b. The filtering rule for uest for virtual default gateway IP address has therefore to check that the following conditions are met: 1. Frame type field in header is set to 0806 (16) (ARP); 2. ARP operation field is set to 0001 (16) (ARP request); 3. ARP hardware and protocol type and size fields are set to appropriate values for IPv4 and ; 4. ARP target IP address is equal to virtual default gateway IP address for ELAN. If all conditions (1. to 4.) are true, filter must discard the frame, otherwise it has to allow the frame to be forwarded to all s. Note that it is not necessary to check that destination address is set to FF:FF:FF:FF:FF:FF (broadcast), as uests are always sent as broadcasts by definition. By utilizing proxy-arp functions on all edge LAN switches, combined with the filtering as described above, we fulfilled both the functional requirements A and B from the beginning of this section. Thereby we were ready to proceed with the practical implementation. VI. OUR IMPLEMENTATION In our tests and in later deployment to production environment, the following equipment was used: IBM S ATM switches; IBM Layer-3 LAN switches; IBM routers; IBM 8210 MSS S// servers. We will restrict our example on brief description of filtering implementation on the IBM 8210 Multiprotocol Switched Services (MSS) / servers, as it is the fundamental processing component that enables the introduction of proposed virtual IP default gateway address model into the LANE environment. MSS-zg1 FILTER config for ELAN 'su-it'>show Bus Filter Items Default Action: INCLUDE Preferred List: EXCLUDE LIST Enabled?: YES MAC Filter Items: NONE DEFINED PROTOCOL Filter Items: NONE DEFINED SLIDING WINDOW Filter Items: (1)Name:vdg arp rej su-it Base: MAC Offset: 12 List: EXCLUDE LIST Enabled: YES Value: A1A2A3A4A5A6B1B2B3B4C1C2C3C4C5C60A01C802 Mask: FFFFFFFFFFFFFFFFFFFF FFFFFFFF IP Filter Items: NONE DEFINED Fig. 6. filter configuration on IBM 8210 MSS discards all uests for virtual default gateway address ( ) on ELAN su-it

6 Fig. 6. shows the typical filtering configuration for a single ELAN. On our sample ELAN, named su-it, IP subnet /22 is used. We choose to use as the virtual default gateway address for this subnet. IBM 8210 implementation allows network administrators to define custom sliding window filters, defined by an arbitrary byte string pattern, logical AND mask value, and an offset from the beginning of the data frame. The filtering pattern will be compared to each frame received by the, and appropriate action (drop/pass) will be executed on the first-match basis. The filter shown on Fig. 6. will drop (EXCLUDE action) all uest frames for IP address (0A01C802 in hexadecimal notation), and pass (INCLUDE action) all other traffic unmodified. We have used the described filtering configuration, together with the appropriate proxy-arp static mappings on edge LAN switches, to construct both the laboratory and production ELANs enabled for efficient distributed IP forwarding. The production campus ATM backbone in Financial Agency (FINA) HQ in Zagreb operates reliably with described enhancements in distributed IP forwarding. The network backbone consists of 4 IBM 8265 ATM switches, located in two buildings interconnected with redundant (dark fiber and free space laser) OC-3c 155 Mbps links. More than 1200 end stations are concentrated on 28 Layer- 3 edge switches, connected to the ATM backbone. The network is logically divided in separated ELANs, of which the major 8 are configured for distributed IP forwarding using the described virtual default gateway model. Detailed description of this installation can be found in [10]. bandwidth savings, and lower average end-to-end delays. Furthermore, moving IP forwarding functions to edge LAN switches can also result in more efficient placement of ingress points in more advanced ATM shortcut routing systems, such as NHRP or MPOA [9]. However, this can be observed only on edge LAN switches that support one or both of the mentioned protocols. Unfortunately, not all vendors offer the necessary filtering functionality in their / implementations. Therefore, as our model relies heavily on filtering, it will not be possible to deploy it in environments with such / systems installed. The proposed model has proven to be stable and reliable in both laboratory and production environments. The future research could include overall network performance evaluation with and without the proposed extensions, in various network configurations. The impact of additional processing, as imposed by our method, on overall throughput and performance, should be analyzed in more detail, especially prior to deployment to LANE environments with intensive broadcast/multicast traffic patterns. VIII. ACKNOWLEDGEMENTS The authors would like to thank the whole ZAP-LAN project team for all the patience and time spent in the joint efforts in designing and implementation of the campus ATM backbone in FINA (former ZAP). VII. CONCLUSION We have presented a new model for managing distributed IP forwarding in LANE based environments. Main benefits of our method can be summarized as follows: Efficient and even distribution of IP packet forwarding tasks in LANE environments with large number of edge LAN switches; Removal of bandwidth and CPU constraints associated with traditional one-armed routing model, resulting in good aggregate throughput scaling; Simplified IP configuration of end stations, by retaining the possibility of application of DHCP services, in environment with multiple IP packet forwarding devices on a single IP subnet; Improved overall network reliability and availability, by mitigation of single points of failure on the IP layer; Avoidance of unnecessary and sub-optimal packet traversing through the ATM backbone associated with one-armed router model, resulting in IX. REFERENCES [1] The ATM Forum: LAN Emulation Over ATM - Version 1.0, af-lane , [2] W. Stevens: TCP/IP Illustrated, Vol. 1: The Protocols, Addison-Wesley, [3] S. Knight et al.: Virtual Router Redundancy Protocol, IETF RFC 2338, [4] Nways Multiprotocol Switched Services - Configuring Protocols and Features, Volume 1, IBM Corporation, [5] M. Kos, A. Bazant: Introduction to ATM, 2 nd edition, MIPRO, [6] V. Jacobson, R. Braden, D. Borman: TCP Extensions for High Performance, RFC 1323, [7] R. Droms: Dynamic Host Configuration Protocol, IETF RFC 2131, [8] Plummer: An Address Resolution Protocol, IETF RFC 826, [9] U. Black: ATM - Vol. III: Internetworking with ATM, Prentice Hall, [10] ZAP LAN Project Documentation, IBM/ZAP internal, (in Croatian), unpublished

ATM in TCP/IP environment: Adaptations and Effectiveness

Bremen Institute of Industrial Technology and Applied Work Science ATM in TCP/IP environment: Adaptations and Effectiveness Dipl.-Ing. Kai-Oliver Detken, BIBA ATM Traffic Symposium, Mykonos, Greece, September