Designing an Exchange 2000/2003 Routing Group Connector Topology

Transcription

1 Pg. 1 Designing an Exchange 2000/2003 Routing Group Connector Topology By: Craig Borysowich Chief Technology Architect Imagination Edge Inc. Version 3.7

2 BACKGROUND Large Exchange 5.5 environments use X.400 connectors to route both mail and replication traffic. Most implementations have selected a Hub and Spoke design for their X.400 topology. Most servers have an X.400 connector directly to the hub, but some sites will go through a Bridgehead X.400 connector to the Exchange routing hub. OBJECTIVE As migrations into Exchange 2000/2003 environments increase, we need to design an efficient way to route mail messages while managing overall governance. The SMTP mail routing is required for the following: 1. Within the Sites (Intra-Site), 2. Inter-Site 3. Internet Mail Microsoft Exchange 2000 enables great performance improvements with its message routing when compared to Exchange 5.5. One of the improvements is the ability to split Exchange 5.5 sites into Routing Groups and Administrative Groups. Routing Groups are based upon network topology and are designed to enhance the efficiency of message delivery. Exchange 2000 also utilizes SMTP to deliver messages as its preferred protocol instead of X.400. It is anticipated that each Exchange site or group of sites will develop a mail routing topology and strategy in concert with the rest of the Exchange sites in an organization as an integral part of the overall mail routing design. In cases where an organization has distributed management and administration of their Exchange Infrastructure, a team should be established to agree to a set of standards and a common design approach. This team should include members that are familiar with the current LAN/WAN topology, current Active Directory implementation and Design, and current Exchange 5.5 background for the organization. Page. 1

3 Exchange 2000/2003 routing basics Terminology Routing Topology The routing topology denotes the overall interconnections that multiple sites in an Exchange organization route mail between themselves and with the Internet and other foreign mail systems. Routing Group (RG) A Routing Group is the management unit in the Routing Topology focused on systems that are: always connected; connected via reliable/always available network links; configured for point-to-point connectivity Administrative Group (AG) An Administrative Group is an object management unit where the objects that can be managed include: servers, routing groups, folders, etc. The Administrative Groups can be delegated to manage any grouping of objects. So, Administrative Groups are created for managing objects. The Routing Groups are created for managing the Routing Toplogy. Routing Groups must be created within an Administrative Group they are subordinate to an Administrative Group. A single Administrative Group could contain all of the Routing Groups with other Administrative Groups managing other server and service components. Servers can be Drag & dropped between Routing Groups. Routing Group Master (RGM) Each Routing Group contains a Routing Group Master, which is the first server added to a Routing Group. The Routing Group Master Role can be changed by an administrator. The Master co-ordinates changes to the Link State information within the Routing Group. Link State Information (LST) Information about the state of messaging routes (links) in an Exchange 2000/2003 messaging system derived from the link state algorithm to quickly and frequently calculate the state of system links for up-todate status about routes. Exchange 2000 servers use link state information to make the best routing choice at the source rather than sending a message down a path where a link is not working. This eliminates message bounce and looping. Bridgehead Server (BH) The Bridgehead Server is an Exchange server that connects to other Exchange servers using the same communications protocols so information can be passed from one server to another. In Exchange 2000, a bridgehead server is a connection point from a routing group to another routing group, remote mail system, or other external mail system. Page. 2

4 Routing Group Connector (RGC) A connector that specifies the connection of a local routing group to a server in a remote routing group. It also specifies the local bridgehead server, if any, and the connection cost, schedule, and other configuration properties. All of the Bridgehead servers in a Routing Group communicate their connectivity information to the Routing Group Master, but only for changes. The Master then maintains/rebuilds the link state information and communicates it to all of the servers in the Routing Group. If the Master goes offline, then the servers continue as normal based on the last report of the Link State information received from the Master. The normal flow of mail within a Routing Group will also pass Link State Information. The Connectivity between all servers within a Routing Group is SMTP. For link state updates, Microsoft uses a proprietary binary data protocol that does not use the SMTP virtual Server. The Microsoft Exchange Routing Service handles all routing updates within the Routing Group. This service executes within the INETINFO.EXE and each Bridgehead server binds to the Master on TCP port 691 and vice versa to all servers in the Routing Group for link state updates. Connectivity between Routing Groups is via a Bridgehead server. A Simplified overview of Message routing in Exchange 2000/2003 follows: Page. 3

5 Fig 1: Simplified Message Flow for Exchange 2000/2003 Page. 4

6 The following three routing methodologies should be considered when designing a routing topology: 1. Full Mesh Topology 2. Limited Mesh Topology 3. Modified Hub & Spoke Topology. TOPOLOGY Pros Cons Full Mesh 1. While this topology offers greater efficiency of mail routing, this topology is highly dependant on the underlying network topology - If underlying Network topology is Hub-and-spoke then mesh does not add as much efficiency as it could. It does allow for redundant paths and will bypass the overhead required of the HUB to route the messages.). 2. No single point of failure 3. Most redundant topology mail routes calculated and delivered based on OSPF, dynamically modified when outages in bridgeheads occur thus taking advantage of performance characteristics. Limited Mesh Modified Huband- Spoke 1. While this topology offers greater efficiency of mail routing, this topology is highly dependant on the underlying network topology - If underlying Network topology is Hub-and-spoke then mesh does not add much efficiency as it could. It does allow for redundant paths and will bypass the overhead required of the HUB to route the messages). 2. No single point of failure 3. Semi-redundant topology- mail routed over various paths due to performance characteristics 1. Offers optimum governance in managing: a. Virus outbreaks b. Spam c. Tracing mail 2. Reduces the number of connectors from the site bridgehead server to the hub. 1. Lacks adequate governance to manage: a. virus, b. spam c. mail flow trace d. and other threats to mail environment. 2. An equal partner in mail flow and Exchange 2000 from functional perspective (Site bridgeheads could be routing other sites mail if redundant bridgeheads are not implemented for each site) 1. Moderate governance to manage: a. virus, b. spam c. mail flow trace d. and other threats to mail environment. 2. An equal partner in mail flow and Exchange 2000 from functional perspective (Site bridgeheads could be routing other sites mail if redundant bridgeheads are not implemented for each site) 1. Limited to risk to intra-site and internet mail flow if both servers became unavailable. Mail within a site would not be impacted. 2. Would require a centralized redundant environment to offer high service availability. 3. Follows old style 5.5 design models. Routing improvements in Exchange 2000/2003 should eliminate the need for this model. Page. 5

7 ROUTING GROUP TOPOLOGY RECOMMENDATION Considerations Whether you are coexisting and migrating from an existing Exchange 5.5 organization or building a new organization, you should take the following design considerations into account when selecting a topology: In native mode, Exchange 2000 can be extremely dynamic because you can change the memberships of the routing groups and administrative groups. This is useful when administration models or network topologies change. Exchange 2000 provides load balancing in the form of a round-robin DNS between servers, both sources and targets. A round-robin DNS is a mechanism that directs incoming requests to servers on a rotating basis. This is done by looping through a list of IP addresses belonging to the servers in the configuration. When an client attempts to access a mailbox on an Exchange server, the client is given the first IP address on the list. The second client request is given the second IP address in the list, and so on. If there are four servers on the round-robin list, all four IP addresses are used before the first IP address is used again, and the loop starts over. In addition, Exchange 2000 offers improvements over the Exchange 5.5 Site Connector if one of the source bridgehead servers is down Exchange connectors automatically try not to use that server until it comes back up. If there are multiple connectors with the same cost, each server picks a random connector and uses it for a period of time. Over multiple servers, this functionality simulates round-robin behaviour. When a client must use an alternate server to access public folder content, Exchange 2000 uses routing groups to calculate the closest available server, which it determines by using a cost property set on the Routing Group connector. The cost for each Routing Group connector is stored in a single cost database that is shared with routing calculations. Redundant cost tables maintained in previous versions of Exchange are eliminated. In Exchange 5.5, site affinities were not transitive. For example, if you establish an affinity between site 1 and site 2, and between site 2 and site 3, you do not automatically get affinity between site 1 and site 3. In Exchange 2000, affinities are transitive. If all routing groups in an organization are connected to allow to flow, all servers receive public folder referrals. If a server does not contain replicas of public folders, you can mark specific Routing Group connectors to deny public folder referrals; the client contacts the next server in the referral list. Because SMTP is used in a routing group and SMTP is more resilient over slow links, routing groups can span slow networks more effectively than RPC, which is the native transfer mechanism for Exchange 5.5. Thus, your organization may be able to deploy fewer routing groups by using Exchange 2000 than by using sites with Exchange 5.5. SMTP mail is not compressed; though you reduce network bandwidth requirements with SMTP, the increased load on the Exchange server CPU would reduce server performance. Unlike Exchange 5.5, where intra-site and inter-site RPC communications are encrypted, SMTP communications in Exchange 2000 are not encrypted. However, each server uses SMTP authentication with Kerberos. There are two options for encryption, although neither is done by default: Internet Protocol security (IPSec), which is built into Microsoft Windows 2000, and Transport Layer Security (TLS) encryption, which is built into the SMTP service. Upgrading an Exchange 5.5 hub, in a hub-and-spoke design, can significantly increase hub performance and stability because performance is enhanced by Exchange 2000 and the use of SMTP. Link state routing is most effective in an organization with multiple routing groups using multiple paths. In a traditional hub-and-spoke design, you do not see as much improvement as you do in a mesh network or a network with multiple connections. Page. 6

8 Adding routing groups increases the size of the link state table and increases potential link state status messages. This can increase the amount of link state message traffic. Exchange no longer includes a Dynamic RAS connector. Instead, Exchange 2000/2003 uses the on-demand capabilities in Windows 2000/2003, specifically, the combination of the Routing Group connector and the Routing and Remote Access demand-dial interface. Exchange 2003 Specifics Microsoft highlights that Exchange 2003 further improves upon the link state routing features of Exchange 2000 by reducing the amount of link state traffic in two ways. Performance is improved over what are commonly known as oscillating links, or links that are intermittently available and unavailable. Exchange 2003 reduces link state traffic by attempting to determine if the connector is oscillating. If there are multiple conflicting state changes in the link state queue for a connector in a given interval, the connector is considered an oscillating link and its link state remains up (in service or available for that period). Leaving an oscillating connector up rather than continually changing the link state reduces the amount of link state traffic that is replicated between servers. Performance is improved in the case of a site to which there is only one route. In this case, Exchange 2003 reduces link state traffic by determining that no alternate path exists and suppressing the link state information. If no alternate path exists for a link, the link state is always marked as up (in service). Exchange lets mail queue for delivery and sends it when the route becomes available. Both of these changes enhance performance because they reduce the propagation of link state information. Other than the above, there is not much in the Exchange 2003 product that would really impact your routing topology decision. Routing Group Topology Preference In evaluating the various topologies, an organization should prioritize moving to the full-mesh topology. Scaling back to the limited mesh topology if necessary, and then falling back to a 5.5 style hub/spoke topology only in the most dire of circumstances. Any perceived benefits in the hub/spoke topology to aid in combating virus or spam outbreaks are essentially mythical. In a properly designed and managed Exchange 2000/2003 environment, you should be able to halt routing in the mesh faster and more easily than in a hub/spoke implementation. Final Notes on Routing Group Connection Options Assuming that the Full Mesh Routing Group Topology that is recommended above is accepted, we must now look at how the Routing Group(RG) Connectors are going to be configured. In current X.400 connection topologies, sites have either a single or a redundant connection with the Hub, if this connection goes down, outbound messages from the site are queued locally until the connection is back up. As well inbound messages to the site are queued at the hub until the connection again becomes available. This configuration has a single point of failure to each site as well as a single point of failure at the hub. Full Mesh topology removes the need for running a hub as a single point of failure and allows for mail to find an operation route through a myriad of avenues. Remember that is a communications tool and that all parties within all Exchange sites should want mail to get to its destination. Even if that means that a few microseconds of CPU time has to be spent forwarding mail between two sites that have lost a direct connection. Page. 7

9 Fig 2: Detailed Exchange Message Flow Diagram Page. 8

10 Planning Routing Groups A routing group must consist of servers in the same forest. In fact your entire Org must be contained in the same forest. Major mail routing issues will occur if you try to split an Org between multiple forests. A permanent connection must exist between all servers participating in the same Routing Group. Servers that are on intermittent or on-demand connections must be isolated in their own Routing Group. All servers within the routing group must be able to contact the Routing Master. When you use multiple Routing Groups, you need to have connectors (i.e. bridgeheads) between two servers in those routing groups. All mail traffic being routed will go in and out through the bridgehead. You can have multiple servers in a Routing Group acting as bridgeheads to other Routing Groups. Exchange 2000/2003 will load balance the connections after a period of time and if one connection goes down, Exchange will re-select an operational route. In most cases the Routing Groups should be connected and configured as a Routing Group Connector. In cases where you are routing mail to through the Internet, then you connectors should be configured as SMTP. Only when routing to a legacy X.400 Administrative Management Domain (ADMD) or metadirectory should you use the X.400 Routing Group Connector configuration. Configuring the Routing Group Connector in RGC mode is protocol independent with SMTP native mode and RPC available for 5.5 co-existence. The connector is Directory aware for the Remote Bridgehead server. Securing Mail Routing Security for routing mail can be a big issue for many organizations. Legacy X.400 connections were obfuscated, but not encrypted. SMTP connections are not encrypted, but also obfuscated using Transport Neutral Encapsulated Format. There are two ways to secure mail routing: SMTP encryption/authentication; or using IPSec. It is highly recommended to look at using IPSec for transport security between Exchange Servers. Not only with this secure and encrypt your mail being routed between sites, but it will secure all IP-based communications between your Exchange servers. IPSec can play one of three Roles on a server: -Basic IP port filtering - like a firewall on the server -Authenticating Hosts before and during communications -Encrypting data exchanged between servers IPSec can also perform all three of these functions on a server at the same time. Page. 9