SOME PRINCIPLES FOR DESIGNING A WIDE-AREA OPTICAL NETWORK

Transcription

1 SOME PRNCPLES FOR DESGNNG A WDE-AREA OPTCAL NETWORK Biswanath Mukherjee, S. Ramamurthy, Dhritiman Banerjee Department of Computer Science University of California Davis, CA mukherj e,ramu, banerj ed@cs.ucdavis.edu Amarnat h Mukherjee College of Computing Georgia nstitute of Technology Atlanta, GA amarnat h@cc.gatech.edu Abstract We explore design principles for next generation optical wide-area networks, employing wavelength-division multiplexing (WDM), and targeted to nationwide coverage. This almost-all-optical network will exploit wavelength multiplezers and optical switches in routing nodes, so that arbitmry virtual topologies may be imbedded on a given physical network. The virtual topology, which is packet switched and which consists of o set of all-optical lightpaths, is set up to ezploit the relative strengths of both optics and electronics - viz. packets of information are carried by the virtual topology as far as possible in the optical domain, but packet forwarding from lightpath to lightpath is performed via electronic switching, whenever required. Algorithms ure developed so that the WDM-based network architecture will (a) provide a high aggregate system capacity due to spatial reuse of wavelengths, and (b) support a large and scalable number of users, given a limited number of wavelengths. We illustrate our approaches by employing ezperimental trafic statistics collected from NSFNET. 1 ntroduction New research directions are needed for the development of nationwide all optical networks. These networks should be able to tap into the tremendous transmission bandwidth potential of a single strand of fiber (on the order of a few terabits per second) as well as exploit other attractive properties of optics. However, they must also allow mechanisms by which end-users operating with slower (opto)electronic components (most likely operating in the gigabits-per-second range) can interface with the network. To correct this bandwidth mismatch, wavelength-division multiplexing (WDM) is a key technique for allowing all of the end-users equip ment to operate at only electronic speeds. By using WDM, the fiber can carry a high aggregate system capacity of multiple parallel channels, with each channel operating at much slower speeds [8, 16, 20, 21, 231. B. Mukherjee, S. Ramamurthy and D. Banerjee were supported by NSF Grant No. NCR A. Mukherjee was supported by NSF Grant No. NCR Our objective is to explore new architectures for the next generation of networks employing optical technology for wide-area (nationwide) coverage. We examine an almostall-optical wide-area WDM network architecture which utilizes wavelength multiplexers and optical switches in a routing node, so that an arbitrary virtual topology can be imbedded on a given physical fiber network. The virtual topology, which is packet switched and which consists of a set of alloptical lightpaths, is set up to exploit the relative strengths of both optics and electronics - viz. packets of information are carried by the virtual topology as far as possible over the same wavelength in the optical domain (i.e. there is no wavelength conversion in a lightpath), but packet forwarding from lightpath to lightpath is performed via electronic switching, whenever required. As alternatives to this packet-switched virtual-topology architecture, other approaches have also been proposed, e.g., all-optical packet switches (the Quadro approach) [lo] and circuit-switched architectures (with lightpaths set up on demand) [2, 13, 241. While all three approaches have merit, we proceed with the virtual topology architecture since algorithms for current packet-switched networks can be easily adapted for this approach. This architecture is a combination of the well-known single-hop [8, 9, 15, 201 and multihop [l, 4, 5, 6, 11, 12, 17, 18, 21, 23, 251 approaches employed in several WDM networks/proposals, and it attempts to exploit the characteristics of both approaches. A lightpath in this architecture provides single-hop communication. However, by employing a limited number of wavelengths, it may not be possible to set up lightpaths between all user pairs on the network [7]; as a result, multi-hopping between lightpaths may be necessary. n addition, when the prevailing traffic pattern changes, a different set of lightpaths forming a different multihop virtual topology may be more desirable. A networking challenge is to perform the necessary reconfiguration with minimal disruption to the network s operations. n this architecture, using wavelength multiplexers instead of a passive star has the advantages of higher aggregate system capacity due to spatial reuse of wavelengths d.l.l W94 $ EEE

2 (which we refer to as course-grain space-division multiplexing (SDM)), and larger number of users given a limited number of wavelengths. On a microscopic level, we intend to optimize the cost, efficiency and throughput of the routing node by theoretical analysis. On a macroscopic level, we intend to investigate the overall design, analysis, and op timization of a nationwide WDM network consistent with device capabilities, e.g., aimed at upgrading the current NSFNET. Section 2 explains the system architecture including motivation, a statement of the general problem, and an illustrative example. The problem is formulated as a combinatorial optimization problem in Section 3. Since the problem is conjectured to be NP-hard, heuristic approaches for its solution are developed in Section 4. Details of our approches to obtain the experimental data are provided in Section 5. Results of appljing the experimental data to our algorithms are discussed in Section 6. Section 7 concludes the paper with a discussion on future research. 2 System Architecture 2.1 Motivation To motivate this problem, consider the National Science Foundation Network (NSFNET) T1 backbone. The backbone consists of over a dozen nodes, each of which contains one or more computers operating as packet switches. These switches are connected with one another via optical fibers to form an irregular mesh structure. Store-and-forward packet switching is performed at the nodes. Although fiber is employed to connect the various nodes, the tremendous transmission bandwidth of the fiber is not exploited since data transmission on each fiber link is performed only at T1 (1.544 Mbps) rates. While WDM networks are being investigated, the Asynchronous Transfer Mode (ATM) network solution has been making rapid progress in parallel. Because of the tremendous investment already made in the development of ATM, it is presumable that wide-area, nationwide networks such as the NSFNET may also soon start supporting ATM. n addition, with the advent of the National Research and Education Network (NREN), many networking issues may change drastically; however, we expect that the underlying physical network will continue to be a fiber plant on which our WDM solutions will still be applicable. Accordingly, our research on all optical networks is fueled by the following observations. Any future technology must be incrementally deployable. That is, instead of scrapping an existing and operational wide-area fiber-based network such as NSFNET, we examine how such networks can be u p graded to support WDM. n this regard, we show later how one might replace (or upgrade) the existing packet switches in NSFNET with wavelength-routing switches (WRS). 2.2 General Problem Statement The problem of imbedding a desired virtual topology on a given physical topology (fiber network) is formally stated in greater detail below. We are given the following inputs to the problem. 1. A physical topology G(p) = (V,E(p)) consisting of a weighted undirected graph, where V is the set of network nodes, and E(p) is the set of links connecting the nodes. Undirected means that each link in the physical topology is bidirectional. nodes correspond to network nodes (packet switches) and links correspond to the fibers between nodes; since links are undirected, each link may consist of two fibers or two channels multiplexed (using any suitable mechanism) on the same fiber. Links are assigned weights, which may correspond to physical distances between nodes. A network node i is assumed to have a D(i) x D(i) wavelengthrouting switch (WE), where D(i), called the degree of node i, equals the number of links emanating out of (and terminating at) node i. 2. An N x N traffic matrix, where N is the number of nodes in the network, and the (i,j)-th element of the traffic matrix is the average traffic flow from node i to node j. Flow from node i to node j may be different from the flow from node j to node i. 3. The number of wavelength-tunable lasers (transmitters) and wavelength-tunable filters (receivers) at each node. Our goal is to determine the following. 1. A virtual topology G(v) = (V,E(v)) as another graph where the outdegree of each node is the number of transmitters at the node and the indegree of each node is the number of receivers at the node. The nodes of the virtual topology correspond to the nodes in the physical topology. Each link between a pair of nodes in the virtual topology corresponds to a direct all-optical lightpath between the corresponding nodes in the physical topology. 2. A wavelength assignment for the links of the virtual topology such that, if two lightpaths share a common physical link, they must necessarily have different wavelengths. 3. The sizes and configurations of the wavelength-routing switches at the intermediate nodes. Once the virtual topology is determined and the wavelength assignments have been performed, the switch sizes and configurations follow directly. Communication between any two nodes now takes place by following a path (a sequence of lightpaths) from the source node to the destination node on the virtual topology. Each intermediate node in the path must perform (1) an electro-optic conversion, (2) electronic routing (possibly ATM switching, if needed), and (3) optical forwarding onto the next lightpath. Below, we provide an illustrative example to highlight this architecture, and later in Section 3, we pose the problem formally as an optimization problem. 2.3 An llustrative Example The NSFNET s T1 backbone consisted 14 nodes, with links connecting most of these nodes to one another, as well 1 d.l.2 111

3 , as a few links connecting to the outside world. For illustration purposes, we supplement this physical topology by adding two fictitious nodes, AB and XY, to capture the effect of NSFNET s connections to Canada s communication network, CA*net. Node XY is connected to thaca (NY) and Princeton (NJ) nodes of NSFNET, while node AB is connected to the Seattle (WA) and Salt Lake City (UT) nodes, where we have employed the last link as a fictitious one to render the physical topology richer and fault tolerant. As a result, we arrive at the modified physical topology, hereafter referred to as the physical topology, of Fig. 1. For illustration purposes, we choose a regular virtual topology, which, because of its well-defined structural pattern, requires little proceasing time for routing purposes; this may be a desirable property for high-speed networks which require fast packet processing at intermediate nodes. For example, instead of examining the entire destination address, an intermediate node in a regular structure can perform packet forwarding by processing only a subset of the destination s address. Although one could have an irregular structure for a virtual topology ATM network since the latter does not constrain itself to a particular structure, we envision that other services or subnets may be imbedded on the WDM solution as well, and the ATM subnet can operate on any virtual structure, including a regular structure. While this issue needs a detailed investigation, for purposes of illustration we assume that the physical topology of Fig. 1 has to be imbedded onto a 16-node hypercube virtual topology (e.g., Fig. 2). An embedding which results from the optimization criteria of Section 4.3 is shown in 1. n general, our approaches can be applied to other, possibly arbitrary, virtual topologies as well. Note that each virtual link in the virtual topology is a lightpath or a clear channel with electronic terminations at its two ends only. For example, the CAl-NE virtual link in Fig. 1 could be set up as an all-optical channel on one of several possible wavelengths on one of several possible physical paths, e.g., CA-UT-CO-NE, or CA-WA-L-NE, or others (see Fig. 1). According to our optimization solution outlined in Section 4.3, the first path is chosen on wavelength 2 for this CA-NE lightpath. This means that the WRSs at the UT and CO nodes must be properly configured to establish this CA1-NE lightpath. For example, the switch at UT must have wavelength 2 on its fiber to CA1 connected to wavelength 2 on its fiber to CO. Note that these connections are reciprocal since the virtual topology in this case is also an undirected graph, i.e., the CA-NE connection implies two directed lightpaths, one from CA1 to NE and the other from NE to CA. The solutions in Fig. 1 requires a maximum of 7 wave- Previous work on hypercube embedding has been reported in ll]. However, our work differs signifiurntly in the following ways: (1) we employ actual fiber distances between nodes and minimize the average message delays rather than hops; (2) we develop new heuristic algorithms based on simulated annealing and revers8e-order-refinement (see Section 4); and (3) we evaluate our approaches by using experimental traffic statistics collected from NSFNET. lengths per fiber. However, lower-cost solutions for slightly reduced performance may also be possible to obtain, e.g., by setting up some lightpaths that are not necessarily routed over the shorest physical distance. For the solution in Fig. 1, the details of a WRS, viz. the one at UT, are shown in Fig. 3. Note that this switch has to support four incoming fibers plus four outgoing fibers, one each to nodes AB, CA1, CO, and M, as dictated by the physical topology of Fig. 1. Not all wavelengths on all fibers are utilized, e.g., only wavelengths 2 and 4 are need on the CA1-UT fiber, while wavelengths 1, 2, 3, and 5 are used on the UT-CO fiber. n general, each switch also interfaces with four lasers (mputs) and four filters (outputs), with each laser-filter pair dedicated to accommodate ea& of the four virtual links which a node has to support on the virtual topology. However, in this example, the virtual topology embedded is an incomplete hypercube with nodes AB and XY considered non-existant. Hence some nodes, including UT, have fewer than 4 neighbors. For our present solution, three laser-filter pairs at UT need to be operated - one on wavelength 1 (for connection to physical neighbor CO, to support the lightpath UT-TX); another on wavelength 4 (for connection to physical neighbor CA, to support the lightpath UT-CA); and a third on wavelength 5 (for connection to physical neighbor CO, to support the lightpath UT-CO) (see Fig. 1). Labels on the switch s output ports indicate which wavelength on which input fiber or local laser is connected to which wavelength on which output fiber or local filter. For example, the labels 11 2b 3d 51 on the output fiber to CO indicate that the UT-CO fiber uses four wavelengths 1, 2, 3, and 5, with wavelengths 2 and 3 being clear channels through the UT switch and directed to the physical neighbors CA1 and M, respectively, while wavelengths 1 and 5 connect to two local lasers. The design of the UT switch can take several forms. First, the wavelengths on each incoming fiber need to be separated using a grating demultiplexer. And, finally, information from multiple WDM channels will need to be multiplexed before launching them back on an output fiber. n between, several different solutions to the switch design are possible. A totally-hardwired solution would require direct connections from each demux ouput and local laser to each mux input and local filter. Unfortunately, this approach is not traffic sensitive since, when the traffic pattern changes and a different virtual topology is more desirable, this switch does not provide support for such a reconfiguration to occur. A more-attractive alternative is to employ an array of optical (lithium niobate) switches, one per wavelength, be tween the demux and mux stages. These switches can be reconfigured under electronic control, and this property can be exploited for reconfiguring the network s virtual topology on demand. One approach would be to have the local lasers/filters normally operated on fixed wavelengths, but a facility to tune them to different wavelengths must be provided. Fig. 3 shows a few more interesting issues of this architecture. The box labelled ROuter is an electronic switch d.l.3

4 Figure 1: The physical topology with imbedded wavelengths corresponding to an optimal solution (all transceiver pairs at any node must be tuned to different wavelengths). which takes information from terminated lightpaths (C 4b 5c) as well as a local source (labelled Workstation ), and routes them to the local lasers (lightpath originators) and the local destination. 3 Formulation of the Optimization Problem The following notation will be used: s and d employed as subscripts or superscripts will denote the source and destination of a packet, respectively; i and j will denote end nodes in a lightpath; while m and n will denote endpoints of a physical link that might occur in a lightpath. 3.1 Given: 0 Number of nodes in the network = N. 0 Maximum number of wavelengths per fiber = M (a system-wide parameter). The physical topology Pmn, where P, = Pnm = 1 if and only if there exists a fiber link between nodes m and n, where m, n = 1, 2, 3,..., N; Pmn = Pnm = 0 otherwise (i.e., fiber links are assumed to be bidirectional). 0 The fiber distance d, from node m to node n. For simplicity in expressing message delays, d,, is expressed as a propagation delay (in time units). dm, = dnm since fiber links are bidirectional. d,, = 0 if and only if Pmn = 0. 0 The number of transmitters at node i = Ti. The number of receivers at node i = Ri (Ti, Ri 2 1). 0 The traffic matrix denoting the amount of traffic flow from node s to node d, with A,, = 0 for a, d = 1, 2,..., N. 3.2 Variables: Any solution to the problem must provide values for the following primary variables of the system Primary variables: 0 Virtual topology: The variable V = 1 if there exists a lightpath from node i to node j in the virtual topology; V,, = 0 otherwise. Virtual topology route: The variable ; U: = 1 if virtual link V,, is in the multihop path from node s to node d; sd U,, = 0 otherwise. Physical topology route: The variable pxn = 1 if the fiber link Pmn is used in the lightpath for virtual link K,; px, = 0 otherwise Auxiliary Variables: These variables are directly obtained from the primary variables and the given constants. They help in formulating the optimization criteria. 0 Virtual link length: The variable D,, is the length of the lightpath between nodes 8 and j, and it is specified by: 0 Virtual Path Length: The variable Lad is the length of the virtual path between nodes s and d, and it is given by : 1 d.l.4 113

5 3.3 Constraints: e On virtual topology connection matrix V,j: e On physical route variables pzn: e On virtual route variables U$: J j 3.4 Objective - Optimality Criterion: The objective is to minimize the average message delay as follows: Minimize 8,d XsdLsd 3.5 Comments The above formulation does not solve the wavelength assignment problem explicitly although it restricts the number of wavelenghts per fiber. Wavelength assignment is the actual assignment of wavelenghts to various transmitters and receivers. This optimization problem is expected to be NP-hard (several sub-problems of this problem are NP-hard). The problem can be split into two sub-problems. 0 Find a good virtual topology given a physical topology, the traffic matrix, and the degrees of the various node. (This problem has been studied quite extensively but we dont know of solutions except fixed topologies such as ShuffleNet, de Bruijn graph, etc.) Assign wavelengths optimally to the various links of the virtual topology. (This problem is NP-hard as shown in [12].) 4 Heuristic Algorithms 4.1 Solution Approach The problem of designing optimal virtual topologies (without assuming any underlying physical topology) has been studied before [3, 181. n general, the optimal virtual topology problem has been conjectured to be NP-hard. Accordingly, heuristic approaches have been employed to solve these problems [3, 181. As a first step, regular topologies such as a hypercube may be chosen. Regular topologies have been studied quite extensively. The main advantages of regular topologies are their easy and efficient routing and maint ainance. Once a regular topology has been chosen, the virtual topology design problem reduces to a node-mapping problem. The objective of the node-mapping problem is to map the nodes of the regular virtual topology to the nodes of the physical topology such that some predefined optimality criterion is satisfied. Example optimality criteria include minimizing the mean network-wide message delay [3, 61, or minimizing the maximum link flow in the network [6, 181. The nodemapping problem has been studied in Merent contexts before [6]. The wavelength assignment problem is to assign wave lengths to the various transmitters and receivers at all the nodes in the network. That is, this problem is analogous to assigning wavelengths to the different lightpaths in the network. A version of this problem was shown to be NP-hard in [ll]. For a particular regular topology such as the hypercube and for arbitrary physical topologies, a polynomial-time algorithm to solve this problem is not known. Since the search space for this optimization problem is very large, some simplifying assumptions are needed to make it tractable. Since regular structures have attractive routing properties, in most of our iuustrative solution approach, it is assumed that the virtual topology is a hypercube. The hypercube virtual topology is particularly attractive when each node in a N-node network has a degree of log(n). n the augmented NSFNET of 16 nodes (Fig. l), the maximum physical nodal degree is 4. For nodes with a physical nodal degree less than 4, it is trivial matter to add additional transceiver pairs to increase their virtual nodal degree to 4. Also, if nodes have virtual degree log(n) but there are fewer than N nodes in the network, it is a simple matter to set up an incomplete (or injured ) hypercube, as we will show in Section 6. The physical topology is arbitrary, but it has the same number of nodes as the virtual topology. t is also assumed that the optimality criterion is minimizing the average propagation delay encountered by all packets. Queueing delays at each intermediate electronic switch encountered by a packet can also be factored in, by assuming some value for the date rate of each WDM channel. However, the propagation delay is the dominant component for message delays in high-speed networks; so, in most situations, we will ignore the queueing delay components in order to keep the analysis simple Hypercube Embedding We assume shortest-path routing on the physical topology, i.e., among different possibilities for setting up a lightpath, that path which is the shortest path on the physicid is topology is chosen. This may not lead to an optimal solution, but it is a first step and is simple. We also assume shortest-path routing on the hypercube. By employing standard trafficflow-conservation arguments on the virtual links, we obtain the network-wide mean message delay as was shown in Section d.l.5

6 The question is how to determine a Ugood embedding, if not the optimal one. For doing this, we examine two heuristic algorithms - one based on a greedy approach and the other employing a randomized approach based on simulated annealing. The following subsections are devoted to these two heuristics. 4.2 Greedy Approach A high-level description of this algorithm follows: 1. Obtain a reasonable initial embedding, 2. Refine the initial embedding by means of nodeswapping operations, and 3. Perform wavelength assignment nitial Embedding The initial embedding tries to map the nodes of the physical topology.o the virtual topology nodes in such a way that as many links as possible on the physical topology are mapped to the virtual topology links. t does not take into account the link lengths of the physical topology or the traffic matrix. For a physical node A and a virtual node X, we define an objective function F(A,X) = nodal degree of A + k*(number of neighbors of A in the physical topology that are neighbors of X in the mapping obtained so far), where k is a constant which determines how much weight is given to nodes that have already been mapped. (n our numerical example to be outlined later, we employ k = 4.) So, the initial embedding algorithm is the following. 1. While all nodes are not mapped, choose the node with highest F(A,X) value and map it Refinement Once an initial embedding is obtained, we can refine the embedding by perturbing or tweaking it. One way to do this is to swap two nodes in the virtual topology if the swap will improve the objective function (viz. lower the average message delay in this case). The refinement algorithm used here is the following. 1. Let 0 be the order of nodes mapped in the initial embedding algorithm. Let 0 be the reverse of For each node X in 0, choose a node Y on the virtual topology such that swapping X and Y would lower the embedding s mean message delay. The refinement is done in the reverse order of initial node placement since nodes that get placed late in the initial embedding have fewer placement options, and allowing them to be tweaked first results in the maximum gain, on the average. Accordingly, this greedy algorithm will also be referred to as the reverse-order-refinement algorithm Wavelength Assignment Given an embedding, we can determine all the virtual paths that pass through P physical link. Each path must be assigned a unique wavelength. The algorithm used for doing this is the following. 1. Order physical links in decreasing order of the number of virtual paths passing through each link. Let 0 be such an order. 2. For each physical link in 0, assign different wavelengths to all unassigned virtual paths passing through the physical link, such that this assignment is consistent with the assignments already made. By consistent, we mean that each physical link must have differrent wavelengths for each virtual path passing through it and each virtual path must be assigned the same wavelength throughout its physical route, i.e., there is no wavelength conversion on a lightpath. Once we know the wavelengths on each of the physical paths, it is a straightforward matter to figure out the switch sizes at each node. 4.3 Simulated Annealing Simulated annealing (along with genetic algorithms) has been found to provide good solutions for problems of this nature. n the simulated annealing process, the algorithm starts with an initial random configuration for the node embedding. Then, it goes through a series of node-exchange operations so as to provide a lower overall average delay for the network. n a node-exchange operation, adjacent nodes in the virtual topology are examined for swapping in order to arrive at neighbouring configurations, e.g., if node i is connected to nodes j, a, and b, while node j is connected to nodes p, q, and i in the virtual topology, after the node-exchange operation between nodes i and j, node i will be connected to nodes p, q, and j, while node j will be connected to nodes a, b, and i. Neighbouring configurations which give better results (lower average message delay) than the current solution are accepted automatically. Solutions which are worse than the current one are accepted with a certain probability which is determined by a system control parameter. The probability with which these failed configurations are chosen, however, decreases as the algorithm progresses in time so as to simulate the cooling process associated with annealing. The probability of acceptance is based on a negative exponential factor and is inversely proportional to the difference between the current solution and the best solution obtained so far. The initial stages of the annealing process examines random configurations in the search space so as to obtain different initial starting configurations without getting stuck at a local minimum as in a greedy approach. However, as time progresses, the probability of accepting bad solutions goes down, and the algorithm settles down into a minimum after a certain point of time. The state become frozen when there is no improvement in the objective function of the solution even after a large number of iterations. Once a virtual topology is found, the same wavelength assignment algorithm stated earlier in Section is used. 1 d.l.6 115

7 . 5 Experimental Workload To evaluate the performance of the heuristic algorithms, we shall use measured data over an existing wide-area network, the NSFNET backbone. n the sequel, we describe the measurement procedure used to collect the data and the preprocessing needed for its use in the algorithm. We also report on some relevant statistical properties that were observed in the data. 5.1 Meesurement Procedure and Prepro- cessing NSFNET backbone data was collected by the llstat program and made available to us by the National Science Foundation (NSF) through its Merit partnership. Each NSFNET switch consists of a core switch called C- NSS (Core Nodal-Subsystem) and interfaces to the external world called ENSS s. For example, during the time when our data was collected, the C-NSS at node CA (Palo Alto) provided backbone connectivity to the regional networks BARRnet and Fixwest (NASA/Ames), each through a sep arate interface or ENSS. For a description of the NSFNET switch subsystem architecture, including the Core and External Nodal-Sub-Systems, see [14]. The data that we use is based on sampling every 50th packet at each external interface to the backbone (E-NSS), and estimating from this the approximate total number of packets and total number of bytes that flowed through each interface. For each external interface, sampled packets were processed and aggregated by Merit into tuples (A,B,C,D) where: A B C = source campus network, = destination campus network, = total number of bytes that flowed between A and B over a period of 15 minutes, and = total number of packets that flowed between A and B over the same 15 minutes. We obtained approximately four months of data in the D above format from NSF/Merit during the period January 1992 through April Measurements at interfaces consisted of some duplicate information because each interface traced packets independent of the others. The same packet could, therefore, be potentially sampled twice - once at its entry point to the backbone and once at its exit point from the backbone. The duplicate information was filtered out using the following steps: 1. Determination of the geographical point of contact of each campus network and the backbone. While this information is probably available in some database somewhere, we decided to obtain it directly from the data we had, using the following heuristic algorithm. For each campus network X, let Xi be the total number of bytes observed at interface i with X as the source network. Since a campus network sources into the backbone through at most one external interface (through the regional network it is part of), the value of i for which X, is maximum points to its physical geographical location. 2. Collapsing data into a Core NSS - to - Core NSS (origin-destination) matrix form. This involved: (a) filtering out duplicate information in the original data - e.g., by counting only those tuples at an interface for which the source belonged to that specific geographical region, and (b) adding all data that went from one Core NSFNET switch (C-NSS) to another for each time period (in this case, 15 minutes). 5.2 Statistical Properties of Data n this subsection, we show the autcxorrelations in timeseries data for a few selected origin-destination pairs. Since the matrix is large, we restrict ourselves for illustration purposes to a few examples only. Figures 4(a) - 4(d) show the auto-correlation functions for traffic originating at nodes CO (Boulder), L (Champaign), CA (Palo Alto), and UT (Salt Lake City), respectively. Each line in Figure 4(a), for example, represents an 0-D pair with node CO as the origin. For sake of clarity, we have not labelled the destinations. We observe the following: 1. The auto-correlations between different origin-destination (0-D) pairs differ significantly. 2. Most 0-D pairs show a significant lag-1 autocorrelation, implying a high degree of predictability of traffic for successive time-points. This is important if an algorithm has to estimate the traffic matrix for the next time period accurately. 3. For some 0-D pairs, the auto-correlations persist for larger values of lag, suggesting a self-similarity of traffic between these points. (See Figure 4(b).) This is interesting because self-similarity in traffic was recently reported in local area network traffic as well [19]. We observe, however, that for most 0-D pairs over the backbone, the auto-correlations die out fairly quickly. 4. Some 0-D pair auto-correlations show small periodicities, implying periodicities in traffic. The virtual topology design algorithm of this paper would need precise estimates for the time evolution of the origin-destination traffic matrix. For the estimates to be precise, we must have significant auto-correlations for one or more lags. Measurement data shows that this is indeed the case, even for data traffic. This is because aggregated data traffic at the national backbone level is over a large ensemble of users and has been aggregated over a sufficiently long period. Hence, large auto-correlations ought to exist for at least a few lags. We are currently investigating exact time-series prediction models for the data, in conjunction with dynamic virtual-topology update algorithms. That study will be presented in a later paper when results become available d. 1.7

8 6 Experimental Results The simulated annealing algorithm as well as the greedy (reverseorder refinement) algorithm were both used to map the physical topology of Fig. 1 into a virtual hypercube topology. The traffic matrix employed for this mapping is an actual measurement of the traffic on the T1 NSFNET backbone for a lbminute period (11:45 pm to midnight) on January 12, The raw tr&c matrix showing traffic flow in msgs/sec between network nodes is shown in Fig. 5. Nodal distances used are the actual geographical distances, and they are not shown here eeparately. The lbnode hypercube cannot be directly embedded onto the NSFNET since there are only 14 nodes in the network. Hence the fictitious nodes AB and XY were added during the initial embedding stage. Since these nodes do not source any traffic, they do not contribute in any way to the delay obtained in the system. However, due to the partial embedding, some of the nodes (e.g., UT, NE, CO, etc. in Fig. 1) have less than four transceivers. Also, the number of wavelengths needed to support the complete hypercube could, in general, be more than that needed for the partial embedding. n the reverse-order-refinement algorithm, the hypercube embedding as obtained by the initial embedding algorithm was tweaked and the average delay dropped substantially within as few as 10 tweaks. The final solution had an average delay of ms. Note that this number is quite encouraging compared to the coast-to-coast one-way propagation time of nearly 23 ms. n the simulated annealing process, the results were even more impressive. The final solution as shown in Fig. 1 had a network-wide average delay of ms without considering queuing delays, and ms when queueing was considered (assuming mean message length of 100 bytes, link speed of 45 MBps, and a standard M/M/l queueing model for each node). t was found that the traffic matrix could be scaled by a factor of before the link connecting L and PA saturated. t must be kept in mind, however, that the annealing algorithm had an objective function of reducing the delay. Other objective functions which attempt to distribute the traffic could have done better than this solution, e.g., the Min-Max approach which tries to minimize the maximum flow on any link. The hypercube embedding as obtained by the simulated annealing solution was embedded on the physical topology using wavelength assignments based on shortest-path muting on the physical topology. The following two cases, which essentially represent cost trade-offs, must be considered. More than one transceiver at any node can tune to the same wavelength. This means that there must be multiple fibers from the local node to the WRS, thus increasing the size of the switch. The number of wave lengths used in this case was 5 [22] 0 All nodal transceivers must be tuned to different wavelengths. This increases the number of channels needed to support the network to 7 (see Fig. 1). However, the size of the switch at each node is smaller because there needs to be only one fiber from the local host to the WRS. n general, we would expect that the average delay would reduce as the nodal degree is increased. The average measage delay would be minimum if the nodal degree was N - l and the virtual topology was fully-connected. The average delay in this case was computed to be 9.39 ms for our example. To analyze how the delay in the system increased as the nodal degrees are reduced, we ran the simulated annealing algorithm on arbitmry virtual topologies. These dts are compiled in Fig. 6. Since virtual links are bidirectional, if each node has two tranceiver paha, the virtad toplogy is a bidirectiond ring (as in FDD), and the companding average message delay for our example tum out to be ms. As the nodal degree is increased, the average mewage delay drop. We find that, by equipping each node with four transceiver pairs, we can achieve an average message delay of 9.7 ms, which is within 3% of the optimal value (9.39 ms). 7 Discussion and Open Research ssues The virtual topology optimization example discussed and outlined in this paper serves as a good illustration, but it is only a first step before a robust and versatile WDM solution can be found. Specifically, our simple-minded algorithms found a WDM solution whose delay metric was ms, whereas the best solution for this problem turm out to be 9.39 ms (using shortest-path flows but not necessarily WDM-realizable with a limited number of wavelengths). We believe that a significant amount of room exists for developing improved approaches and algorithms, and a number of research issues must be addressed in thw regard, some of which we outline below. First, the suitability of other virtual topologies, both regular and irregular, must be thoroughly investigated. Second, the traffic matrix used in our example was obtained by measurement over a 15-minute period. This may be too small a duration over which the traffic statistics can be considered steady state. Finally, more-sophisticated optimization criteria which can combine both delay minimization and minimization of the maximum link flow need to be examined as well. 8 Acknowledgments Our deep appreciation of thanks go to Eric Aupperle, Bilal Chinoy, Elise Gerich, Susan Horvath, and Mark Knop per from NSF/Merit Partnership for providing us with the NSFNET data. References [] A. S. Acampora and M. J. Karol, An Overview of Lightwave Packet Networks, EEE Network, vol. 3, no. 1, pp , January [2] A. Aggarwal, A. Bar-Noy, D. Coppersmith, R. Ramaswami, B. Schieber, and M. Sudan, Efficient routing and scheduling algorithms for optical networks, BM Research Report RC18967, d.l.8 117

9 J. A. Bannister, L. Fratta, and M. Gerla, Topological design of the wavelength-division optical network, Proc., EEE NFOCOM 90, San Francisco, CA, pp , June S. Banerjee, J. ness, and B. Mukherjee, New Modular Architectures for Regular Multihop Lightwave Networks, Pm., Optical Fiber Commun. Conf., San Jose, CA, Feb K. Bala, Routing in Linear Lightwave Networks, Ph.D. Dissertation, Dept. of Electrical Engineering, Columbia University, Oct S. Banerjee, B. Mukherjee, and D. Sarh, Heuristic algorithms for constructing near-optimal structures of linear multihop lightwave networks, Pm., EEE nfo 92, Florence, taly, pp , May R. A. Barry and P. A. Humblet, Bounds on the nnmber of wavelengths needed in WDM networks, Pm., OMAN Summer Topicals, Santa Barbara, CA, pp , August C. A. Brackett, Dense Wavelength Division Multiplexing Networks: Principle and Applications, EEE Journal on Selected Areas in Communication, vol. 8, no. 6, pp , August M.-S. Chen, N. R. Dono, and R. Ramaswami, A Media Access Protocol For Packet-Switched Wavelength Division Multiaccess Metropolitan Area Networks, EEE Journal on Selected Areas in Communication, vol. 8, pp , August Chlamtac and A. Fumagalli, Performance of reservation based (Quadro) WDM star networks, Proc., EEE NFOCOM 92, Florence, taly, May [11]. Chlamtac, A. Ganz, and G. Karmi, Lightnet: Lightpath based solutions for wide bandwidth WANs, Proc., EEE NFOCOM 90, San Francisco, CA, pp , June [12]. Chlamtac, A. Ganz, and G. Karmi, Transport optimization in broadband networks, Proc., EEE NFO- COM 91, Bal Harbour, FL, pp , April [13]. Chlamtac, A. Ganz, and G. Karmi, Lightpath communications: A novel approach to high bandwidth op tical WANs, EEE Tmnsactions on Communications, vol. 40, no. 7, July [14] K. C. ClaRy, G. C. Polyzos, and H. W. Braun, Traffic characteristics of the T1 NSFNet backbone, Proc., EEE nfo 99, San Francisco, [15] N. R. Dono, P. E. Green, Jr., K. Liu, R. Ramaswami, and F. F.-K. Tong, A wavelength &vision multiple act- network for computer communication, EEE J. Selected Areas in Communications, vol. 8, pp , Aug [16] P. E. Green, The Future of Fiber-optic Computer Networks, EEE Computer, vol. 24, no. 9. pp , September [17] G. R. Hill, Wavelength domain optical network techniques, Proceedings of the EEE, vol. 77, pp , Jan [18] J.-F. P. Labourdette and A. S. Acampora, Logically rearrangeable multihop lightwave networks, EEE tuns. Commun., vol. 39, no. 8, pp , Aug [19] W. E. Leland, M. S. Taqqu, W. Willinger and D. V. Wilson, On the self-similar nature of Ethernet traffic, Pm., ACM Sigwmm 99, San Francisco, pp , September [20] B. Mukherjee, WDM-based local lightwave networks - Part : Single-Hop systems, EEE Network Magazine, vol. 6, no. 3, pp , May [21] B. Mukherjee, WDM-Based Local Lightwave Networks - Part 11: Multihop Systems, EEE Network Magazine, vol. 6, no. 4, pp , July [22] B. Mukherjee, S. Ramamurthy, D. Banerjee and A. Mukherjee, Some Principles For Designing A Wide Area Optical Network, Tech. Report No. CSE93-10, Dept. of Comp. Sci, Univ. of California, Davis, Sept 93. [23] R. Ramaswami, Multiwavelength Lightwave Networks for Computer Communication, EEE Communications Moaazine, vol. 31, no. 2, pp , February R. Ramaswami and K. Sivarajan, Optimal routing and wavelength assignment in all-optical networks, BM Research Report RC18964, June T. E. Stern, Linear lightwave networks: How far can they go? Proc., EEE Globecom 90, San Diego, CA, pp , Dec _... :.h. :._ Figure 2: A 16-node hypercube virtual topology embedded on the physical topology of Fig. 2. This is also the find solution to our embedding-problem (average m-age delay = 9.7l5 ms)* d.l.9 1

10 Wavelength Routing Switch Workstation Figure 3: Details of the UT wavelength-routing switch. Figure 5: Traffic matrix (in bytes/sec between nodes) s w (e) SM. W-CAl (P.loAl(0) (d) Sarce Nod. - UT (SSnLak. Cny) Figure 4: Example auto-correlations in time series data for origin-destination pairs. Figure 6: Performance improvement with nodal degree for arbitrary virtual topology. 1 d.l