A joint multi-layer planning algorithm for IP over flexible optical networks

Transcription

1 A joint multi-layer lanning algorithm for IP over flexible otical networks Vasileios Gkamas, Konstantinos Christodoulooulos, Emmanouel Varvarigos Comuter Engineering and Informatics Deartment, University of Patras, and Comuter Technology Institute and Press «Diohantus», Patras, Greece {gkamas, kchristodou, Abstract - We consider the multi-layer network lanning roblem for IP over flexible otical networks, which consists of three sub-roblems at two layers: the Routing roblem at the IPlayer (IPR), the Routing, Modulation Level (RML) and the Sectrum Allocation (SA) roblems at the otical layer. The inut includes the IP end-to-end traffic matrix, the modular model of the IP/MPLS routers and the feasible transmission configurations of the flexible otical transonders. Demands are served for their requested rates by selecting the IP/MPLS routers modules to be used, the routes in the IP (virtual) toology, and the corresonding aths and sectrum slots in the underlying otical toology, together with the otical transonders configurations. The roosed algorithm follows a multi- aroach that accounts jointly for the IPR, the RML and the SA roblems. It serves demands one-by-one, re-using existing equiment and favouring the deloyment of new equiment to be re-used by subsequent connections, aiming to minimize the total network. The roblem definition is generic and the roosed algorithm is alicable to both fixed- and flex-grid otical networks. We evaluate the erformance gains that can be obtained by the roosed joint multi-layer network lanning solution, as oosed to a sequential lanning solution that searately lans the IP and otical layers. We also comare a flexible network, using flex-grid otical switches and flexible otical transonders, to a mixed line rate (MLR) network, using fixed-grid or flex-grid otical switches but fixed otical transonders. Index Terms Planning; Flex-grid; IP over WDM; IP over flexible (elastic) otical networks; Routing Modulation level and Sectrum allocation (RMLSA); Distance adative routing and sectrum allocation; Multi- algorithm. I. INTRODUCTION The continuous growth of consumers IP traffic, due to the introduction of broadband access and FTTx technologies, and the bursty nature of the alications characterized by richcontent and high-rate, like Video-on-Demand and HDTV, has brought to light the inefficiency of current WDM otical networks. Modern alications do not only entail an increase in the traffic volume, but also an increase in the unredictability and the dynamic nature of this underlying traffic growth. Moreover, these alications come with stringent service level requirements and heightened end-user Quality of Exerience (QoE) exectations. In this evolving network oerations environment, simly roviding more caacity is no longer sufficient. Wavelength Division Multilexing (WDM) otical networks are tyically used today in core and metro networks. They are usually designed with an overrovisioning factor and are oerated statically and indeendently, without taking into account short- or mid-term traffic dynamics at their edges. This is mainly due to the comlicated otical connection establishment rocess that needs to account for the hysical layer (imairments). Thus, WDM networks are not only rigid and static in hysical terms, but also rigid and constrained in the oerational sense, resulting in oor utilization, stranded caacity, and inability to react to new service demands in a timely manner. Multi-layer otimization across the IP and the otical layer is often mentioned but seldom truly erformed in a joint manner, as decisions at the IP layer are taken without considering their imact on the otical layer, and then the otical layer is (re)configured to address the required changes. Photonic advances have enabled transmission of 100 Gbs, and coherent detection has lowered the need to account for hysical layer imairments, but it is obvious that the traditional WDM aroach of ugrading the network by statically utting abundant caacity will not be an efficient solution in the forthcoming years. Planning of the otical core and metro network has to be done in a joint multi-layer rocess to bring forward savings both in its electrical router edges and the otical network. The rigid bandwidth and reach granularity of WDM networks make such lanning inefficient. Moreover, there is also the need to make the otical layer more dynamic and agile, by considering it as art of the whole network, to increase its efficiency and enable end-to-end service rovisioning. This is also the aroach favored by Software Defined s (SDN) technology, where rogrammability and flexibility through centralized control is meant to hold for both the IP and the otical layers. Recently flexible (or elastic) otical networks have emerged [1]. Flexible networks are based on (i) flex-grid technology that enables the slicing of the sectrum according to the needs, as oosed to the rigid granularity of WDM networks, but also on (ii) flexible transonders, known also as bandwidth variable transonders (BVTs), which can tune their transmission arameters, trading off the transmission reach for sectrum and/or rate. Flexible otical networks solve various inefficiency roblems of traditional WDM networks, roviding a fine granular solution for sub- and suer-wavelength caacity. Moreover, their increased flexibility fits well with the dynamic multi-layer network oeration that is actually needed. To enable the multi-layer network lanning and oeration of flexible otical networks in a joint and effective manner, aroriate control lane extensions and algorithms are

2 needed. The resent aer takes a first ste towards this end, by focusing on the joint multi-layer lanning of a flexible otical network. In the general case, the multi-layer network lanning roblem consists of roblems in two layers: the Routing sub-roblem at the IP layer (IPR), and the Routing, Modulation Level and Sectrum Allocation (RMLSA) subroblem at the otical layer. Further, the RMLSA roblem can be broken into two substituent sub-roblems, namely a) routing and modulation level decision and b) sectrum allocation (RML+SA). Thus the multi-layer network lanning roblem consists of three sub-roblems: IPR+RML+SA that can be jointly or sequentially solved. A variation of the IPR roblem is also referred to in the literature as traffic grooming, while the RMLSA roblem is also referred to as Distance-adative RSA. Distance adativity creates interdeendencies between the routing at the otical (RML) and the IP layers (IPR) and the sectrum allocation (SA), making it hard (and also inefficient) to decoule these sub-roblems. The roosed joint multi-layer network lanning algorithm treats the demands of the IP-layer traffic matrix one-by-one, in some articular sequence. So it is alied here to solve the lanning roblem, but it can also be used with minor modifications for dynamic network oeration ( one time roblem) as well. The roosed algorithm solves the multilayer network lanning roblem (IPR+RML+SA) jointly and is an extension of the algorithm used in [2], which considered jointly only the IPR+RML subroblems. As discussed above, we decided not to decoule the IPR, the RML and the SA roblems, because when flexible transonders are used the rate and sectrum of the otical connection deends on its hysical length (RML decision), which in turn affects the IPR and SA decision. Algorithms that decoule these three sub-roblems by sequentially lanning the IP and otical layers, or erforming sectrum allocation indeendently are bound to be inefficient and waste resources. The multi-layer network lanning algorithm that jointly erforms the routing at IP and otical layers and the sectrum allocation follows a multi- aroach [3]. In multi- (or multi-constrained) algorithms, each link is characterized by a vector of arameters, in contrast to the traditional single routing where each link is characterized by a scalar. By defining aroriate comonent-wise oerations between vectors, we can calculate the vector of a ath. In the roosed algorithm the -vector conveys information regarding: the transonders transmission reach, the use of already established connections, the of the IP and otical layer, and the sectrum availability, which are used to solve the joint IPR+RML+SA roblem. The flexible transonders used are assumed to be characterized by a set of so called transmission tules that identify the reach at which a transmission is feasible, given the arameters that are under our control, such as the rate, the sectrum used for the transmission, and the modulation level. Feasibility here refers to the hysical layer and signifies accetable bit-error rate or accetable quality of transmission (QoT) [4], [13], [21]. The roosed lanning algorithm is general and can be alied to any tye of otical network: otical networks emloying flex-grid otical switches and flexible or even fixed transonders, and WDM networks emloying fixed-grid otical switches and single or multile tyes of fixed transonders [also referred to as single-line-rate (SLR) and mixed-liner-rate (MLR)]. The only requirement is to describe the inut in the form of feasible transmission tules. In the simulations conducted, we used the roosed joint multi-layer network lanning algorithm to lan both flexible networks emloying flex-grid otical switches and flexible otical transonders and MLR networks emloying fixed-grid or flexgrid otical switches but fixed otical transonders and comare their erformance. We also distinguished two cases: Joint Multi-Layer Planning (JML-NP), where the IPR, RML and SA sub-roblems are jointly solved, and Sequential Multi-Layer Planning (SML-NP), where the IPR, RML and SA sub-roblems are sequentially solved. Using realistic, network, and traffic models, taken by IDEALIST roject [4], we found that significant and sectrum savings can be obtained through joint multi-layer otimization over the IP and the otical layer (JML-NP), as oosed to lanning the two layers sequentially (SML-NP). We also found that when lanning the network in a joint multilayer otimization manner (JML-NP case), the flexible network outerforms the MLR network deloying fixed-grid or flex-grid otical switches and fixed otical transonders, in terms of for medium and high loads and in terms of maximum sectrum used in all cases. Moreover, in the SML- NP case the flexible network outerforms the MLR network with resect to and sectrum under all load conditions. The rest of the aer is organized as follows. Section II resents the related work, while Section III describes the network architecture and the multi-layer network lanning roblem. Section IV describes the joint multi-layer network lanning algorithm. Simulation results are resented in Section V. Finally, our conclusions are given in Section VI. II. RELATED WORK Multi- Routing and Wavelength Assignment (RWA) algorithms for fixed-grid WDM otical networks have been investigated in the ast. A multi- aroach with arameters being the OSNR, the number of free wavelengths, and the link is resented in [6]. In [7], imairment-aware multi- RWA algorithms for online traffic in transarent otical networks are roosed. The vector includes imairment generating source arameters or noise-related arameters, so as to indirectly or directly account for the otical layer effects. In [8], the authors extend the work of [7] to account for regenerators and resent a multi- aroach for translucent WDM networks. We now turn our attention from WDM to flexible otical networks. In [9] the authors address the offline RSA roblem with dedicated ath rotection in elastic otical networks and they rovide an Integer Linear Programming (ILP) formulation to solve it. A distance-adative RSA algorithm for dynamic flexible networks is roosed in [10], in order to select the roer modulation format according to the transmission reach. In [11], a dynamic RMLSA scheme in flexible otical wavelength-division multilexed networks with modulation format conversion ability is roosed. The RMLSA roblem for lanning a flexible otical network has been investigated in [12]. Algorithms for lanning flexible otical networks under hysical layer constraints are also roosed in [13], while in [16] a nonlinear rograming model is roosed to formulate the comlete RSA roblem, at which the sectrum continuity constraints, the transmission distance constraints, and the relationshi between the traffic bitrate and the signal bandwidth are jointly considered.

3 Traffic grooming algorithms for IP over otical networks has also received a great deal of attention. In [14] an IP over flexible otical multi-layer routing and grooming algorithm is roosed, which emloys a bandwidth threshold, an auxiliary grah, and two grooming olicies. The authors in [15] introduce a multi-layer auxiliary grah to jointly solve the IPlayer routing and otical-layer RSA, and they also roose various traffic-grooming olicies. Finally, in [5] the authors roose a novel multi-layer caacity lanning aroach for IP over otical networks. They give extra attention on the rocess of creating the IP (virtual) toology (considering the imact at the IP layer toology when they byass routers), leveraging a commercial IP lanning tool; then they consider the design and imact of several multi-layer restoration schemes. To the best of our knowledge multi- algorithms have not yet been alied to flexible otical networks and/or IP over WDM or flexible online or offline roblems. Since such roblems are quite comlicated, with many otimization arameters involved, multi- seems a reasonable aroach to address them. Note that the multi- framework adoted here is mainly used for serving a single demand (online traffic) [3], [6], [7], [8], which is the case also for the roosed algorithm, although it is used here in an iterative way for serving all demands and thus lan the whole network. The novelty of our roosed solutions comared to revious works is threefold. First, the roblem definition and the network lanning algorithm roosed is quite general and takes generic but realistic transmission secifications as inut (based on [19],[20],[21]), which are given in the form of feasible transmission configurations of the transonders used. So, it can be used for joint or sequential multi-layer lanning of both flexible and fixed-grid otical networks, using fixed or flexible transonders. Second, the roosed joint multi-layer network lanning algorithm includes distance adativity/ modulation level decisions, making the roblem more realistic and difficult due to inter-deendencies between the RML and the IPR and SA sub-roblems. So, the roosed algorithm is more sohisticated comared to revious algorithms, solving jointly all the interrelated sub-roblems. Third, in contrast to revious works, we consider more arameters in our otimization formulation. In articular, we consider more accurately the IP layer, by using a detailed modular model for the IP/MPLS routers deloyed at the edges of the otical network. Moreover, our algorithm, in addition to allocating the routes at the IP and the otical layers and sectrum at the otical layer in a cross-otimized way, it also selects the transmission configurations of otical transonders and the number of IP/MPLS routers modules needed. III. NETWORK ARCHITECTURE AND PROBLEM DESCRIPTION We are given an otical network domain that consists of otical switches and fiber links. The otical switches function as Reconfigurable Otical Add Dro Multilexers (ROADMs) emloying the flex-grid technology, and suort otical connections (lightaths) of one or a contiguous number of 12.5 GHz sectrum slots. At each otical switch, none, one or more IP/MPLS routers are connected (these routers comrise the edges of the otical domain). Short reach transceivers are lugged to the IP/MPLS routers leading to flexible (tunable) transonders at the ROADMs. Alternatively, flexible (tunable) colored transceivers could be lugged to IP/MPLS routers orts, generating signal that could directly enter the otical network domain. Since the two above alternatives are almost equivalent, in terms of and functionality, we will focus in the transonder case. A transonder is used to transform the electrical ackets transmitted from the IP source router to the otical domain, acting as an otical transmitter in this case (E/O conversion). The traffic entering the ROADM (otical switch) is routed over the otical network in lightaths (all-otical connections). We assume that a number of transmission arameters of the flexible transonders are under our control, affecting the otical reach at which they can transmit. At the destination of a lightath the ackets are converted back to electrical signal at the transonder that functions as an otical receiver in this case (O/E conversion). The ackets at the receiver are forwarded and handled by the corresonding IP/MPLS router. This IP/MPLS router can be: (i) the final destination of some ackets in the domain, in which case these ackets will be forwarded further towards their final destination through other domains or lower hierarchy level networks attached to that router, or (ii) an intermediate ho, in which case the related ackets will re-enter the otical network to be eventually forwarded to their domain destination. Note that lightaths are bidirectional and thus in the above descrition an oosite directed lightath is also installed, and the transonders used act simultaneously as transmitters and receivers. Also, note that acket rocessing is erformed electronically and in articular at the IP/MPLS routers, while otical switches function as transarent ies between IP/MPLS router end-oints. We are also given the traffic matrix Λ that corresonds to the IP traffic at the IP/MPLS routers that is forwarded over the otical domain under study. Our goal is to establish lightaths, and route the traffic over these lightaths and through ossibly intermediate IP/MPLS routers to the end IP/MPLS router destination. For this reason the network is also referred to as IP over flexible otical network. We assume that the IP/MPLS routers are modular (a more detailed descrition will be given in the following). We also assume that at the otical layer the otical switches and fiber links are deloyed, but not the transonders. As discussed in the introduction, the lanning of an IP over flexible network consist of three inter-related sub-roblems: (i) the IP routing (IPR), (ii) the Routing and Modulation Level (RML), and (iii) the Sectrum Allocation (SA). In the IPR roblem we decide on the modules to install at the IP/MPLS routers, how to ma traffic onto the lightaths (otical connections), and which intermediate IP/MPLS routers will be used to reach the domain destination. In the RML roblem we decide how to route the lightaths and also we select the transmission configurations of the flexible transonders to be used. In the SA we allocate sectrum slots to the lightaths, avoiding slot overlaing (assigning the same slot to more than one lightaths) and ensuring that each lightath utilizes the same sectrum segment (sectrum slots) throughout its ath (sectrum continuity constraint). The use of flexible otical transonders, where the rate, reach, and sectrum are not given but have to be decided, is the reason RML decisions affect the two other sub-roblems, significantly comlicating the network lanning roblem. Following the IDEALIST model [4], we view an IP/MPLS router as a modular device, built out of (single or multi) chassis, that incororates the hysical and mechanical assembly, the internal switch, the ower sulies, the cooling system, the control and management lane and the related software. A chassis rovides a secified number of bi-

4 directional slots with a nominal (maximum) transmission seed. Into each router slot, a linecard of the corresonding seed can be installed. Each linecard rovides a secified number of orts at a secified seed and occuies one slot of the IP/MPLS router. In our erformance results we consider a scalable multi-chassis router for core nodes, with u to 72 chassis, and a 16 router slot caability er chassis. Note, however, that our roblem definition and roosed algorithms are generic and can work with other router models as well. S1 R1 lightath hysical link virtual link inter-layer link IP-layer (Virtual Toology) S2 R2 S3 R3 otical-layer (Physical Toology) IP/MPLS router flex-grid otical switch S4 R4 flexible transonder grey short reach transceiver Figure 1 Architecture of IP over flexible otical network Regarding the otical network, we assume the use of flexible transonders, also referred to as bandwidth variable transonders (BVTs) that can control a number of arameters, such as the modulation format, the utilized sectrum, the baudrate and the rate that they transmit. The configurations of a transonder of tye t are indicated by transmission tules (d t,r t,b t,c t), where d t is the reach for which a transmission of rate r t (gbs) using b t sectrum slots (including guardband) is feasible (that is, it has accetable QoT). The last arameter c t corresonds to the of the transonder. Note that the definition of a secific rate and sectrum incororates the choice of the modulation format of the transmission. Different tyes of transonders, with different s have different sets of transmission tules, and this definition is generic, so as to be able to formulate any such otion. A fixed transonder can be exressed by a set consisting of a single tule in the above formulation. The transonders are driven by equal rate linecards at the IP/MPLS routers, with each linecard suorting one or more transonders of the same tye. The otical network toology and the IP/MPLS router edges are reresented by a directed grah G=(N, L), where N is the set of nodes and L is the set of links. The grah consists of two tyes of nodes, IP (or virtual) nodes and otical (or hysical) nodes, and two layers, the IP (or virtual) layer, where all the IP nodes are located, and the otical (or hysical) layer, where all the otical nodes are located. A virtual node reresents an IP/MPLS router, while an otical (hysical) node reresents a flex-grid otical switch, assumed to suort F sectrum slots of a given granularity, e.g GHz. In the network grah G, the set L consists of three tyes of links, inter-layer (l ov or l vo), otical (l o) and virtual (l v) links. An interlayer link connects a virtual (electronic/ip) node with an otical node and reresents the use of a (flexible or fixed) transonder, that is, it corresonds to an O/E or E/O conversion. Note that we distinguish between the two directions of the inter-layer links, with l ov denoting an oticalto-virtual and l vo a virtual-to-otical inter-layer link, since we create single direction aths that we make bidirectional in the end. For each tye (one or more) of transonders available, we create corresonding inter-layer links with distinct vectors (we will come back to this in the next section). An otical link l o corresonds to a fiber and connects two otical switches. For otical links we kee track of their sectrum slots availability as will be described in the next Section. A virtual link l v corresonds to a lightath that connects two IP/MPLS routers. Thus, a virtual link between two IP/MPLS routers is created when a lightath with residual caacity is established between the two otical switches connected to the IP/MPLS routers. This lightath can traverse one or more otical links and ass transarently over zero or several intermediate otical nodes. So, at the beginning of lanning, the set L of grah G includes only the otical links (l o) and inter-layer links (both l ov and l vo) connecting IP nodes to otical nodes, while as the network is lanned in iterative stes and lightaths are established the related virtual links l v are added in the grah. Figure 1 resents an illustrative examle of the examined IP over flexible network, where four IP/MPLS routers comrise the IP (virtual) layer, four flex-grid otical switches comrise the otical (hysical) layer and two tyes of transonders are used, each one suorting different transmission tules. Also three lightaths (S 1 S 4, S 1 S 2, and S 2 S 4) have been established at the otical layer, and the three related virtual links (from node R 1 to node R 4, from node R 1 to node R 2 and from node R 2 to node R 4) were created at the IP layer. Note that the oosite direction virtual links were also created, that may have different remaining caacity. A new demand from ingress node R 2 to egress node R 4 can be served by lightaths already established from node S 2 to node S 4, if their remaining caacity is adequate, or by establishing a new otical lightath. IV. JOINT MULTI-LAYER NETWORK PLANNING ALGORITHM In this section, we describe the joint multi-layer network lanning algorithm for IP over flexible otical networks. As stated before, the develoed algorithm is used to serve a single demand and thus in order to serve the whole traffic matrix we iteratively use them for each demand, one-by-one, in a articular order (Section IV.B). Since different orderings for serving the demands may result in different s, simulated annealing techniques can be used to find good orderings. A. Multi- IPR+RML+SA algorithm We now describe the multi- algorithm to jointly solve the IPR+RML+SA roblem for a single demand. The lanning roblem assumes as inut the network toology and the traffic matrix. We order the demands, and serve them oneby-one, using the algorithm resented in this section. So at each ste, the single demand multi- algorithm takes as inut the demand to be served (source or ingress IP/MPLS router, destination or egress IP/MPLS router, and demand rate), the IP over flexible network described by grah G, the transonders feasible configurations (described by the set of related tules) and the state of the network, in terms of reviously established lightaths (including the sectrum

5 utilization of links) and IP routing decisions. Following the multi- aroach we define for each tye of link (interlayer, otical and virtual) a vector that incororates information regarding both the otical and the virtual (IP) layers (Section IV.A.1.1). Aroriate oerators are also defined to combine the vectors of links to obtain the vector of aths (Section IV.A.1.2). The algorithm carries out two stes: in the first ste (Section IV.A.2.1) it calculates the vectors of candidate non-dominated aths from the source to the destination. A domination relationshi is used to rune the solution sace by removing aths that would not be selected by the otimization function (second ste). Then, in the second ste (Section IV.A.2.2) an otimization function is alied to the vectors of the candidate aths found in ste 1, which transforms the multi- vectors into single- scalars, and the algorithm selects the otimum. A.1 Comuting the vector of a ath 1) Cost vector of a link Each link is assigned a vector consisting of 6 arameters, two of which are actually vectors. These arameters are the following: An integer variable D l reresenting the length of the link. The length of a virtual (l v) or an inter-layer link (l ov or l vo) is equal to 0. An integer variable C l reresenting the transonder s. In case of an inter-layer link (l ov or l vo), C l is equal to the of the articular transonder used (when multile transonder tyes are available, we create one inter-layer link for each tye), and 0 for other tyes of links. A float variable P l reresenting the additive of the modular router (having as reference its u to this oint), which again is non-zero only for an inter-layer link (l ov or l vo), and zero otherwise. For the given state of the network, the IP/MPLS router at the start of a l vo link, or the end of a l ov link, has a number of chassis, linecards and free orts. A new linecard must be installed when there are no free orts (all installed linecards are full), and a new chassis must be installed when all the slots of the chassis are used. So, (i) when there are free orts at the router, P l=0, (ii) when linecards are full but the chassis is not, P l is set to the of a new linecard of the related rate, and (iii) when we need to add a chassis, P l is set to the additive of the chassis and a linecard. Note that the of adding a chassis might not be linear. H r, d, b, r, d, b,..., r, d, b of A vector l m m m size 2m (m is the number of feasible transmission tules), defined only for a virtual-to-otical inter-layer link (tye l vo), whose i-th element (r i,d i,b i) records a transmission tule, where d i is the feasible reach for rate r i and sectrum b i for the secific transonder. These are taken from the transmission tules of the corresonding transonder that the inter-layer link reresents. Note that the vector H l is defined only for links of tye l vo, and is emty for the other direction (l ov) and other tyes of links. A Boolean variable F l that is equal to 1 if l is a virtual link (reresenting an established lightath), and 0 otherwise. W l A Boolean vector of size F (F is the number of sectrum slots) reresenting the slot availability of the otical links. In articular element W l,i is equal to 1 if the i-th slot on otical link l is available and 0 otherwise. For all other links (inter-layer and virtual links), the vector has elements equal to 1. Thus, the vector 2) Cost vector of a ath V l characterizing a link l is given by: V l D, C, P, H l, F, W l (1) l l l l A ath in the grah is built link by link, so that at each ste a ath is extended by adding a virtual, otical or inter-layer link. We assume that a ath consists of a number of subaths m, where each sub-ath m is defined as the ath between two consecutive IP nodes (a sub-ath corresonds to a lightath). A sub-ath can be a virtual link (an already established lightath), or a new lightath (a sequence of a virtual-to-otical inter-layer link, followed by one or more otical links, followed by an otical-to-virtual inter-layer link). For each sub-ath m of the latter category (new lightath), the algorithm decides on its rate, denoted by r max, which is chosen the maximum rate of the tules available when reaching the sub-ath s terminating IP node. Similarly to a link, a ath is characterized by a vector V D, C, P, H, F, W, R,, whose six first arameters are the reviously described ones for a link, lus two new arameters: denoting the set of the chosen rates R r max of the revious sub-aths, and defining the list of identifiers of the links that comrise ath. Note that arameter D reresents the otical length of the current subath, that is the length of the otical links from the last IP/MPLS router (last oint where E/O conversion was erformed). Assume that we extend ath by adding link l to obtain ath. The vector of is calculated using the associative oerator,,,, to and, lov lvo l o l v V deending on the tye of the link l. To be more secific: if l is an otical link (l o) or a virtual link (l v) V ' V ( ) V D D, C C, P P,{( r, d, b ) H lo lv l l l l i i i D D d and Q( W & W l ) b }, F F, W & W l, R,{, l} (2) l i i l if l is a virtual-to-otical inter-layer link (l vo) V ' V V 0, C C, P P, H l, F F,1, R,{, l } (3) lvo l l l l if l is an otical-to-virtual inter-layer link (l ov) ' l ov l l l l l R rmax ri ri di bi H Q W bi l i V V V D D, C C, P P,, F F, W & W l, V l, max (,, ) and ( ),{, } (4) where & denotes the bitwise Boolean and oeration, and QW ( ) denotes the largest void of the slot availability vector W. Note that the sectrum continuity constraint is enforced in sub-aths by the definition of the slot availability vector and the & oeration. The values of the arameters of the

6 links differ deending on the tye of link. Thus, e.g. although in both otical and virtual links the length of the new ath is defined as the sum of the length of ath lus the length of the additional link l (D =D +D l), D l is zero for a virtual link and equal to the length of the fiber for an otical link. We used different of definitions associative oerators for the links in order to erform secial actions: to reset the length (D =0), reinitialize the transmission tule set ( H ' Hl) and reset the sectrum availability ( ) when extending the ath by adding a virtual-to-otical link, and to reset the transmission tule set ( W ' 1 H ' ) and select the maximum rate tule r max ( max r ( r, d, b ) H and Q( W ) b ) when extending the ath i i i i i i by adding an otical-to-virtual link. A.2 Single demand algorithm descrition The roosed multi- routing algorithm consists of 2 stes, the first of which comutes the set of non-dominated candidate aths to serve the demand, and the second one selects the otimal ath. We assume that the we know the network toology, the current state of the network (established lightaths, sectrum slots utilization, used router modules) and the feasible transmission configurations of the available flexible (or fixed) transonders. The algorithm runs for a secific demand with source s and destination d, both IP (virtual) nodes of grah G, and a demanded rate r. In case a demand requires rate bigger than the one suorted by the transonders, then it is slit to sub-demands of the suorted rates, and the algorithm is executed that many times. The algorithm constructs a reduced grah G A = (N A, L A) from G, which includes all nodes and all links excet for the virtual links (established lightaths) that have remaining caacity lower than the demanded r. 1) Ste 1: Comuting the set of non-dominated aths P n-d The goal of this ste is to find a set of candidate aths for efficiently serving the demand. The algorithm that comutes the set of non-dominated aths P n-d runs on the reduced grah G A. For given source and destination nodes s,d N A, we define: M i as the set of vectors of the aths from node s to node n i N A. M M as the set of all vectors to all nodes. is f i i f f is i i M M as the set of final vectors to node n i. M M M as the set of all final vectors to all nodes. max R max( r R ) as the maximum rate among all m m sub-aths of, that is, among all elements of R. This algorithm also utilizes a domination relationshi between aths that have the same end-node (and same source by definition) to reduce the number of aths considered. In articular, we say that ath 1 dominates ath 2 with the same end-node (notation 1> 2), if the following holds: 1 2 iff F F 1 2 L L and and 1 2 C and P C P max max R R and W W (5) Α ath that dominates another ath has smaller length, less additive network ( of transonders and additive of routers), utilizes less virtual links, has higher maximum rate among its sub-aths and has available at least the same sectrum slots. Since the otimization function f to be alied to the vectors of the aths in the second ste of the multi routing algorithm is monotonic in each of these s, the dominated aths would have never been selected, so the solution sace along with the execution time of the algorithm will be reduced, without losing otimal solutions. Pn-d Comute_the_set_of_Non_Dominated_Paths (GA,s,d,r, ##Initialization M f {}, M {} for all links M M l L A f starting at s N A, M M V l while ##Choose the otimum ath vector find ath in M whose vector has minimum network. In case of link V tie, select the one with minimum number of virtual links, etc. n ending node of selected ath f f M M V ##Obtain new aths and discard dominated aths for all l n, n j L ) n j N A neighbors of n (connected through A ##Obtain the vector of the new ath j V j ' V V l, with or o or or v vo ov V l ) deending on the ##Discard the new ath j if it runs out of transmission tules if (l is otical link) and ({( ri, di, bi ) H j ', ri r and bi Q( W )} j ' ) n j check the next neighbor end if ##Check if the new ath j is dominated for all ( j are aths ending at node n j) V j M i ' if j j ( > is the domination relationshi) check the next neighbor n j (discard the new ath j ) ' else if j j M M { V j }(discard the old ath j) end if end for M M { V j '}, (add the new ath j to the set of aths) end for end while f end algorithm (return P M ) nd d Figure 2 Pseudo-code of the algorithm which comutes the set of nondominated aths from the given source to the given destination The algorithm used to comute the set of non-dominated aths is a generalization of Dijkstra s algorithm that only considers scalar link s and is described in seudo-code in Figure 2. It first obtains a non-dominated ath between the source and a direct neighbor node. This ath is selected so as to have the smallest network, in case of a tie, the smallest number of virtual links, etc. By definition, this ath is nondominated since the arameters that comrise the vectors are additive and non-negative and this ath has at least one arameter smaller than the other aths. The algorithm marks this ath as final, and extends it through the outgoing links of its end node, so as to calculate new aths and their vectors using the aroriate associative oerator, according to the added link. Two additional checks are erformed when adding an otical link: we check if the extended ath has at least one transmission tule (i) with higher rate than the required and (ii) requiring bandwidth smaller than the largest sectrum void ( i ( ri, di, bi ) H j ', ri r and bi Q( W ) j ' ), and discard it if one of the above does not hold.

7 Then, the algorithm selects a non-final ath between the source and one of its neighbors, or between the source and one of the neighbors of the reviously considered neighbor, marks it as final, extends it using the corresonding outgoing links, calculates new aths, and so on. For each new ath that is calculated, the algorithm alies the domination relationshi between the new ath and all the aths with the same end node that have been reviously calculated. The new ath is discarded if it is dominated by one of the reviously calculated aths; otherwise, it is added to the set of non-dominated aths of the secific end node, while all the reviously calculated aths that are dominated by it (if any) are discarded. The basic difference with Dijkstra s algorithm is that a set of non-dominated aths between the source and each node is obtained instead of a single ath; a node for which a ath has already been found is not finalized (as in the simle Dijkstra case), since it may have to be considered again later. Actually, instead of finalizing nodes, we finalize the aths. The algorithm finishes when no more aths can be extended (all aths are final) and returns the set of non-dominated aths P n-d that has been calculated between the source s and destination d. V 1 P 1 R l=80 Gbs L l=700 km V 2 L l=1600 km L l=500 km P 2 V 3 P 3 L l=1200 km L l=500 km V 4 P 4 V 5 L l=500 km R l=100 Gbs P 5 L l=500 km L l=600 km Figure 3 - IP over flexible network grah V 6 P 6 L l=500 km Ll=300 km For examle consider the network with grah G of Figure 3. We suose a transonder configuration with km reach for a transmission of 100 Gbs. Also four lightaths have been established: (a) from virtual node V 1 to node V 3 through nodes P 1P 2P 3, with remaining caacity R l=80 Gbs, (b) from virtual node V 2 to node V 7 through nodes P 2P 5P 7, with remaining caacity R l=100 Gbs, and the oosites, (c) from V 3 to V 1 through nodes P 3P 2P 1 with remaining caacity R l=80 Gbs, and (d) from V 7 to V 2 through nodes P 7P 5P 2 and remaining caacity R l=100 Gbs. Assume that we want to serve a new demand with rate 100 Gbs from V 1 to V 7. We construct the reduced network grah G A, where we remove the virtual links V 1V 3 and V 3V 1, as their remaining caacity is not sufficient. The algorithm begins by initializing the set M with the link V 1P 1. Then it examines the aths in set M, selects V 1P 1 (since it is the only one available), and makes it final (adding it to set M f ). Then the algorithm exands V 1P 1 to all the neighbors of node P 1, that is using the links P 1P 2, P 1P 3 and P 1P 4. The ath V 1P 1P 3 is discarded as its length exceeds the maximum transmission reach (no remaining transmission tule has rate higher than the demanded). The vectors of the two new aths V 1P 1P 2 and V 1P 1P 4 are comuted and inserted in the set M. Then the algorithm will search the set M, select V 1P 1P 2 that has the shortest length, finalize it and exand it. Links P 2V 2, V 7 P 7 P 2P 3 and P 2P 5 will be examined. Path V 1P 1P 2P 3 will be discarded (as its length which is km exceeds the maximum transmission reach of transonder which is km), while the other two will be included in set M. Note that the length of the ath V 1P 1P 2V 2 will be reset to zero, enabling it to reach P 3 at a later iteration. Also ath V 1P 1P 2V 2 when selected will also be exanded using the virtual link V 2V 7 (reviously established lightath). The algorithm will continue its execution by selecting at each ste the best non-final ath comuted u to that oint, and examining its neighbors. When more than one aths reach the same end node, it will aly the domination relationshi to discard those that are worse with resect to all arameters. The execution of the algorithm is comleted when M=M f, that is, when there are no more non-final aths to exand. 2) Ste 2: Selecting the otimal ath In the second ste of the multi- routing algorithm, we aly an otimization function f( ) to the vector of all aths in P n-d calculated in the first hase. The function f yields a scalar er candidate ath enabling us to select the otimal one. Note that the otimization function has to be monotonic in each of the comonents. We used an otimization function that calculates for each candidate ath the additive network (C +P ) and selects the one with the smallest, while in case of tie, the ath with the smallest number of virtual links (established lightaths) is selected, and in a case of a tie, the one with the maximum rate among all sub-aths. Note that in aths creation rocess (first ste) we have already taken decisions on what transmission tule to use for each sub-ath (the one with the highest rate from the ones available which also requires bandwidth that is smaller than the longer sectrum void). We decided to focus on and not to consider the sectrum in the otimization function defined above, but the algorithm has verified that the candidate aths assed in this hase, have at least the required sectrum slots for each sub-ath to establish the related lightaths. In this study we use the first-fit olicy to allocate sectrum to each sub-ath, but other olicies can be used, such as most-used, best fit, etc. The algorithm can be modified to take into account the sectrum at sub-ath tule selection and/or at the otimization function, which could decide not only the ath to use but also otimize sectrum allocation jointly. Finally, note that different otimization functions can be defined for all demands, but also we can have different functions for different demands, deending on demand s QoS requirements and other considerations; for examle, we could choose the ath with minimum number of virtual links, or the one that achieves the higher added caacity over ratio, to name just a few. B. lanning - iterative execution of algorithm The roosed joint multi-layer network algorithm described above is designed to serve a single demand. To serve the whole traffic matrix and lane the whole network we order the demands and serve them one-by-one. We kee track of the modules installed at the routers, the established lightaths, the assignment of demands to them, their remaining caacity, and the sectrum utilization of the links. When a demand is served we udate accordingly the above data structures and the grah G and then move to serve the next demand. Since the order in which the demands are served lays an imortant role, simulated annealing can be used to search among different orderings to imrove the solution [13]. V V

8 This article has been acceted for ublication in a future issue of this journal, but has not been fully edited. Content may change rior to final ublication. Citation information: DOI C. Generality of the roosed algorithm Although the algorithm described above is used for solving the joint multi-layer network lanning roblem in IP over flexible networks, it is quite generic and can be also used for (a) lanning otical networks deloying fixed-grid or flex-grid otical switches and fixed otical transonders of single or multile tyes (SLR or MLR), (b) sequential lanning of IP over flexible or fixed-grid otical networks (what we call sequential multi-layer network lanning, SML-NP) and (c) minimizing network arameters other than the, for examle the energy consumtion. To be more secific, to lan a SLR or MLR network, the algorithm takes as inut the descrition of the caabilities of the fixed transonders in the tule format, discussed in Section III. This is quite straightforward; the fixed rate transonders of a SLR network are described with a single transmission tule, while the fixed rate transonders of a MLR network are described by a set of single tules, and in articular the set consists of as many tules as the number of transonder tyes available. In case that we want to sequentially lan the IP and otical network (SML-NP), we consider that the IP routing decisions (IPR) are taken without taking into account the distance constraints and the RML decisions. To imlement this, in the case of SML-NP we set the reach of the flexible (or fixed) transonders equal to infinity and use an otimization function that does not include the of the otical layer. When solving the RML roblem for SML-NP case we use an otimization function that neglects the IP/MPLS router s s. Moreover, the inter-layer links are assumed to use transonders only at the source and the destination, while all other interlayer links are assumed to be deloyed with otical regenerators. To be more secific the same rocess is followed in both JML-NP and SML-NP cases. At the end of the lanning algorithm, all intermediate transonders that are not used for grooming uroses are relaced by regenerators (to save in ). Since no grooming is erformed in the SML-NP case at intermediate nodes, all intermediate transonders become regenerators, while in JML-NP, this relacement deends on the traffic and is seldom erformed. In future we lan to integrate regeneration lacement in the algorithm formulation. Finally, to minimize other network arameters, like energy consumtion, aroriate changes must be done at the structures used at the roosed algorithm. More secifically, a) the link and ath arameters of vector should be relaced with corresonding energy consumtion arameters, like energy consumtion of (flexible or fixed) transonders and additive energy consumtion of routers, b) in the domination relation alied at the first ste of the multi- algorithm, the of aths should be relaced by the energy consumtion of aths, and c) in the otimization function alied at the second ste of the multi- algorithm, the network should be relaced by the network energy consumtion, to select the ath with the smallest additive network energy consumtion. V. PERFORMANCE RESULTS To evaluate the erformance of the roosed algorithm, we conducted a number of simulation exeriments using Matlab. At the exeriments resented below, we define the following cases of networks: (b) MLR otical network emloying 12.5 GHz flex-grid otical switches and fixed 40 Gbs, 100 Gbs and 400 Gbs transonders (flex-grid/fixed-), (c) flexible otical network emloying 12.5 GHz flex-grid otical switches and flexible transonders, also referred to as BVTs (flex-grid/flex-). The reason that in the first case we are not assuming the use of 400 Gbs transonders is that such devices are exected to require 75 GHz sectrum, which does not fit in traditional 50 GHz fixed-grid WDM systems. We also distinguished the case where a joint or sequential multi-layer network lanning (JML-NP or SML-NP, resectively) solution is alied to each of the above networks. For all these cases (three different tyes of networks and the two lanning otions), we evaluate the roosed algorithm that is quite generic (as discussed in Section IV.C). Figure 4 - Generic DT toology with 12 nodes and 40 directed links We used two reference networks in our simulations with different characteristics in terms of number of nodes and link lengths: the Deutsche Telekom (DT) and the GEANT network, so that the results obtained are reresentative of real networks. DT (Figure 4) is an IP backbone network with 12 otical nodes, a single IP core router er otical node, and 20 link edges (40 bidirectional edges) with an average length of 243 km and maximum length of 459 km. It interconnects PoPs and it serves internal traffic generated and consumed by residential subscribers exclusively. GEANT (Figure 5) is an IP backbone network with 34 otical nodes and 54 link edges (108 bidirectional edges) with an average length of 752 km and maximum length of 2361 km. The GEANT toology used at the simulations is that of February 2009, as we have not available information for later years. For the case of DT network, we created traffic matrices for the eriod from 2014 u to 2024 based on real traffic for 2011, assuming 35% uniform increase er year, while for the case of GEANT network, we created traffic matrices for the eriod from 2014 u to 2024, based on real traffic for 2009, assuming 25% uniform increase er year. The erformance metrics we used for the comarisons, are the maximum sectrum used (measured in GHz), the of transonders, the of routers and the total of network comuted as the sum of transonders and routers, which is our main focus. (a) MLR otical network emloying fixed-grid 50 GHz otical switches and fixed 40 Gbs and 100 Gbs transonders (fixed-grid/fixed-),

9 of a multi-chassis router is comuted according to the following equation: nch nch P 6, 02 nch 1, 76 9,11, 9 3 (6) C nch, 2 nch 72 K where n ch is the number of linecard chassis installed, C is the total switching caacity in Tb/s, and K is the caacity of a fully equied linecard chassis (taken to be Gbs here). The s of transonders, linecards and routers used in our simulations are derived from the CAPEX model defined in the context of the EU roject IDEALIST [4]. Reach (Km) Caacity (Gb/s) Required Sectrum (in GHz) Cost (ICU) Figure 5 - Generic GEANT toology with 34 nodes and 108 directed links In the case of the flex-grid networks (flex-grid/fixed- and flex-grid/flex-), each fiber of the DT network has 320 sectrum slots available, each of width 12.5 GHz. For GEANT we assumed two fibers with a maximum of 640 sectrum slots available. In the case of the fixed-grid network (fixedgrid/fixed-), we assumed 80 wavelengths of 50 GHz width each for the DT network, and 2 fibers for the GEANT with 160 wavelengths in total. For the GEANT network we assumed that the sectrum in both fibers is 4THz, but we cannot switch lightaths from one fiber to the other, so in this sense the sectrum can be considered as a continuous 8THz-wide segment. The reason for making this choice is that currently the devised algorithm considers a single contiguous set of sectrum to model the otical links, but note that enabling switching between the fibers would increase the network efficiency. The roosed algorithm can be extended to integrate this additional degree of flexibility, but this was omitted for the sake of brevity. The transmission tules (reach, rate, sectrum, ) of the used flexible and fixed transonders (bandwidth variable and fixed bandwidth transonders) are shown in Table 1 and Table 2 resectively, and are based on [19], [20], [21]. Note that the transmission tules shown in Table 1 concern a single tye of flexible transonder. In case of SML-NP, the of the regenerators was set as 80% of transonder [18]. Note that the of flexible transonders was taken to be 30% higher than that of the equivalent 400 Gbs fixed rate transonder. The different tyes and the related of the linecards (assumed to be 400 Gbs caacity in total) are shown in Table 3. Note that we assumed that flexible transonders are always interfaced with 400 Gbs linecards, although they could be configured to transmit at lower loads. The of an IP/MPLS router with 1 chassis of 16 slots is shown in Table 3, while the Reach (Km) , , , , , , , , , , , , , Table 1 Transmission tules of flexible transonders Caacity (Gb/s) Required Sectrum (in GHz) Cost (ICU) Table 2 - Transmission tules of fixed-grid transonders Tye Core router with 1 chassis of 16 slots and 400 G caacity er slot Cost (ICU) x40G linecard for core router x100G linecard for core router x400G linecard for core router 2.74 Table 3 - Cost of linecards and single chassis IP/MPLS routers in Idealist Cost Units (ICU)