Aternative Decompositions for Distributed Maximization of Network Utiity: Framework and Appications Danie P. Paomar and Mung Chiang Eectrica Engineering Department, Princeton University, NJ 08544, USA Abstract Network utiity maximization (NUM) probems provide an important approach to conduct network resource management such as end-to-end rate aocation. In the existing iterature, distributed impementations are typicay achieved by the means of the so-caed dua decomposition technique. However, the span of decomposition possibiities incudes many other eements which thus far have not been fuy expoited such as the use of the prima decomposition technique, the versatie introduction of auxiiary variabes, and the potentia of mutieve decompositions. This paper presents a systematic framework to expoit the potentia of the aternative decomposition structures as a way to obtain different distributed agorithms, each with a different tradeoff among convergence speed, message passing amount and asymmetry, and distributed computation architecture. Many specific appications are considered to iustrate the proposed framework, incuding resource-constrained and directcontro rate aocation, and rate aocation among QoS casses and with mutipath routing. For each of these appications, the associated generaized NUM formuation is first presented, foowed by the deveopment of nove aternative decompositions and numerica experiments on the resuting new distributed agorithms. Keywords: Rate contro, Congestion contro, Resource aocation, Mathematica programming/optimization, Network utiity maximization, Distributed agorithm, Network contro by pricing. I. INTRODUCTION A. Motivation Why woud one care about a systematic theory of aternative decompositions for variants of Network Utiity Maximization (NUM) probems? There are two main reasons: it eads to the most appropriate distributed agorithm for a given network resource aocation probem, and it quantifies the comparison across architectura aternatives of distributed, ayered network contro. First, since the pubication of the semina paper [1] by Key, Mauoo, and Tan in 1998, the framework of NUM has found many appications in network resource aocation agorithms and Internet congestion contro protocos, e.g., [], [3], [4], [5], [6], [7]. The key innovation from this series of work is to interpret source rates as prima variabes, ink congestion prices as dua variabes, and a TCP Active Queue Management (AQM) protoco as a distributed agorithm over the Internet to sove an impicit, goba utiity maximization and its Lagrange dua probem. Different TCP-AQM protocos sove for different concave utiity functions using different ink prices. This mode impies that the equiibrium properties of a arge network under TCP/AQM contro, such as throughput, deay, queue engths, oss probabiities, and fairness, can be readiy understood by studying the underying noninear utiity maximization probem. In addition to this reverse engineering direction, aocation of imited network resources, such as power, bandwidth, and rate, among competing users can aso be formuated by generaizing the basic NUM in [1] to more sophisticated formuations. Amost a the papers in the vast, recent iterature on NUM use a standard dua-based distributed agorithm. Contrary to the apparent impression that such a decomposition is the ony possibiity, there are in fact many aternatives to sove a given network utiity probem in different but a distributed manners. Each of the aternatives provides a possiby different tradeoff among three important considerations: convergence speed, amount and asymmetry of message passing s communication overhead, and architecture of distributed computation. There is no universay best way to distribute the soution process across a network: which aternative is the most desirabe depends on the specific probem formuation. Thus motivated, we deveop a systematic framework of aternative decompositions in this paper and appy it to four network rate aocation probems motivated by practica needs and constraints. Second, the framework of NUM has recenty been substantiay extended from an anaytic too of reverse-engineering TCP congestion contro to a genera approach of understanding interactions across ayers. One possibe perspective to rigorousy and hoisticay understand ayering is to integrate the various protoco ayers into a singe coherent theory, by regarding them as carrying out an asynchronous distributed computation over the network to impicity sove a goba optimization probem. Different ayers iterate on different subsets of the decision variabes using oca information to achieve individua optimaity. Taken together, these oca agorithms attempt to achieve a goba objective. This approach exposes the interconnection between protoco ayers and can be used to study rigorousy the performance tradeoff in protoco ayering, as different ways to distribute a centraized computation. Since the design of a compex system wi aways be broken down into simper modues, a ayering as optimization decomposition theory wi aow us to systematicay carry out this ayering process and expicity trade off design objectives.
Each different decomposition represents a new possibiity of network architecture. But to deveop such a theory, aternative decompositions must be fuy expored to understand architectura possibiities, both verticay across functiona modues, i.e., the ayers, and horizontay across disparate network eements. This paper primariy studies aternatives of horizonta decompositions, athough some resuts are directy appicabe to vertica decompositions as we, e.g., the resuts in Section VI can be readiy appied to joint TCP and MAC design in [8]. B. Existing Work There are at east three eves of understanding as to what it means to efficienty sove a utiity maximization probem. First, a convex optimization (minimizing a convex function over a convex constraint set) is easy to sove because a oca optimum must aso be gobay optima, whereas a nonconvex one is very difficut [9]. Second, there are provaby poynomia-time but centraized agorithms, such as the interior-point method, to sove a convex optimization. Third, distributed agorithms can be found to converge to the goba optimum. It is the third eve that we concern ourseves in this paper. There is indeed a arge body of resuts on distributed computation, some of which are summarized in standard textbooks such as [10], [11], [1], [13] and others. Our goa here is certainy not to survey these known genera resuts in inear programming, graph theory, or decomposabe probems. Instead we focus on the engineering probems of network rate aocation through probems in the form of noninear, couped NUM, and deveop nove distributed agorithms through a systematic method of aternative decompositions. The semina paper [1] in 1998 outines two major casses of approaches to sove the basic version of NUM: prima-based and dua-based. It is important to note that both approaches in [1] adopt a differentia equation technique, anayzed through penaty function and Lyapunov argument, thus different from the anguage of prima and dua decomposition in this paper. Simiar to one of the first pubications in reverse-engineering TCP congestion contro [6], many recent papers on distributed resource aocation with optimization modes are based on Lagrangian reaxation and one-eve, fu dua decomposition. In fact, as iustrated in this paper through many appications, this standard dua decomposition is ony one of the many choices one can make, incuding muti-eve, indirect, and hybrid prima-dua decompositions. Despite its popuarity, the standard dua decomposition may not be the best choice. It is aso important to notice that the term prima-dua agorithm is used in [3] to describe the purey dua-based agorithm because both the prima probem and the Lagrange dua probem are being soved simutaneousy. This is different from both the prima-dua interior-point method in centraized soution of convex optimization [9] and the prima/dua decompositions for distributed agorithms deveoped in this paper. Prima decomposition has remained in the shadow of dua decomposition and its empoyment is scarce, athough it is just TABLE I SUMMARY OF THE DECOMPOSITIONS CONSIDERED IN THE APPLICATIONS ( DENOTES EXISTING ALGORITHM AND NEW ALGORITHM). Section Prima Fu Dua Partia Dua Prima-Dua Dua-Dua II-E - - - - III - - - IV - - - V - - - VI - - starting to take off in wireess transceiver design and power contro probems. Recent exampes incude: [14], where inear transceivers for communication through MIMO channes were designed to minimize the average BER; [15], where inear MIMO transceivers were designed for muticarrier systems; [16] where different distributed agorithms were obtained in the context of wireess power aocation; and [17], where both a prima and a dua decomposition were considered for resource aocation. However, none of these pubications present the decomposition aternatives for distributed rate aocation probems in Sections IV, V, and VI of this paper. C. Summary of Resuts We first present a systematic framework in Section II for aternative decompositions and how that woud ead to an array of choices of distributed agorithms. Section II thus serves both as a review of the necessary background and a summary of our new extensions in decomposition theorems. In particuar, Lemmas 1 and 3 extend existing resuts on subgradients, and the techniques of mutieve and indirect prima/dua decompositions are systematicay introduced in the context of NUM probems. The core of this paper then consists of Sections III to VI, covering four appications of distributed rate aocation: power-constrained rate aocation in Section III, rate aocation among different quaity-of-service (QoS) groups in Section IV, hybrid rate-based and pricing-based rate aocation in Section V, and rate aocation with mutipath routing in Section VI. In particuar, the distributed agorithms obtained in Subsections IV-B, IV-C, V-C, VI-B, and VI-D are new. The types of decompositions deveoped in each appication are summarized in Tabe I (when there are two eves of decompositions, they are separated by a dash, and for simpicity of terminoogy, we differentiate between fu and partia dua decomposition in the name ony in decompositions with one eve). In a appications, after the optimization formuation is ceary expained, we deveop aternative decompositions and show the benefits of fuy exporing the space of possibe distributed agorithms. In some cases the distribution of computationa oad and asymmetry of message passing are much more desirabe in one of the possibe aternatives, and in other cases the convergence can be acceerated as confirmed in the numerica exampes in Section VII.
Origina Probem Decomposition Subprobem 1 Master Probem...... prices/resources Subprobem N Fig. 1. Decomposition of a probem into severa subprobems controed by a master probem through prices (dua decomposition) or direct resource aocation (prima decomposition). II. SYSTEMATIC FRAMEWORK FOR DECOMPOSITIONS: REVIEW AND EXTENSIONS We first present a systematic framework to decompose a given optimization probem. In the rest of this paper after this section, we wi see how different combinations of the basic eements in Subsections II-A to II-C ead to different distributed agorithms in network utiity probems, among which one wi typicay be preferabe to the others depending on the specific appication. Whie most of the concepts in this section are quick summaries of known resuts (e.g., Subsections II-D and II- E), a coupe of extensions are aso carried out (e.g., Lemmas 1 and 3) and some new techniques that wi be very usefu ater in this paper are introduced (e.g., Subsections II-B and II-C). The basic idea of a decomposition is to decompose the origina arge probem into distributivey sovabe subprobems which are then coordinated by a master probem by means of some kind of signaing (see Fig. 1) [13], [18], [10]. Most of the existing decomposition techniques can be cassified into prima decomposition and dua decomposition methods. The former is based on decomposing the origina prima probem, whereas the atter based on decomposing the Lagrange dua of the probem [19], [18]. Prima decomposition methods have the interpretation that the master probem directy gives each subprobem an amount of resources that it can use; the roe of the master probem is then to propery aocate the existing resources. In dua decomposition methods, the master probem sets the price for the resources to each subprobem which has to decide the amount of resources to be used depending on the price; the roe of the master probem is then to obtain the best pricing strategy. A. Direct Prima and Dua Decompositions A prima decomposition is appropriate when the probem has a couping variabe such that, when fixed to some vaue, the rest of the optimization probem decoupes into severa subprobems. Consider, for exampe, the foowing probem: i f i (x i ) y,{x i} x i X i A i x i y y Y. i (1) Ceary, if variabe y were fixed, then the probem woud decoupe. Therefore, it makes sense to separate the optimization in (1) into two eves of optimization. At the ower eve, we have the subprobems, one for each i, in which (1) decoupes when y is fixed: f i (x i ) x i x i X i () A i x i y. At the higher eve, we have the master probem in charge of updating the couping variabe y by soving: y i f i (y) (3) y Y where fi (y) is the optima objective vaue of probem () for a given y. If the origina probem (1) is convex (meaning that the objective function is concave and the feasibe set is convex), then the subprobems as we as the master probem are a convex. If the function i f i (y) is differentiabe, then the master probem (3) can be soved with a gradient method. In genera, however, the objective function i f i (y) may be nondifferentiabe and a subgradient method is a convenient approach which ony requires the knowedge a subgradient 1 for each fi (y) as given by [18, Sec. 6.4.][13, Ch. 9] s i (y) =λ i (y), (4) where λ i (y) is the optima Lagrange mutipier corresponding to the constraint A i x i y in probem (). The goba subgradient is then s (y) = i s i (y) = i λ i (y). The subprobems in () can be ocay and independenty soved with the knowedge of y. A dua decomposition is appropriate when the probem has a couping constraint such that, when reaxed, the optimization probem decoupes into severa subprobems. Consider, for exampe, the foowing probem: i f i (x i ) {x i} x i X i i h i (x i ) c. i, (5) Ceary, if the constraint i h i (x i ) c were absent, then the probem woud decoupe. Therefore, it makes sense to reax the couping constraint in (5) as {x i} x i X i i i f i (x i ) λ T ( i h i (x i ) c) such that the optimization separates into two eves of optimization. At the ower eve, we have the subprobems, one for each i, in which (6) decoupes: x i f i (x i ) λ T h i (x i ) x i X i. 1 Given a convex function f, a vector s is a subgradient of f at a point x if f (z) f (x) +(z x) T s, z [13], [18]. For a concave function, the inequaity in the previous condition is in the opposite direction. (6) (7) 3
At the higher eve, we have the master dua probem in charge of updating the dua variabe λ by soving the dua probem: minimize g (λ) = λ i g i (λ)+λ T c (8) λ 0 where g i (λ) is the dua function obtained as the maximum vaue of the Lagrangian soved in (7) for a given λ. This approach is in fact soving the dua probem instead of the origina prima one. Hence, it wi ony give appropriate resuts if strong duaity hods (e.g., when the origina probem is convex optimization and there exists stricty feasibe soutions [9]). If the dua function g (λ) is differentiabe, then the master dua probem in (8) can be soved with a gradient method. In genera, however, it may not be nondifferentiabe and a subgradient method is a convenient approach which ony requires the knowedge a subgradient for each g i (λ) as given by [18, Sec. 6.1] s i (λ) = h i (x i (λ)), (9) where x i (λ) is the optima soution of probem (7) for a given λ. The goba subgradient is then s (λ) = i s i (y) +c = c i h i (x i (λ))). The subprobems in (7) can be ocay and independenty soved with knowedge of λ. Genera Resuts. We now present (skipping the proof due to space imit) the foowing new resut to be used ater in the paper, which generaizes the known resut in [18, Sec. 6.4.][13, Ch. 9] (where the particuar resut in (4) is obtained) and gives the subgradient for a more genera case of prima decomposition: Lemma 1: Consider the foowing function defined as the optima vaue of a maximization probem: f (x) sup f 0 (x, y) (10) y:f i(x,y) 0 where f 0 is concave, the f i s are convex, and strong duaity hods for any given x. Then, f (x) is concave 3 and a subgradient is given by s x (x) =s 0,x (x, y (x)) S x (x, y (x)) λ (x) (11) where s 0,x (x, y) is a subgradient of f 0 (x, y) with respect to x, S x (x, y) is a matrix containing in the ith coumn a subgradient of f i (x, y) with respect to x, y (x) is the vaue of y that achieves the supremum in (10) (assumed to exist) for a given x, and λ (x) is the optima Lagrange mutipier associated with the constraints f i (x, y) 0, i, of the maximization in (10) (which is obtained for free each time that f (x) is evauated at some point). We wi aso ater need the foowing we-known resut: Strong duaity can be shown, for exampe, by Sater s condition, which simpy requires (for any given x) the existence of a point y that satisfies the constraints with strict inequaity f i (x, y) < 0, i. 3 Proving concavity of f ony requires concavity of f 0 and convexity of f i, i. Lemma : Consider the foowing dua function defined as the supremum of a partia Lagrangian: { g (λ) sup f 0 (x) } λ i f i (x). (1) x:g i(x) 0 i Then, g (λ) is convex and a subgradient, denoted by s λ (λ), is given by s λi (λ) = f i (x (λ)) (13) where x (λ) is the vaue of x that achieves the supremum in (1) (assumed to exist) for a given λ (which is obtained for free each time that g (λ) is evauated at some point). Note that if there is a unique vaue x (λ) that achieves the supremum in (1) for any given λ, then g (λ) is differentiabe and g (λ) = s λ (λ) (this happens, for exampe, if f 0 is stricty concave and the f i ;s are inear) [18, Prop. 6.1.1]. B. Indirect Prima and Dua Decompositions Often the probem can be reformuated and more effective prima and dua decompositions can be indirecty appied. The introduction of auxiiary variabes provides much fexibiity in terms of choosing a prima or a dua decomposition and the resuting distributed agorithm. The basic techniques are iustrated as foows. Probem (1) contains a couping variabe and was decouped in ()-(3) via a prima decomposition approach. However, it can aso be soved with a dua decomposition by first introducing the additiona variabes {y i }: i f i (x i, y i ) {y i},{x i} x i X i A i x i y i y i = y y Y. i (14) This way, we have transformed the couping variabe y into a set of couping constraints y i = y which can be deat with using a dua decomposition. Consider now probem (5) which contains a couping constraint and was decouped in (7)-(8) via a dua decomposition. By introducing again additiona variabes {y i } the probem becomes: {{y i}x i} i f i (x i ) x i X i h i (x i ) y i i y i c. i, (15) This way, we have transformed the couping constraint i h i (x i ) c into a couping variabe y = [ y1 T,, ] T yt N which can be deat with using a prima decomposition. C. Mutieve Prima and Dua Decompositions An important technique that eads to aternatives of distributed architectures is to appy prima/dua decompositions recursivey: The basic decompositions are repeatedy appied to a probem to obtain smaer and smaer subprobems as 4
Master Probem Then, g (λ) is convex and a subgradient, denoted by s λ (λ), is given by Fig.. Subprobem 1 Secondary Master Probem... prices / resources Subprobem N prices / resources Subprobem First Leve Decomposition Second Leve Decomposition Exampe of a mutieve prima/dua decomposition with two eves. iustrated in Fig.. For exampe, consider the foowing probem which incudes both a couping variabe and a couping constraint: y,{x i} i f i (x i, y) x i X i i h i (x i ) c A i x i y y Y. i (16) One way to decoupe this probem is by first taking a prima decomposition with respect to the couping variabe y and then a dua decomposition with respect to the couping constraint i h i (x i ) c. This woud produce a three-eve optimization probem: a master prima probem, a secondary master dua probem, and the subprobems. Observe that an aternative approach woud be to first take a dua decomposition and then a prima one. Another exampe that shows fexibiity in terms of different decompositions is the foowing probem with two sets of constraints: f 0 (x) x f i (x) 0 i (17) g i (x) 0. One way to dea with this probem is via the dua probem with a fu reaxation of both sets of constraints to obtain the dua function g (λ, µ). At this point, instead of minimizing g directy with respect to λ and µ, it can be minimized over ony one set of Lagrange mutipiers first and then over the remaining one: min λ min µ g (λ, µ). This approach corresponds to first appying a fu dua decomposition and then a prima one on the dua probem. The foowing new resut (proved through Lemmas 1 and ) characterizes the subgradient of the master probem at the top eve. Lemma 3: Consider the foowing partia minimization of the dua function g (λ) =infg (λ, µ) (18) µ where g (λ, µ) is the dua function defined as { g (λ, µ) sup f 0 (x) λ i f i (x) } µ i g i (x). x X i i (19) s λi (λ) = f i (x (λ, µ (λ))) (0) where x (λ, µ) is the vaue of x that achieves the supremum in (19) (assumed to exist) for a given λ and µ, and µ (λ) is the vaue of µ that achieves the infimum in (18) (aso assumed to exist). Aternativey, probem (17) can be approached via the dua but with a partia reaxation of ony one set of constraint, say f i (x) 0 i, obtaining the dua function g (λ) to be minimized by the master probem. Observe now that in order to compute g (λ) for a given λ, the partia Lagrangian has to be d the remaining constraints g i (x) 0 i, for which yet another reaxation can be used. This approach corresponds to first appying a partia dua decomposition and then, for the subprobem, another dua decomposition. When there is more than one eve of decomposition, and a eves conduct some type of iterative agorithms, such as the subgradient method, convergence and stabiity are guaranteed if the ower eve master probem is soved on a faster timescae than the higher eve master probem, so that at each iteration of a master probem a the probems at a ower eve have aready converged. If the updates of the different subprobems operate on simiar timescaes, convergence of the overa system can sti be guaranteed under certain technica conditions [0], [10], and indeed is observed empiricay in the numerica exampes to be presented ater in this paper. D. Review: Subgradient Method After performing a decomposition, the resuting master probem is generay nondifferentiabe. Subgradient methods arise then as exceent approaches to sove these nondifferentiabe probems: they simpy require the vaue of a subgradient at any given point [19], [18]. Subgradient methods are distinguished by their simpicity, itte requirements of memory usage, and amenabiity for parae impementation [19], [18], which are precisey the main interests in this paper. Consider the foowing genera concave maximization over convex constraint set: f 0 (x) x (1) x X. The subgradient method generates a sequence of feasibe points {x (t)} as [18, Sec. 6.3.1]: x (t +1)=[x (t)+α (t) s (t)] X () where s (t) is a subgradient of f 0 (x) at x (t), [ ] X denotes the projection onto the feasibe convex set X, and α (t) is a positive scaar stepsize. Such an iteration ooks ike a gradient projection method except that a subgradient is used instead of the gradient (which may not exist). However, there is a fundamenta difference: each new iteration may not improve the objective vaue as happens with a gradient method. What makes the subgradient method work is that for sufficienty 5
sma stepsize α (t), the distance of the current soution x (t) to the optima soution x decreases. There are many resuts on convergence of the subgradient method [19], [18]. For constant step size α (t) = α and constant step ength α (t) = α/ s (t), the subgradient agorithm is guaranteed to converge to within some range of the optima vaue; in other words, the subgradient method finds an ɛ-suboptima point within a finite number of steps. For the diminishing step size rue α (t) = 1+m t + m, where m is a fixed nonnegative number, the agorithm is guaranteed to converge to the optima vaue. E. Review: Standard Dua-Based Agorithm for Basic NUM Before concuding this section on a systematic framework of aternative decompositions, we briefy review the standard way [3] to sove the basic NUM probem [1] for distributed endto-end rate aocation, which iustrates a simpe appication of the one-eve, fu dua decomposition. In the rest of this paper, we wi see a number of more sophisticated NUM formuations motivated by new appication contexts and a much richer array of decomposition aternatives, beyond the we-known probem and soution method in this subsection. Consider a communication network with L inks, each with a fixed capacity of c, and S sources or nodes, each transmitting at a source rate of x s. Each source s emits one fow, using a fixed set of inks L(s) in its path, and has a utiity function U s (x s ). NUM is the probem of maximizing the tota utiity s U s (x s ), over the source rates x, subject to inear fow constraints s: L(s) x s c for a inks : s U s (x s ) x 0 s: L(s) x (3) s c where the utiities U s are stricty concave functions (the probem is therefore a convex optimization). The standard distributed soution to (3) is based on a dua decomposition. We first form the Lagrangian of (3): L (x, λ) = U s (x s )+ λ c x s s s: L(s) = U s (x s ) λ x s + c λ s L(s) = L s (x s,λ s )+ c λ (4) s where λ 0 is the Lagrange mutipier (ink price) associated with the inear fow constraint on ink, λ s = L(s) λ is the aggregate path congestion price of those inks used by source s, and L s (x s,λ s )=U s (x s ) λ s x s is the sth Lagrangian to be d by the sth source. The dua decomposition resuts then in each source s soving, for the given λ s : x s (λs ) = arg max x s 0 [U s (x s ) λ s x s ] s (5) which is unique due to the strict concavity of U s. The master dua probem is minimize g (λ) = λ s g s (λ)+λ T c λ 0 (6) where g s (λ) =L s (x s (λ s ),λ s ). Since the soution in (5) is unique, it foows that the dua function g (λ) is differentiabe and the foowing gradient method can be used: λ (t +1)= λ (t) α c s: L(s) x s(λ s (t)) (7) where t is the iteration index, α>0 is a positive stepsize, and [ ] + denotes the projection onto the nonnegative orthant. Note that the term c s: L(s) x s (λs (t)) is the gradient of g(λ) with respect to λ. The dua variabe λ (t) wi converge to the dua optima λ as t and, since the duaity gap for (3) is zero and the soution to (5) is unique, the prima variabe x (λ (t)) wi aso converge to the prima optima variabe x. Summarizing, the origina basic NUM probem in (3) can be distributivey soved with the subgradient update in (7) carried out independenty by each the ink and the maximization in (5) soved independenty by each source. Notice that there is no need for expicit message passing since λ s can be measured by each source s as queuing deay and s: L(s) x s can be measured by each ink as the tota traffic oad. III. APPLICATION 1: POWER-CONSTRAINED RATE ALLOCATION We start the appications sections with the simpest and recenty studied extension of the basic NUM, before moving on to more invoved formuations and nove soutions in Sections IV, V, and VI. A. Probem Formuation In some appications such as wireess broadcast or DSL access, distributed rate aocation can be carried out over transmission pipes of different sizes, with the hep of adaptive resource aocation in the physica ayer. This is an exampe of baancing suppy of resources and demand of ink capacities buit from the imited resources. Consider now the basic NUM in (3) but with variabe ink capacities {c (p )}, each of which depends on the aocated resource p, such as transmit power, with a constraint on the maximum tota resource P T. For many modes such as TDMA + 6
or FDMA, c is a stricty concave function 4 s U s (x s ) x,p 0 s: L(s) x s c (p ) (8) p P T. Athough ony sighty more sophisticated than the basic NUM, this probem aready contains sufficient eements such that one can try different decompositions. We wi consider two decompositions: a prima decomposition with respect to the power aocation, and a dua decomposition with respect to the fow constraints. B. Prima-Dua Decomposition Consider first a prima decomposition of (8) by fixing the power aocation p. Ceary, the ink capacities become fixed numbers and probem (8) becomes a basic NUM ike (3), which can be soved via a dua decomposition as expained in Subsection II-E. The master prima probem is p 0 U (p) p P T, (9) where U (p) is the optima objective vaue of (8) for a given p. Since a subgradient of U (p) with respect to c is given by the Lagrange mutipier λ associated with the constraint s: L(s) x s c in (8), it foows that a subgradient of U (p) with respect to p is given by λ c (p ). Therefore, the master prima probem (9) can be soved with a subgradient method by updating the powers as p (t +1)= p (t)+α λ 1 (p (t)) c 1 (p 1 (t)).. λ L (p (t)) c L (p L (t)) P (30) where [ ] P denotes the projection onto the feasibe convex set P {p : p 0, p P T }, which is a simpex. Due to the projection, this subgradient update cannot be performed independenty by each ink and requires some centraized approach. The projection of a point p 0 (the expression inside the outer bracket in (30)) onto the simpex P, i.e., p =[p 0 ] P, can be easiy obtained in the foowing waterfiing form [15]: p = ( p 0 γ) + (31) where the watereve γ is chosen as the minimum nonnegative vaue such that p P T. Observe that ony the computation of γ requires a centra node since the update of each power p can be done at each ink. C. Dua-Dua Decomposition Consider now a dua decomposition of (8) by reaxing the fow constraints s: L(s) x s c (p ): [ ( x,p 0 s U s (x s ) L(s) λ p P T. ) x s ] + c (p ) λ (3) 4 A reated and different mode has been recenty studied in [1]. The prima-dua soution in Subsection III.B was first proposed in [17], and that in Subsection III.C was first proposed in []. This probem decomposes into one maximization for each source, as (5) in the basic NUM, pus the foowing additiona maximization to update the power aocation: λ c (p ) p 0 p (33) P T which can be further decomposed via a second-eve dua decomposition yieding the foowing subprobems p 0 with soution given by λ c (p ) γp (34) p =(c ) 1 (γ/λ ) (35) and a secondary master dua probem that updates the dua variabe γ as [ ( γ (t +1)= γ (t) α P T + p (γ (t)))]. (36) The master dua probem is updated as in the standard NUM (7). D. Summary We have obtained two different distributed agorithms for power-constrained rate aocation in (8): prima-dua decomposition: the master probem (9) is soved with the subgradient power update in (30) carried out by the inks with a sma centra coordination (due to the projection on the simpex) and then, for a given set of powers, the resuting basic NUM is soved via the standard dua-based decomposition in (5) and (7). This impies two eves of decompositions: on the highest eve there is a master prima probem, on a second eve there is a secondary master dua probem, and on the owest eve the subprobems. dua-dua decomposition: the master dua probem is soved with the standard price update in (7) which is carried out independenty by each ink and then, for a given set of prices, each source soves its own subprobem as in (5) and subprobem (33) is soved with some centra node updating the price with (36) and each ink obtaining the optima power with (35). This approach contains two eves of decompositions: on the highest eve there is a master dua probem, on a second eve there are rate subprobems and a secondary master dua probem, and on the owest eve the power subprobems. In both approaches, the ony expicit signaing required is the power-price γ from the centra unit to the inks and possiby the powers from the inks back to the centra node. E. Specia Case: Ceuar Downink Power/Rate Contro An interesting specia case of the signa mode in (8) arises in ceuar downink power/rate contro with the fow 7
constraints on each downink connection modeed in the high SNR regime of a CDMA system with orthogona codes: s U s (x s ) x,p 0 x s og (g s p s ) s p s P T s (37) where g s is the channe gain of the sth user. This probem can be soved in many different combinations of mutieve prima-dua decompositions, each with a different signaing scheme and convergence speed (see Subsection VII-A for an empirica comparison of the convergence of severa methods). IV. APPLICATION : QOS RATE ALLOCATION A. Probem Formuation Sometimes a rate aocation mechanism needs to differentiate users in different QoS casses. For exampe, the tota ink capacity received by each QoS cass must ie within a range prescribed in the service eve agreement. Such constraints introduce new couping to the basic NUM probem and ead to aternative decomposition possibiities. We wi see in this section two different distributed agorithms to sove this type of QoS rate aocation probem, both with a differentia pricing interpretation to the new set of Lagrange mutipier introduced. Therefore, these agorithms provide an intuitive pricing aternative to the recent proposas of NUM-based rate aocation among different QoS casses in [3], [4]. Consider now the basic NUM in (3) but with different casses of users that wi be treated differenty. The idea of having severa casses of users is, for exampe, to impose imits on the maximum rate and to guarantee a minimum rate for each cass. To simpify the exposition we consider ony two casses of users, but the resuts extend straightforwardy to more casses of users. Denoting by y (1) and y () the aggregate rates of casses 1 and, respectivey, aong the th ink, the probem formuation is x,y (1),y () s U s (x s ) 0 s S x i: L(s) s = y (i), i =1, y (1) + y () c c (i) min y(i) c (i) max. (38) Observe that in the absence of the constraints c (i) min y(i) c (i) max, probem (38) becomes the basic NUM in (3). Aso note that if probem (38) is feasibe, then the equaity fow constraints can be rewritten as inequaity fow constraints, as we wi hereinafter assume. We wi consider two decompositions: a prima decomposition with respect to the aggregate rate of each cass, and a dua decomposition with respect to the tota aggregate rate constraints from both casses. B. Prima-Dua Decomposition Consider first a prima decomposition of (38) by fixing the aggregate rates y (1) and y (). Probem (38) becomes two independent subprobems, for i =1,, identica to the basic NUM in (3): s S x 0 i U s (x s ) s S x i: L(s) s y (i) (39) where the fixed aggregate rates y (i) pay the roe of the fixed ink capacities in the basic NUM of (3). These two independent basic NUMs can be soved as expained in Subsection II-E. The master prima probem is ( U ) ( y (1),y () 1 y (1) + U ) y () 0 y (1) + y () c (40) c (i) min y(i) c (i) max i =1, ( where U ) i y (i) is the optima objective vaue of (39) for a given y (i), with a subgradient given by the Lagrange mutipier λ (i) associated to the constraints s S x i: L(s) s y (i) in (39). Observe that λ (i) is the differentia set of prices for the QoS cass i. The master prima probem (40) can now be soved with a subgradient method by updating the aggregate rates as [ y (1) (t +1) y () (t +1) ] = [ [ y (1) (t) y () (t) ] [ + α λ (1) ( y (1) (t) ) λ () ( y () (t) ) Y (41) where [ ] Y denotes the projection onto the feasibe convex set Y { (y (1), y ()) : y (1) + y () c, c (i) min y(i) c (i) Nicey enough, this feasibe set enjoys the property that it ]] } max i =1,. aready naturay decomposes into a Cartesian product for each of the inks: Y = Y 1 Y L. Therefore, this subgradient update can be performed independenty by each ink simpy with the knowedge of its corresponding Lagrange mutipiers λ (1) and λ (), which in turn are aso updated independenty by each ink as in Subsection II-E. C. Partia Dua Decomposition Consider now a dua decomposition of (38) by reaxing the fow constraints s S x i: L(s) s y (i) : [ ( ) ] s S x,y (1),y () 0 1 U s (x s ) L(s) λ x s + ( ) ] s S [U s (x s ) L(s) λ x s +λ (1)T y (1) + λ ()T y () y (1) + y () c c (i) min y(i) c (i) max i =1,. (4) This probem decomposes into one maximization for each source, as (5) in the basic NUM, pus the foowing additiona maximization to update the aggregate rates: y (1),y () 0 λ (1)T y (1) + λ ()T y () y (1) + y () c c (i) min y(i) c (i) max i =1, (43) 8
which can be soved independenty by each ink with knowedge of its corresponding Lagrange mutipiers λ (1) and λ (), which in turn are aso updated independenty by each ink (c.f. Subsection II-E). The master dua probem corresponding to this dua decomposition is updated with the foowing subgradient method (simiary to (7)): [ ( λ (i) (t +1)= λ (i) (t) α y (i) (t) )] + x s (λ(i)s (t)) s S i: L(s), i =1,. (44) D. Summary We have obtained two different distributed agorithms for rate aocation among QoS casses in (38): prima-dua decomposition: the master probem (40) is soved with the subgradient update for the aggregate rate in (41) carried out independenty by each of the inks and then, for a given set of aggregate rates, the two resuting basic NUMs are independenty soved via the standard dua-based decomposition in (5) and (7). This impies two eves of decompositions: on the highest eve there is a master prima probem, on a second eve there is a secondary master dua probem, and on the owest eve the subprobems. There is no expicit signaing required. partia dua decomposition: the master dua probem is soved with the standard price update for each cass in (44) which is carried out independenty by each ink and then, for a given set of prices, each source soves its own subprobem as in (5) and subprobem (43) is independenty soved by each ink. This approach contains ony one eve of decomposition and no expicit signaing is required. Observe that in the prima-dua decomposition approach each ink updates the aggregate rates on a sower timescae and the prices on a faster timescae, whereas in the partia dua decomposition approach each ink updates the prices on a sower timescae and the aggregate rates on a faster timescae (actuay in one shot); therefore, the speed of convergence of the partia dua approach shoud be faster in genera. In both cases, the users are privy of the existence of casses and ony the inks have to take this into account by having one price for each cass. In other words, this is a way to give each cass of users a different price than the one based on the standard dua-based agorithm so that they can be further controed. The next appication hinges on this observation. V. APPLICATION 3: HYBRID RATE-BASED AND PRICING-BASED RATE ALLOCATION A. Probem Formuation One extreme way to contro the rate aocation process is to directy give each source the rate they can use, at the expense of a centraized computation. At the other extreme, we can optimize the system in a fuy distributed way via pricing, as in the basic NUM of Subsection II-E, at the expense of trusting the sources even though they can be noncooperative and try to obtain more bandwidth by using a more aggressive utiity function. Neither of these two extreme approaches is competey satisfactory in a appications, and hybrid soutions between rate-based and window-based rate aocation are desirabe for both robustness of fair aocation against aggressive users and speed of converging to the correct rate aocation equiibrium. New congestion contro protocos using direct rate aocation have recenty been proposed, such as RCP [5] that is based on a heuristic computation of the processor-sharing type of rate aocation by each router that a fow traverses. We now describe a systematic method to perform distributed and direct rate aocation to each user. The key idea is to use the approach of Section IV but with one cass for each user. The probem formuation becomes x,{y (s) } 0 s U s (x s ) x s y (s) s, L (s) s y(s) c c (s) min y(s) c (s) max. (45) Note that if a source s does not use a path, then y (s) is taken as zero in the constraint s y(s) c. B. Prima Decomposition If we now take a prima decomposition approach, then the master prima probem wi be in charge of the update of y (s) and each user wi simpy choose x s equa to the minimum of the y (s) aong its path in order to satisfy x s y (s) L (s). This approach constitutes in fact one of the extreme methods in which each user is directy given the amount of bandwidth it can use. C. Partia Dua Decomposition We may aso take a dua decomposition approach by reaxing the fow constraints [ ( ) ] x,{y (s) s U s (x s ) L(s) λ(s) x s } 0 + s L(s) λ(s) y (s) (46) s y(s) c c (s) min y(s) c (s) max s. This probem decomposes into one maximization for each source, as (5) in the basic NUM, with λ s = L(s) λ(s) being the aggregate path price specific for user s, pus the foowing additiona rate-bounding maximization to obtain the y (s), for each ink : {y (s) } 0 s: L(s) λ(s) y (s) s: L(s) y(s) c (s),min y(s) c c (s),max s : L (s). (47) This probem can be soved independenty by each ink as a way to distribute its capacity c among the sources using the 9
ink according to the weights given by the prices λ (s), which are different for each source. The master dua probem corresponding to this dua decomposition is soved with the foowing subgradient price update step (simiary to (7)): [ ( )] + λ (s) (t +1)= λ (s) (t) α y (s) (t) x s (λs (t)), s : L (s). (48) D. Summary We have expored different decompositions for the hybrid rate/pricing-based rate aocation in (45): prima decomposition: it eads to a direct rate aocation and is based on one eve of decomposition. This approach requires the signaing to inform each user what rate to transmit at. partia dua decomposition: the master dua probem is soved with the price update in (48) which is carried out independenty by each ink and then, for a given set of prices, each source soves its own subprobem as in (5) and the bounding rates of subprobem (47) are aso obtained independenty by each ink. This approach ony shows one eve of decomposition and does not require any expicit signaing. It is a hybrid of rate-bounding and pricing-feedback mechanisms. VI. APPLICATION 4: MULTIPATH-ROUTING RATE ALLOCATION A. Probem Formuation Consider now a more genera setup of the basic NUM of Subsection II-E where each source can choose among severa possibe paths (possiby using a weighted combination of them). The structure of a network with S sources, L inks, and J paths can be summarized with the L J path avaiabiity 0 1 matrix H defined by { 1 if the jth path uses the th ink [H],j = 0 otherwise together with the J S path choice nonnegative matrix W 5 defined by { wjs if the sth source uses the jth path [W] j,s = 0 otherwise where w js indicates the percentage of the rate of the sth user aocated to the jth path and has to satisfy w js > 0 and j w js =1. These two matrices can be combined into the routing matrix R = HW that tes how much each source is using each ink. 5 This notation foows that in [6]. However, the probem being considered here is to design rate aocation agorithm with a fixed H and W, whereas the probem considered in [6] is to anayze the effect of joint routing and rate aocation with W being a variabe. To start with, the probem can be directy formuated with the routing matrix R ike the basic NUM in (3): s U s (x s ) x 0 (49) Rx c and then the standard dua-based decomposition agorithm can be used. We wi ater see that it may be more fexibe to formuate the probem aternativey in terms of H and W as foows: x,y 0 s U s (x s ) Wx y Hy c (path constraint) (ink constraint) (50) where y contains the aggregate rate aong the th path. B. Prima-Dua Decomposition We can now consider a prima decomposition approach of (50) by fixing the path rates y. Probem (50) becomes then a basic NUM where y pays the roe of the ink capacities in (3). This probem can be soved via the standard dua-based agorithm as reviewed in Subsection II-E. The master prima probem is U (y) y 0 (51) Hy c where U (y) is the optima objective vaue of (50) for a given y, with subgradient given by the Lagrange mutipier λ associated to the constraints Wx y in (50). As usua, the master prima probem (51) can be soved with a subgradient method by updating the path rates as y (t +1)=[y (t)+αλ (y (t))] Y (5) where [ ] Y denotes the projection onto the feasibe convex set Y {y : y 0, Hy c}. In principe, this subgradient update cannot be performed independenty by each path due to the projection onto Y, which makes it impractica. C. Partia Dua Decomposition We can aso take a partia dua decomposition of (50) by reaxing ony the constraint Wx y (simiary to [8]): x,y 0 s U s (x s )+γ T (y Wx) (53) Hy c. This probem decomposes into one maximization for the sources as in (5) for the basic NUM: x 0 s [U s (x s ) γ s x s ], (54) where γ s = γ T W :,s = j J(s) γ jw js is the aggregate price for the sth source, pus one maximization for the path rates: γ T y y 0 (55) Hy c which has to be soved in a centraized way. The master dua probem updates the prices as γ (t +1)=[γ (t) α (y Wx (γ (t)))] +. (56) 10
D. Fu Dua Decomposition Yet another different way to sove probem (50) is with a fu dua decomposition by reaxing both constraints Wx y and Hy c: x,y 0 s U s (x s )+γ T (y Wx)+λ T (c Hy) (57) which can be rewritten as x,y 0 s [U s (x s ) x s γ s ]+ j y j ( γj λ j) + λ T c (58) where λ j = λ T H :,j = L(j) λ is the aggregate price of the jth path and γ s = γ T W :,s = j J(s) γ jw js is the aggregate price for the sth source. This probem separates into a maximization over x, as in (5) for the basic NUM, and a maximization over y, which is unbounded uness γ j = λ j. Therefore, the optima choice for the master dua probem is γ j = λ j and then γ s = j J(s) λj w js = j J(s) w js L(j) λ. Hence, this approach reduces to the standard dua-based agorithm appied to probem (49). Now consider a variant of this rate aocation probem with mutipath-routing, where the objective of Internet Service Provider (ISP) is combined with the end user utiity objective. In today s operating environment of the Internet, the ISP controing each Autonomous System tries to minimize a tota convex cost function of the ink utiizations [7]. Suppose the cost function is quadratic, and the network utiity maximization is now formuated as maximizing the weighted difference between end user utiity and ISP cost: s U s (x s ) θy T y (59) where θ is the weight. Observe that by taking θ sufficienty sma the quadratic term becomes negigibe and we are back to the origina probem (50). Repeating the same fu reaxation as before, one gets the foowing maximization probem: x,y 0 s [U s (x s ) x s γ s ] + j y ( j γj λ j) θy T y + λ T c. (60) This probem separates as before into a maximization over x, as in (5) for the basic NUM, and a maximization over y with optima soution given by y j = 1 ( γj λ j) j. (61) θ Then, the master dua probem has to update two sets of prices: λ (t +1)=[λ (t) α (c Hy (t))] + (6) γ (t +1)=[γ (t) α (y (t) Wx (γ (t)))] + (63) where y (t) y (λ (t), γ (t)) is the optima y for the given λ (t) and γ (t) as in (61). Note that the matrix-vector products can be convenienty written as [Hy] = j: L(j) y j and [Wx] j = s:j J(s) w jsx s. 18 16 14 1 10 8 6 4 0 Evoution of λ 4 for a methods Method 1 (subgradient) Method (Gauss Seide for a ambdas and gamma) Method 3 (Gauss Seide for each ambda and gamma sequentiay) Method 4 (subgradient for gamma and exact for a inner ambdas) Method 5 (subgradient for a ambdas and exact for inner gamma) Method 6 (Gauss Seide for a ambdas and exact for inner gamma) Method 7 (Jacobi for a ambdas and exact for inner gamma) 5 10 15 0 5 30 35 40 45 50 iteration Fig. 3. Evoution of λ 4 for the seven methods based on a dua decomposition. E. Summary We have expored severa possibiities for distributed agorithms for rate aocation with mutipath-routing possibiities in (50): standard dua decomposition: by reformuating the probem as in (49) we recover the basic NUM formuation and the standard dua-based agorithm can be readiy used. prima-dua decomposition: the master prima probem (51) is soved with the path rate subgradient update in (5) and then, for a given set of path rates, the resuting basic NUMs is soved via the standard duabased decomposition in (5) and (7). Unfortunatey, due to the projection in (5) a centraized computation is required, which makes this approach impractica. partia dua decomposition: the master dua probem is soved with the price update in (56) and then, for a given set of prices, each source soves its own subprobem as in (54) and subprobem (55) is soved in a centraized way, making this approach aso inconvenient. fu dua decomposition: the master dua probem is soved with the price updates in (6)-(63) and then, for a given set of prices, each source soves its own subprobem as in (54) and the path rates are obtained as in (61). This approach contains one eve of decomposition: on the higher eve the master dua probem and on the ower eve the source-rate and path-rate subprobems. Expicit signaing is required for the update of the price γ (t) in (63), and for the computation of the path rate in (61) (which can be done either at the receiver of the path or through heuristic-based computation distributed across routers aong the path). 11
Cass 1 Source 1 Source Cass 3 5 Destination rate 8 7 6 5 4 Evoution of source rates with the prima based agorithm with QoS Tota Cass 1 Cass Source 1 Source Source 3 Source 4 Source 3 3 3 Source 4 1 Fig. 4. Bock diagram of the considered exampe of NUM with priorities. 0 0 50 100 150 00 50 iteration VII. NUMERICAL EXAMPLES A. Downink Power/Rate Contro The purpose of this subsection is to iustrate the convergence behavior of different decomposition approaches can be quite different, using the downink power/rate contro formuated in (37) as the context. Fig. 3 shows the evoution of one of the dua variabes (λ, γ) under the iterations for seven methods based on various combinations of prima/dua mutieve decompositions and variants of Gauss-Seide and Jacobi iterations of subgradient cacuations. Without going into the detais due to space imit, the seven methods are respectivey based on the foowing decompositions: fu dua + subgradient for λ and γ, fu dua + Gauss-Seide for λ and γ, fu dua + Gauss-Seide for each λ i and γ, dua-prima (for λ and γ), dua-prima (for γ and λ), dua-prima + Gauss- Seide, dua-prima + Jacobi. B. QoS Rate Aocation To iustrate the distributed agorithms for a rate aocation among QoS casses (as in Section IV), we consider a simpe exampe consisting of four sources transmitting to the same destination and sharing a common ink as shown in Fig. 4. Users in cass 1 are very aggressive, with utiity functions U 1 (x) =1og(x) and U (x) =10og(x), whereas users in cass are not aggressive, with utiity functions U 3 (x) = og(x) and U 4 (x) =og(x). If no QoS contro is incuded in the design and the standard dua-based distributed agorithm of Subsection II-E is used, then the aggressive users of cass 1 get most of the avaiabe capacity in the common ink. In particuar, cass 1 gets a rate of 4.5 out of the tota avaiabe rate of 5, eaving cass ony with a rate of 0.5. This is precisey the kind of unfair behavior that can be avoided with QoS contro. Figs. 5 and 6 show the evoution of the rates of the sources when QoS contro is incuded in the distributed agorithms based on a prima decomposition and on a dua decomposition, respectivey (as described in Section IV). In particuar, the rate for each cass has been imited to 3. As can be observed, the rate of cass 1 now tends to the imit of 3 and, since the ink capacity is 5, cass is eft with a rate of (as opposed to 0.5 obtained without QoS contro). Hence, the distribution of the tota rate between both casses is more fair. Both prima-based Fig. 5. Evoution of the rates with the prima-based agorithm for a NUM with QoS contro. rate 8 7 6 5 4 3 1 Evoution of source rates with the dua based agorithm with QoS Tota Cass 1 Cass Source 1 Source Source 3 Source 4 0 0 50 100 150 00 50 iteration Fig. 6. Evoution of the rates with the dua-based agorithm for a NUM with QoS contro. and dua-based agorithms show a simiar convergence (a constant stepsize of 0.05 was used for a subgradient updates). Note that the prima-based agorithm contains two eves of subgradient updates and, in principe, the inner subgradient agorithm shoud run unti convergence before updating the outer subgradient. In practice, however, this is not necessary and both subgradients can run simutaneousy (in genera using a smaer stepsize for the outer subgradient so that it works on a sower timescae). C. Mutipath-Routing Rate Aocation We now consider a NUM with different grouping of the path and ink constraints as described in Section VI. In particuar, we generate a random network topoogy with S =4sources, J =1paths, and L =36inks, such that each user uses 3 paths and each path uses 5 inks. Fig. 7 shows the evoution of the rates of the sources for the standard dua-based agorithm based directy on the routing matrix R = HW. Fig. 8 shows the evoution of the rates of the sources with a fu dua-based agorithm (incuding the quadratic term θy T y with θ =0.001), which foows cosey the performance of the standard agorithm. In practice, the optima soution for the path rates in (61) eads to a arge dynamic range that can ead to instabiity; this can be 1