The Price of Selfishness in Network Coding Jason R. Marden, Member, IEEE, and Michelle Effros, Fellow, IEEE

Transcription

1 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL The Price of Selfishness in Network Coding Jason R. Marden, Member, IEEE, and Michelle Effros, Fellow, IEEE Abstract A game-theoretic framework is introduced for studying selfish user behavior in shared wireless networks. The investigation treats an -unicast problem in a wireless network that employs a restricted form of network coding called reverse carpooling. Unicast sessions independently choose routes through the network. The cost of a set of unicast routes is the number of wireless transmissions required to establish those connections using those routes. Game theory is employed as a tool for analyzing the impact of cost sharing mechanisms on the global system performance when each unicast independently and selfishly chooses its route to minimize its individual cost. The investigation focuses on the performance of stable solutions, where a stable solution is one in which no single unicast can improve its individual cost by changing its route. The results include bounds on the best- and worst-case stable solutions compared to the best performance that could be found and implemented using a centralized controller. The optimal cost sharing protocol is derived and the worst-case solution is bounded. That worst-case stable performance cannot be improved using cost-sharing protocols that are independent of the network structure. Index Terms Distributed control, game theory, network coding, price of anarchy (PoA), reverse carpooling. I. INTRODUCTION T HE network coding literature treats the design and performance of network codes aimed at goals, such as maximizing capacity, minimizing power consumption, or improving the robustness of communication in a network environment. While early results primarily treat the multicast problem where a single source transmits the same information to all sinks in the network more recent work changes the focus to problems where multiple independent communication sessions share the network environment. A multiple unicast problem, characterized by a list of source-sink pairs with an independent information flow to be established from each source to its corresponding sink, is one example of such a coding scenario. Multisession network coding problems, such as the multiple unicast problem, differ from the single-session network coding problems in that they establish Manuscript received June 29, 2009; revised February 17, 2011; accepted October 06, Date of current version March 13, This work was supported by the Social and Information Sciences Laboratory at the California Institute of Technology, DARPA ITMANET Grant W911NF , and the Lee Center for Advanced Networking at the California Institute of Technology. The conference version of this paper appeared in [19] and [20]. J. R. Marden is with the Department of Electrical, Computer, and Energy Engineering, University of Colorado, Boulder, CO USA ( jason. marden@colorado.edu). M. Effros is with the Department of Electrical Engineering, California Institute of Technology, Pasadena, CA USA ( effros@caltech.edu). Communicated by M. C. Gastpar, Associate Editor for Shannon Theory. Color versions of one or more of the figures in this paper are available online at Digital Object Identifier /TIT competition between independent information flows for shared network resources. While multicast problems are reasonably well understood, far less is known about the multiple unicast problem. A spectrum of approaches ranging from centralized optimization and control to totally distributed design and operation is possible for tackling multisession coding problems. In centralized approaches, such as [31], the operations at all nodes of the network are designed by a code designer with access to complete information about the network and all sessions competing for network resources. At the other end of the spectrum are distributed approaches. such as [15], which tackle large optimization problems by treating either nodes or sessions as independent decision makers in a shared network environment. Distributed algorithms achieve savings in computation and coordination by dividing a large, multiagent optimization problem into smaller subproblems. The cost of this savings is that the solution obtained may be suboptimal since it is not, in general, possible to divide the problem into subproblems for which finding an optimal solution to each subproblem guarantees an optimal solution to the complete problem. This problem is exacerbated by the desire to create subproblems that can be solved with the incomplete information available to the individual sessions in the shared network environment. This paper seeks to quantify the degree to which dividing the network optimization problem into subproblems, solved by selfish network participants (in this case, unicast sessions), degrades system-wide performance. This work is in line with a variety of other recent papers that use game-theoretic methods to analyze network coding problems treating either individual unicasts or individual nodes in the network as selfish decision makers [6], [10], [16], [23], [26], [28]. Most of these results are only applicable in restricted settings. For example, the authors of [6] focus primarily on a single-source multicast with network coding. In that setting, the authors derive a cost mechanism (i.e., a procedure for distributing the cost of a particular edge to the players using that edge) so that a Nash equilibrium exists and the flow allocation at a Nash equilibrium corresponds to the minimum cost. An alternative example is [26], where the authors focus on a generalization of the butterfly network with two users. The authors propose cost functions for the two users and show that the desirable capacity-achieving solution emerges as a dominant strategy equilibrium point of the game. We focus on a class of network coding problems consisting of multiple self-interested unicasts in a wireless network that employs reverse carpooling. We then derive performance bounds for networks of arbitrary sizes and topologies. To make the analysis tractable, we use the number of transmissions in the wireless network as the measure of network performance. This performance measure, a common proxy for power consumption /$ IEEE

2 2350 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL 2012 in wireless networks, simplifies analysis by removing concerns about interference that would arise in networks with synchronized transmissions or delay constraints. The goals of this paper are: 1) to quantify the degradation in performance due to the selfishness of participating agents and 2) to identify tractable and implementable mechanisms for influencing agent behavior in this noncooperative setting. We therefore formulate the network coding problem as a cost-sharing problem. A system designer establishes a formula by which participants in this case individual unicasts are charged for their network use. Setting such a charging mechanism establishes the optimization functions for the individual unicasts. We focus on the design of such cost-sharing protocols to ensure that the network behavior is desirable for all possible network sizes and topologies and all possible collections of unicasts. To ensure that the resulting analysis is tractable and implementable, we require that the cost-sharing protocol be designed without knowledge of the network or the unicasts that seek to traverse that network. To understand the impact of protocol design on the network performance, we derive mathematical bounds on the impact of the protocol choice. Protocol design has been studied extensively in a variety of application domains, including network formation [1], [2], [9]; network routing [27]; and bandwith allocation [14]. Prior results are application specific; neither the approaches nor the performance metrics readily extend to new problems, such as the network coding problem. Our work studies protocol design for network coding over networks of arbitrary size and topology. The discussion that follows introduces a family of protocols for which we demonstrate that Nash equilibria are guaranteed to exist. We compare the performance of competing protocols by bounding the efficiency of pure Nash equilibria for all possible network sizes and topologies and any number and distribution of unicasts through the network. We measure efficiency using two well-known measures of worst case performance: price of anarchy (PoA) and price of stability (PoS). Each takes the form of a bound on the ratio between the system cost associated with a pure Nash equilibrium and the optimal system cost [27]. The ratio of the worst Nash equilibrium to the optimal system cost is called the PoA; the ratio of the best Nash equilibrium to the optimal system cost is called the PoS. Both are greater than or equal to 1 since the cost associated with a Nash equilibrium is always greater than or equal to the optimal system cost. A PoS close to 1 ensures that there exists a pure Nash equilibrium with performance close to the optimal network behavior. A PoA close to 1 ensures that the performance associated with every pure Nash equilibrium is close to the optimal network behavior. Central to our results is a derivation of the feasible region for the efficiency of all possible cost-sharing protocols. The resulting region, which appears in Fig. 1, demonstrates a fundamental tradeoff between the PoA and PoS. We derive the set of optimal cost-sharing protocols that establish all points on the lower boundary of this region and show that no alternative design could achieve better performance Our results can also be interpreted from the standpoint of distributed optimization. While game theory is typically employed in problems characterized by multiple agents with competing interests, and distributed optimization is typically Fig. 1. Optimal efficiency bounds for cost-sharing protocols in multiple unicast wireless networks employing reverse carpooling network codes. used in problems characterized by a single agent with access to a collection of computational resources, the tools of game theory are also applicable in the latter case. That is, one mechanism for finding a distributed solution is to break the problem into subproblems and then employ noncooperative optimization on those subproblems. These techniques are particularly useful in problems with environmental uncertainties and communication limitations between the computational devices [3], [17]. Taking this perspective, the unicasts are not selfish agents but computers tasked with finding solutions to network coding problems of sizes exceeding those that can be handled by a single device. Here, the design of cost-sharing protocols establishes a framework through which the parallel computations interact. This approach allows individual devices to make decisions using only local information. Once this interaction framework is established, we can employ a variety of well-established learning algorithms that provide convergence to various forms of equilibria [18], [22], [34] [36]. Many of these algorithms provide convergence to pure Nash equilibria; employing any such learning algorithm yields a distributed, local solution to our network coding problem that achieves efficiency no worse than that predicted by the PoA. For potential games, distributed learning algorithms exist that converge to optimal equilibria (i.e., equilibria that minimize the potential function in a potential game see the definitions below) [4], [7], [8], [21]. These algorithms provide us with a distributed solution to the network coding problem with performance no worse than that predicted by our PoS bound; unfortunately, the convergence rates might be quite poor. The remainder of this paper is organized as follows. In Section II, we give a brief overview of the game-theoretic concepts used in this paper. In Section III, we describe a simple wireless network coding technique called reverse carpooling and consider its application to the multiple unicast problem. In Section IV, we formulate the reverse carpooling problem as a cost-sharing problem. We establish our protocol design space and identify two natural cost-sharing protocols which yield undesirable results. In Section V, we focus on the design of cost protocols with improved performance and in Section VI, we prove the that these cost protocols are optimal. In Section VII, we highlight the implications of our results on alternative network coding problems. Section VIII contains some concluding remarks.

3 MARDEN AND EFFROS: THE PRICE OF SELFISHNESS IN NETWORK CODING 2351 II. BACKGROUND: NONCOOPERATIVE GAMES We consider finite strategic-form games with players denoted by the set. Each player has a finite action set and a cost function where denotes the joint action set. We refer to a finite strategic-form game as a game, and we sometimes use a single symbol (e.g., ) to represent the entire game (i.e., the player set, action sets, and cost functions ). For an action profile, let denote the profile of player actions other than player, i.e., With this notation, we sometimes write a profile of actions as. Similarly, we may write as. We also use to denote the set of possible collective actions of all players other than player. We focus on analyzing equilibrium behavior in distributed systems. The most well-known form of an equilibrium is the Nash equilibrium. Definition 2.1 (Pure Nash Equilibrium): An action profile is called a pure Nash equilibrium if for each player A (pure) Nash equilibrium represents a scenario for which no player has an incentive to unilaterally deviate. We will henceforth refer to a pure Nash equilibrium as purely an equilibrium. One class of games that plays a prominent role in the results contained in this paper is that of potential games. In a potential game, the change in a player s cost that results from a unilateral change in strategy is equal to the change in a global cost function. Definition 2.2 makes this idea precise. Definition 2.2 (Potential Games): A game is called an (exact) potential game if a global function exists so that for every player, for every, and for every (2) The global function is called the potential function for game. In potential games, any action profile minimizing the potential function is an equilibrium; hence, every potential game possesses at least one such equilibrium. Example 2.1 (A 2 Player Potential Game): Consider the following example of a two-player game where each player has an action set and a cost function as expressed in the following cost matrix: Player 1 Player Cost Matrix Player 1 Player Potential Function (1) Fig. 2. Illustration of reverse carpooling. The cost matrix designates the cost each player pays for all action profiles. The first entry in each location represents player 1 s cost while the second entry represents player 2 s cost. For example, and. There is a unique equilibrium. The aforementioned game is a potential game with the potential function. We gauge the efficiency of equilibria using the well-known worst case measures called the PoA and PoS [27]. Let denote a set of games with an invariant global performance metric. For any particular game, let denote the set of equilibria, and and denote the PoA and PoS for the game, respectively, where and is any optimal action profile for the game. We define the price of anarchy and the price of stability for the set of games as III. SIMPLE WIRELESS NETWORK CODING PROBLEM We consider the distributed design of network codes for multiple unicasts in a shared wireless network. We restrict our attention to the simplest form of network codes, where any node relaying one message in each direction between a pair of neighboring nodes can reduce the power required for transmission by broadcasting the bit-wise binary sum of the received messages in a single transmission rather than transmitting the two messages sequentially. Each neighbor can then determine its intended message by adding the information that it sent to the received sum, as illustrated in Fig. 2. This type of coding is sometimes called reverse carpooling since it allows two flows to share a single transmission provided that the two flows traverse the node in opposite directions. The goal of our network code design for this wireless network is to minimize the power required to simultaneously satisfy a given collection of unicast flow demands. For simplicity, we measure the cost of a network coding solution by evaluating the number of transmissions per packet required under steady-state flow conditions. The following notation helps make these ideas concrete. We describe a network by a set of vertices, or nodes, and for each, the neighbors of node are denoted as ; each transmission by node is heard by all of the nodes in and only those nodes. Suppose the network needs to be shared by a finite set of players. Each player represents a single unicast from source to receiver, where. A path (3) (4) (5) (6)

4 2352 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL 2012 from source to terminal is equal to a set of nodes capable of transmitting the information (i.e.,, where denotes the number of nodes in,,, and for all ). We use to denote all paths from to available to player and to denote the paths available to all players. For analysis, it is convenient to label each transmission from by the node from which the information was obtained and the node for which it is next intended so that we can recognize coding opportunities. To that end, let the detailed path of be defined as where represents a transmission by node to node of information that was received from node. The transmission represents a transmission by node to node of information that originated at node. Notice that for any path, there is a unique detailed path consisting of elements corresponding to the transmissions required to send a packet along that path. A node that participates in more than one path may have the opportunity to combine messages and save on transmission costs if the paths traverse the node in opposite directions. Suppose players and traverse node in opposite directions (i.e., ), where player sends message using the transmission and player sends message using the transmission. If and, then node can transmit and both and will be able to decode their intended messages. This allows node to serve players and with one transmission instead of two. This is true since node knows and receives, allowing it to decode its intended message. We assume that each vertex has a cost that measures the number of transmissions by that vertex necessary for any routing profile. For our problem, this cost takes on a simplified form that depends only on each player s transmission through the vertex. Before defining the structure of the cost function for the reverse carpooling setting, we first introduce some notation. For a given routing profile, let be defined as the number of players sending information from to through, i.e., Using only reverse carpooling codes, the transmission cost at node for profile is defined as where and for all. Hence, the power consumption at a vertex depends only on the number of players using each transmission through the vertex. The system cost for a routing profile is (7) (8) (9) that mini- A global planner would like to use a profile mizes the system cost. IV. REVERSE CARPOOLING GAME In this section, we analyze the reverse carpooling problem described in Section III in a game-theoretic framework where the multiple unicasts are modeled as self-interested decision makers. We formulate this game, which we refer to as the reverse carpooling game, as a cost-sharing problem where a system designer specifies a cost protocol, or pricing scheme, for each node in the network. We establish a viable design space for our cost-sharing protocols. We review some traditional cost-sharing models and analyze the existence and efficiency of the resulting equilibria. A. Our Model We consider a game-theoretic model for the reverse carpooling problem where the multiple unicasts are modeled as self-interested decision makers seeking to utilize a common network with nodes and neighbor sets. Each unicast is endowed with a set of actions where each action represents a path from the source to the destination of the specific unicast. Here, we focus on the design of cost functions for the unicasts which results in desirable global behavior. We focus on the design of cost functions that meet two fundamental requirements, locality and scalability, which are of fundamental importance in order to make our cost design tractable and implementable in networks of arbitrary size and topology. Locality: Our first requirement is that the cost function of a unicast depends only on local information. That is, the cost for any unicast for a path should only depend on: 1) the nodes ; 2) the other unicasts that use nodes ; and 3) how the other unicasts used node (i.e., the directed transmissions through node ). Since the system cost expressed in (9) represents a summation of costs over directional transmissions, we focus on the design of cost functions for the unicasts of the following form: for any routing profile unicast is where, the cost of any (10) (11) defines the cost-sharing protocol for the directed transmission and defines the set of unicasts using the directed transmission in the profile. Here, defines the cost that unicast is charged for using the directed transmission in the routing profile. Note that defining a cost-sharing protocol for each directed transmission in the network results in a well-defined game irrespective of the topology of the network or the structure of the

5 MARDEN AND EFFROS: THE PRICE OF SELFISHNESS IN NETWORK CODING 2353 action sets ; hence, it gives rise to a degree of scalability in the design of cost functions. 1 Scalability: Our second requirement is that cost-sharing protocols should be defined without relying on specific information regarding the topology of the network. That is, for any two directed transmissions and, player sets,, and player (12) Accordingly, we represent the set of cost-sharing protocols by a single cost-sharing protocol and drop the explicit dependence on the directed transmission. We define a reverse carpooling game by the tuple as a single choice of, which uniquely defines the set of cost functions for any set of action profiles (i.e., for any routing profile ), the cost of any unicast is (13) We omit explicitly mentioning the topology of the network since the relevant information is directly embedded in the set of available actions for each unicast. We focus on analyzing the existence and efficiency of equilibria over the set of reverse carpooling games using a fixed cost-sharing protocol obtained by extending the game to all possible combinations of action sets (i.e., ). It is important to highlight that the cost-sharing protocol is designed with the goal of influencing the emerging global behavior and need not reflect true system costs. That means we do not enforce the constraint that for any player set, Fig. 3. Nonexistence of equilibria using equal share cost protocol. given directed transmission. That is, for any number of player sets,, and player, we have (14) The following example demonstrates that while the equal share cost design in (14) is natural, it does not guarantee the existence of an equilibrium for any network. Example 4.1: Define the cost function for each unicast in (13) using the equal cost share protocol in (14). Consider the network illustrated in Fig. 3. This network contains three different paths: the top path contains six internal nodes, the middle path contains ten internal nodes, and the bottom path contains seven internal nodes. There are three users seeking to utilize this network: Player 1 has source and terminal and has available paths. Player 2 has source and terminal and has available paths. Player 3 also has source and terminal but only has one available path. Since player 3 only has a single action, this situation represents a game between players 1 and 2 with the following cost matrix: The approach taken in this paper is similar to tolling methods in the traffic network where tolls are introduced with the goal of influencing the emerging global behavior to improve overall congestion [30]. B. Examples of Cost-Sharing Protocols The goal of this paper is to establish a single cost-sharing protocol so that the game that emerges between the unicasts when using the cost function defined in (13) has desirable properties, including the existence and efficiency of equilibria irrespective of the network topology, the number of unicasts, or the available paths to each unicast. To highlight the fundamental challenges of this objective, we review two natural cost-sharing protocols and illustrate their respective limitations. 1) Equal Share-Cost Protocol: The most natural costsharing protocol is to equally share the transmission cost on a 1 Note that a more general representation of the cost-sharing protocol would be needed for an alternative coding scheme such as star coding. We omit explicitly defining this more general representation for a more compact presentation. Cost Matrix Here, the entry 1 is each box corresponding to the player s initial transmission from its source where coding is not a possibility. As you can see, this example does not possess an equilibrium. 2) Marginal Contribution Cost Protocol: An alternative approach for designing cost functions is to assign each player a cost function that reflects the player s marginal contribution to the true system cost as proposed in [33]. That is, for any player sets,, and player, wehave (15) Using the marginal contribution cost-sharing protocol in (15), a player is assigned a cost of 1 for a directed transmission if the number of players using the transmission in his or her direction is strictly greater than the number of players using the transmission in the opposite direction (i.e., ). Otherwise, the player is assigned a cost of 0. When using the

6 2354 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL 2012 cost-sharing protocol in (15), the cost of a unicast takes on the form (16) where is defined as the total number of directed transmissions in player s path, where the number of players using the transmission in his or her direction is strictly greater than the number of players using the transmission in the opposite direction, i.e., where is the indicator function. The following theorem demonstrates that the marginal contribution cost protocol in (15) guarantees the existence of an equilibrium in any reverse carpooling game regardless of the network topology, the number of unicasts, or their respective paths. However, the worst case equilibria are not necessarily efficient. Theorem 4.1: Let be the set of all reverse carpooling games. The marginal cost-sharing protocol as in (15) guarantees the existence of an equilibrium for any game in. The price of stability and price of anarchy are Proof: We start by proving the existence of an equilibrium for any game in. We prove this by showing that the marginal contribution cost protocol ensures that the resulting game is a potential game. Define as the null action of player which represents the situation where player does not send any information through the network. The cost function for player using the marginal contribution cost-sharing protocol in (15) can be expressed as meaning that each player s cost represents the player s marginal contribution to the true system cost. For any player, actions,, and the action profile for all other player Therefore This implies that the game is a potential game with potential function. Hence, an equilibrium is guaranteed to exist. The resulting potential game also establishes our price of stability result. Since the resulting game is a potential game with potential function, for any game, any action profile that minimizes the system cost is an equilibrium; hence,. Fig. 4. Unbounded PoA by using the marginal contribution cost protocol. To show that the price of anarchy is unbounded, consider the following example of a reverse carpooling game with two 2 players as shown in Fig. 4. Let and. The neighborhood of each node is designated by its connected edges. Here, and, where,,, and. The corresponding player and system costs will be represented in matrix form. For simplicity, we refer to and as the top paths and and as the bottom paths Cost Matrix System Cost There are two equilibria of this game: 1) each player selects the bottom path, which yields a system cost of 2 (neither player has the opportunity to reverse carpool) and 2) each player selects the top path, which yields a system cost of since each player reverse carpools on the interior vertices. Therefore, the PoA for this game is. Since is arbitrary, this completes the proof. V. DESIGNING COST PROTOCOLS TO IMPROVE THE POA Both the equal share and marginal contribution cost-sharing protocols have their respective limitations as the previous section demonstrated. In this section, we provide new cost-sharing protocols that always guarantee the existence of an equilibrium while improving the efficiency of the equilibrium behavior over the set of reverse carpooling games. We consider a class of cost-sharing protocols parameterized by of the following form: (17) The cost-sharing protocol in (17) introduces a spectrum of costsharing protocols that range from a fixed cost protocol to the marginal contribution protocol. That is, when this protocol assigns a fixed cost of 1 to each user irrespective of the other decisions of the other unicasts. When, this protocol effectively approaches the marginal contribution protocol in (15). Our motivation for this design stems from the intuition gained in the network in Fig. 4 which illustrates that assigning just marginal costs can result in highly inefficient equilibria as users are not necessarily penalized for long routes. It is important to point out that this protocol design, as with the marginal contribution protocol, does not require any form of global information regarding the network structure or the network demand.

7 MARDEN AND EFFROS: THE PRICE OF SELFISHNESS IN NETWORK CODING 2355 When using the cost-sharing protocol in (17), the cost function of each player takes on the form Let be any equilibria. For any player s cost satisfies be any optimal assignment and and any equilibria, each (21) An alternative representation of the cost in (18) is (18) (19) where and are defined as the total number of directed transmissions in player s path, where the number of players using the transmission in his or her direction is strictly less than or equal to the number of players using the transmission in the opposite direction, respectively, i.e., This cost representation in (19) will be useful in the forthcoming proofs. Theorem 5.1: Let be the set of all reverse carpooling games. For any, the cost-sharing protocol in (17) guarantees the existence of an equilibrium for any game in. Proof: As in the proof of Theorem 4.1, it is straightforward to show that for any, the cost-sharing protocol in (17) guarantees that the resulting game is a potential game with potential function where represents the cost player would incur if the player selected his or her shortest path and for each directed transmission,. Trivially, the total system cost of an equilibrium is no less than the system cost of an optimal assignment, that is, The system cost at the optimal allocation satisfies (22) This bound represents the system cost resulting from an assignment where each player selects his or her shortest path (i.e., a path consisting of the minimal number of nodes, and reverse carpools at each transmission in his or her path. Note that achieving this cost in general is not possible using reverse carpooling since a player can never carpool at his or her originating vertex. However, in the ensuing analysis, we ignore these edge effects, thereby deriving loose bounds that are asymptotically tight as the size of the network grows. Our upper bound on the system cost associated with an equilibrium is highly dependent on. Case 1: : Since,wehave (20) Hence, an equilibrium is always guaranteed to exist. We now provide bounds on the PoA and PoS for this design. We prove the optimality of this cost design in Section VI. Theorem 5.2: Let be the set of all reverse carpooling games. For any, the cost-sharing protocol in (17) guarantees that the PoA is Therefore, the system cost of an equilibrium is bounded above by Combining (22) with (23), we obtain (23). Proof: We divide up the true system cost to help with bounding the PoA. Let player s portion of the true system cost be This implies that for any, the PoA satisfies Then, for any action profile, To show that this bound is tight, consider a slight variation to the example in Fig. 4, modified so that the top path contains

8 2356 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL 2012 Fig. 5. Worst case example: PoA for 2. intermediate nodes, while the bottom path contains intermediate nodes. The example is illustrated in Fig. 5. The cost matrix and system cost are listed below. Notice that there are again two equilibria, namely, the top and bottom paths. The system cost of the equilibrium, when each player selects the top route, is. The optimal system cost is. Therefore, the PoA of this example is, which can be made arbitrarily close to Case 2:,wehave Cost Matrix System Cost : Since Therefore, the system cost of an equilibrium is bounded above by Combining (22) with (24), we obtain (24) Fig. 6. Worst case example: PoA for 2 [1; 2]. This implies that for any, the PoA satisfies. To see that this bound is tight, consider the example in Fig. 6, where each edge segment consists of internal nodes which are not explicitly highlighted in the figure. Eight players are located at eight exterior nodes. Each player s destination is the exterior node two hops clockwise from the player s source. Suppose each player has two available paths to his or her destination: 1) traverse two segments clockwise on the exterior or 2) traverse two segments through the interior. If each player traverses the exterior path, the system cost is and each player s cost is since there are no carpooling opportunities for any player. If a player unilaterally switches to his or her interior path, his or her cost is still since he or she still has no carpooling opportunities. Therefore, this is an equilibrium. If each player travels through the interior, this is also an equilibrium. Each player s cost is since the player carpools on the interior nodes but not the boundary nodes of each edge segment. The system cost of this scenario is. Therefore, this example exhibits a PoA of which can be made arbitrarily close to 2 for any. Theorem 5.2 is a negative result in the sense that it demonstrates that the worst case equilibrium can achieve a cost two times the cost of the optimal solution when (and even higher when ). This implies that the worst-case equilibrium solution may be no better than the solution where each player takes his or her shortest path and no player carpools. Our next result is more positive, showing, in particular, when that the best equilibrium has a cost, at most, 50% higher than the cost of the optimal solution. Recall that the PoA and PoS are worst-case measures and do not reflect average performance as many networks will perform significantly better than this worst-case bound. Rather, these worst-case measures provide lower bounds on the average performance. Theorem 5.3: Let be the set of all reverse carpooling games. For any, the cost-sharing protocol in (17) guarantees that the PoS is Proof: According to Theorem 5.1, we have that any reverse carpooling with cost functions as a potential game with potential function. The existence

9 MARDEN AND EFFROS: THE PRICE OF SELFISHNESS IN NETWORK CODING 2357 Fig. 7. Worst case example: PoS for 0. of this potential function implies that any action profile that minimizes the potential function is also an equilibrium, that is, any action profile constitutes an equilibrium. We next use this fact to bound the price of stability. Let be an equilibrium that minimizes the potential function and be an optimal action profile. Then,. Also, since minimizes Fig. 8. PoA and PoS for the reverse carpooling game as a function of. (25) Expanding out the potential function, we obtain which is simplified as Fig. 9. PoA of 2 for scalable cost functions. (26) For any allocation, the term can be bounded above and below by Therefore, we can bound (26) as which gives the following bound on the price of stability: To see that this bound is tight, consider the two-player example in Fig. 7. Player 1 has source and terminal and can select either the top path consisting of internal nodes or the bottom path consisting of internal nodes, giving. Player 2, indicated by the solid arrow, is fixed on the top path segment giving. The cost of the top segment for player 1 is. The cost of the bottom segment for player 1 is. Therefore, player 1 taking the bottom path is the unique equilibrium. This equilibrium yields a system cost of. The optimal system cost, which occurs when player 1 selects the top path, is. Therefore, the PoS for this example is, which can be made arbitrarily close to. Fig. 8 shows the tradeoff between the PoA and PoS for this cost design. Notice that as, we effectively recapture our original design (i.e., each player s cost function is dominated by the marginal contribution to the true system cost ). In this case, we obtain a price of stability of 1 and an unbounded PoA. Using this design, we can improve the PoA at the expense of some degradation in the PoS. In particular, we can reduce of PoA from to 2 by allowing the PoS to increase from 1 to 1.5. The ensuing section focuses on the optimality of this cost design, showing that it is impossible to achieve a more desirable PoA and PoS with alternative cost-sharing protocols. VI. OPTIMALITY OF DESIGN In this section, we discuss the optimality of the cost protocols in (17). Our first result focuses on the PoA. Since the cost associated with each player choosing his or her shortest path and applying no network coding is, at most, twice the cost of the optimal solution, the PoA described in Theorem 5.2 is high. It turns out that any set of cost-sharing protocols that are local and scalable have at least a PoA of 2. Theorem 6.1: Let be the set of all reverse carpooling games. If the set of cost-sharing protocols are local and scalable, then the PoA is at least 2. Proof: Consider the example in Fig. 9, which is a slight variant of the example in Fig. 6. There are eight players with the same source and terminals as indicated in Fig. 6. Each player

10 2358 IEEE TRANSACTIONS ON INFORMATION THEORY, VOL. 58, NO. 4, APRIL 2012 only has the option of taking an exterior path or an interior path, each consisting of two segments consisting of internal nodes. The interior and exterior paths for player 1 are illustrated. The main difference between this example and the example in Fig. 6 is that the exterior paths of different players are now disjoint. If each player s cost function is scalable as in (12) and if all players take their exterior paths as shown on the right side of the figure, then we have an equilibrium. At this equilibrium, each player is alone on a path consisting of two segments. If a player unilaterally switched to his or her interior path from this equilibrium, the player would also be alone on a path consisting of two segments; hence it would result in the same cost since cost functions are scalable. If each player travels on the exterior, the system cost is since there are no carpooling opportunities for any player. If each player travels through the interior, each player carpools on the interior nodes but not on the boundary nodes of each edge segment. The system cost of this scenario is. Therefore, this example exhibits a PoA of, which can be made arbitrarily close to 2. Theorem 6.1 proves that if the set of cost protocols is designed without knowledge of the network structure (i.e., scalable), then it is impossible to guarantee a PoA strictly less than 2. It is an open research question to understand how to incorporate attainable information regarding network structure into players cost functions to improve the PoA. Our second result characterizes a hard constraint on the relationship between the PoA and PoS for all local cost functions. Theorem 6.2: Let be the set of all reverse carpooling games. Fix any local cost-sharing protocol. If the PoA is, then the PoS. Proof: Consider the example in Fig. 5. In this example, Player 1 has source and terminal and can select either the top path which consists of internal nodes or the bottom path which consists of internal nodes, which we denote as and, respectively. Player 2 has source and terminal and can also select either the top or bottom path. The cost functions and system cost for the four joint action profiles are highlighted in the following payoff matrices: Cost Function System Cost Cost functions and achieve by assumption. Consider the action profile. Since the PoA is, we know that is not an equilibrium. If were an equilibrium, then the PoA for this game would be, which yields a contradiction. Therefore, we know that either player 1 or 2 must prefer the bottom route over the top route for this situation. Without loss of generalities, suppose. Consider the situation where player 2 only has the option of selecting the top path, that is, as in Fig. 7. Note that this restriction does not change the cost functions for player 1. For this game, the cost functions and system cost are now Cost Matrix System Cost This game has a unique equilibrium, namely. The optimal allocation for this game is. Therefore, the PoS for this game is, which can be made arbitrarily close to. Theorems 6.1 and 6.2 prove that the cost-sharing protocol in (17) achieves the optimal relationship between the PoA and PoS as summarized in the following corollary. Corollary 6.1: Let be the set of all reverse carpooling games. The cost-sharing protocol in (17) achieves the optimal relationship between the PoA and PoS over all local and scalable cost protocols. Proof: The cost-sharing protocol in (17) can guarantee a PoA and PoS pair for any. According to Theorem 6.1, we know that it is impossible to guarantee a PoA strictly less than 2 for any local and scalable cost protocols. By Theorem 6.2, we know that for any set of local cost protocols that yield a PoA of, the PoS must be at least. Therefore, the cost functions in (18) achieve the optimal bounds on the relationship between the PoA and PoS. Comparing the efficiency of the stable solutions of the game induced by the cost protocols in (17) with the algorithm shortest path routing with opportunistic coding highlights the advantages of game theory as a mechanism for attaining distributed solutions to network coding problems. The shortest path routing with opportunistic coding entails each unicast selecting the shortest path connecting its source and destination. Given this allocation, all available coding opportunities are utilized [15]. The solution to this algorithm is equivalent to an equilibrium of a game-theoretic design with the cost-sharing protocol in (17) when. Note that this cost design results in independent cost functions (i.e., a player s cost function is independent of the actions of all other players). Since this is a special case of the cost-sharing protocol in (17), Theorems 5.2 and 5.3 provide the following corollary. Corollary 6.2: Shortest path routing with opportunistic coding yields a PoA of 2 and a PoS of 2. Game theory provides the analytical framework necessary to analyze distributed systems consisting of more general interdependent cost functions (i.e., a player s cost function may depend on the actions of all other players). In terms of the reverse

11 MARDEN AND EFFROS: THE PRICE OF SELFISHNESS IN NETWORK CODING 2359 carpooling problem, the analysis presented before demonstrates that it is possible to design interdependent cost functions that entice players to create coding opportunities rather than just utilize available coding opportunities as is the case in opportunistic coding. Fig. 1 highlights the advantage of transitioning from independent to interdependent cost functions; namely, it is possible to improve the efficiency of the best case equilibria without hurting the efficiency of the worst case equilibria. For example, the cost protocols in (17) with can improve the PoS from 2 to 1.5 while maintaining a PoA of 2. Unfortunately, transitioning from independent to interdependent cost functions cannot improve the performance of the worst case equilibria. VII. IMPLICATIONS FOR ALTERNATIVE NETWORK CODING PROBLEMS The approach to network coding considered in this paper is not restricted to network coding problems with unicast users and reverse carpooling coding opportunities. Our approach can also be applied to other problems by characterizing the decision makers, their possible choices (or some subset thereof), and a desired system cost that depends on the collective behavior of all players. The goal is to build upon this characterization to establish cost-sharing protocols for each node that effectively influences group behavior. Note that the marginal contribution of the cost-sharing protocol will guarantee the existence of an equilibrium and a price of stability of 1 irrespective of the given global objective or available coding schemes. For example, consider the distributed design of network codes for multiple multicasts in a shared wireless network where only reverse carpooling coding is allowed. In this setting, each player is now associated with a single source and a set of terminals. An action for player consists of a set of vertices (i.e., ), so that for any terminal, a path exists so that,,, and for all. In this setting, a directed transmission would take on a slightly different form from the unicast scenario as a single transmission from a node could have multiple destinations. Without explicitly stating the form, let represent the total number of transmissions given the choice and let represent the system cost. If each player is assigned a cost function as in (18), then the results of Theorems 5.2 and 5.3 hold. Notice that we did not rely on the fact that the users were unicast sessions anywhere in the derivation of the PoA or PoS. While characterizing the complete action set might be challenging for more general demand structures, the derived bounds on the PoA and PoS compare the system cost associated with any equilibrium to the system cost of the optimal characterized action. Therefore, the results apply even when does not cover all possible action profiles. The general results on optimality of cost functions in Section VI also provide some impossibility results for more general network coding problems. Consider any network coding problem, unicast or multicast, with various coding options so that reverse carpooling is a special case of the allowable coding structures. Then, the worst case examples used to prove Theorems 6.1 and 6.2 are still applicable in this new domain. Therefore, for any network coding problem where reverse carpooling is a special case, it is not possible to construct local and scalable player cost functions so that the PoA is strictly less than 2. Notice, however, that in these scenarios, the PoA of 2 is not necessarily achievable by the shortest paths. Furthermore, it is not possible to guarantee a better relationship between the PoA and PoS than the relationship set forth in Theorem 6.2, though this bound is not necessarily achievable for those generalizations. VIII. CONCLUDING REMARKS The main goal of this paper is to illustrate the role of costsharing protocols in influencing selfish behavior. The studied cost-sharing protocols are both local and scalable and thereby define cost functions for the players irrespective of the given network topology. Alternative coding schemes and objectives can be handled in a similar fashion where cost-sharing protocols need to be designed to ensure the existence and efficiency of the resulting equilibria. While the cost-sharing protocols discussed in this paper may not be optimal for alternative classes of problems, they do provide a starting ground as they guarantee the existence of an equilibrium irrespective of the network topology. Finally, extending this analysis to more realistic wireless communication models with interference constraints is an important future research direction. A second goal of this paper is to illustrate the potential of game theory as a tool for cooperative control of distributed systems [3], [17]. Cooperative control focuses on the design of autonomous agents to optimize a given global objective. Utilizing game theory for cooperative control, or distributed optimization, requires the following items: 1) Game design: The system designer must specify the set of decision makers, which we refer to as agents or players, and their respective actions. Each agent is assigned a local objective function that he or she selfishly seeks to optimize. An agent s objective function may depend solely on his or her action, or more generally on the actions of all agents. 2) Agent decision rules: The system designer must specify an iterative procedure for how each agent selects his or her respective action in response to observed information. The goal in this setting is to design both the game and the agent decision rules so that the emerging global behavior is desirable with respect to the global objective. The theory of learning in games includes several agent decision rules, also referred to as distributed learning algorithms, that provide guarantees on the emerging global behavior. We direct the readers to [11], [13], [29], [32], [35], and [36] for a comprehensive review. The results in this paper could be interpreted along the lines of game design for this prescriptive agenda. Here, we designed the unicasts as self-interested decision makers and the cost-sharing protocols were used to design local cost functions for the individual unicasts. Once the game is appropriately defined, we can refer to off-the-shelf distributed learning algorithms that provide convergence either to different classes of equilibria. There is a rich body of literature on distributed learning algorithms that converge to equilibria in potential games [18], [22], [34] [36]; hence, these algorithms would provide us with a distributed solution to the network coding problem with an efficiency bounds in accordance with the PoA. There is also a body of literature on