Robust Connectivity-Aware Energy-Efficient Routing for Wireless Sensor Networks

Transcription

1 3904 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 Robust Connectivity-Aware Energy-Efficient Routing for Wireless Sensor Networks Charles Pandana and K. J. Ray Liu Abstract In this paper, we consider a class of energy-aware routing algorithm that explicitly takes into account the connectivity of the remaining sensor network. In typical sensor network deployments, some nodes may be more important than other nodes because the failure of these nodes causes the network disintegration, which results in early termination of information delivery. To mitigate this problem, we propose a class of routing algorithms called keep-connect algorithms, that use computable measures of network connectivity in determining how to route packets. The proposed algorithms embed the importance of the nodes in the routing cost/metric. The importance of a node is characterized by the algebraic connectivity of the remaining graph when that node fails. We prove several properties of the proposed routing algorithm including the energy consumption upper bound. Using extensive simulations, we demonstrate that the proposed algorithm achieves significant performance improvement compared to the existing routing algorithms. More importantly, we show that our proposed algorithm is more robust in terms of algebraic network connectivity compared to the existing algorithm. Finally, we present the distributed implementation of our proposed algorithm. Index Terms Sensor network, graph theory, communication system reliability, communication system routing. I. INTRODUCTION ADVANCES in low-power integrated circuit devices and communications technologies have enabled the deployment of low-cost, low-power sensors that can be integrated to form a sensor network. This network has vast important applications, i.e., from battlefield surveillance systems to modern highway and industry monitoring systems; from the emergency rescue systems to early forest fire detection and the very sophisticated earthquake early detection systems, etc. Having the broad range of applications, the sensor network is becoming an integral part of human lives. Moreover, it has been identified as one of the most important technologies nowadays [], [2]. There are many important characteristics of a sensor network. First, the sensor nodes are typically deployed in an area with high redundancy and each of the sensor node has limited energy, therefore it is prone to failure. In order to accomplish the mission, it is essential for the sensor nodes to collaborate. Second, the nodes in the sensor network typically stay in their original deployed places for their entire lifetime. Hence, it is very important to always keep the remaining network Manuscript received April 30, 2007; revised October 2, 2007 and January 5, 2008; accepted March 7, The associate editor coordinating the review of this paper and approving it for publication was P. Fan. C. Pandana is with Arraycomm, San Jose, CA 953 ( cpandana@gmail.com). K. J. Ray Liu is with the Department of Electrical and Computer Engineering, and the Institute of Systems Research, University of Maryland, College Park, MD ( kjrliu@umd.edu). Digital Object Identifier 0.09/T-WC /08$25.00 c 2008 IEEE connected, since the disintegrated clusters of nodes are useless for information gathering. Moreover, due to the immobility of the nodes, it may not be possible to reorganize the remaining nodes to create a new connected network. Due to these characteristics of sensor networks, the design of routing algorithms becomes very different from the typical ad-hoc networks in the following aspect. Instead of minimizing the hop count and delivery delay in the network, the routing algorithms in the sensor networks focus more on extending the scarce battery lifetime of the nodes. Furthermore, most of the existing energy-aware routing algorithms use the time until the first node in the network dies as the definition of network lifetime [3], [4]. Since in many practical sensor applications, the death of the first node may not influence the information collection task; We argue that the definition of the network lifetime should be defined as the time until there is no route from any source to any destination. In other words, the network lifetime should be defined as the time until the network becomes disconnected/disintegrated. Using this definition as the network lifetime, the network connectivity becomes an important criterion to be explicitly considered in the routing algorithm. None of the existing routing algorithm has ever explicitly considered the network connectivity in performing routing task. To be precise, we employ the notion of algebraic connectivity of a graph in the spectral graph theory [5] to quantify the importance of a node. In particular, the importance of a node is quantified by the Fiedler value [6], [7] of the remaining graph when that particular node fails. One property of the Fiedler value is that the larger it is, the more connected the graph will be. Due to this reason, the Fiedler value is also referred to as algebraic connectivity of a graph [6]. By considering the nodes importance from the graph connectivity perspective in the routing design, the node with higher importance will be retained in the network, therefore the connectivity of the remaining network is maintained as long as possible. By embedding the nodes importance in the routing cost, we propose a class of algorithms called keep-connect algorithms to solve the posed problem. Our proposed algorithm has several advantages, namely it is online algorithm, which implies that the algorithm does not require to know ahead of time the sequences of messages to be routed. This characteristic is important, since the information generation typically is not known aprioriin sensor network. The proposed algorithm is efficient in maintaining the connectivity of the remaining network, we demonstrate the effectiveness of our proposed algorithm using extensive simulations. Finally, the proposed algorithm is flexible and can be used along with any other existing energy-aware routing algorithms that employ distributed

2 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 3905 Bellman-Ford/Dijkstra algorithms in their implementations. This paper is organized as follows. We first give the system description and problem formulation in Section II. Several important facts from spectral graph theory are briefly outlined in Section III. In Section IV, our proposed algorithm is explained. Upper bound on the energy consumption of our proposed algorithm is proved in Section V. In Section VI, we present one possible distributed implementation of our proposed algorithm. The effectiveness of our method is presented in Section VII. Finally, conclusions are drawn in Section VIII. II. SYSTEM MODEL AND PROBLEM FORMULATION In this section, we present the network model, review several definitions for sensor network lifetime, and explain the problem formulation. A. Network Model A wireless sensor network can be modeled as an undirected simple finite graph G(V,E), wherev = {v,,v n } is the set of nodes in the network, E is the set of all links/edges, V = n is the number of vertices in the graph, and E = m is the number of edges in the graph. The undirected graph implies that all the links in the network are bidirectional, i.e. node v i is able to reach node v j implies the vice versa. The simple graph implies that there is no self-loop in each node and there are no multiple edges connecting two nodes. And the finite graph implies the cardinality of the nodes and edges is finite. The link (v i,v j ) implies that node v j S vi can be directly reached by node v i with a certain transmit power level in the pre-defined dynamic range, where S vi is the set of nodes that can be directly reached by node v i. We assume that every node has the initial battery energy of E i for i {,..., n}. The energy consumption for packet transmission from node v i to v j is proportional to d(i, j) α, where d(i, j) is the distance between node v i and v j. The path loss exponent α, depends on the transmission environment [8] and typically ranges from 2 to 4. In this paper, we assume α =2for free space propagation. We will also discuss the performance of our algorithm when α =4.0 in Section VII. When the energy in one node is exhausted, we say that the node has failed. B. Definitions of Network Lifetime Depending on the application in the wireless sensor network, there are many definitions of the network lifetime. In [3], [4], the network lifetime is defined as the time until the first node/sensor in the network fails. In contrast, in [9], the network lifetime is defined as the time until all nodes fail. A more general definition on the network lifetime is given in [0]. In [0], they defined the lifetime of sensor networks as the min{t,t 2,t 3 },wheret is the time it takes for the cardinality of the largest connected component to drop below c n(t), wheren(t) is the number of alive nodes at time t, t 2 is the time it takes for n(t) to drop below c 2 n(0), and t 3 is the time it takes for the area covered to drop below c 3 A, wherea is the area covered by the initial deployment of the sensors. In the above definition, c, c 2,andc 3 are the pre-defined constants between zero and one. It is well-known that the network connectivity is a very important characteristic in ad-hoc/sensor networks, therefore it should be taken into account in the network lifetime definition and algorithm design. In sensor network applications, the time until the first node/sensor fails may not serve as a good definition of the network lifetime, since the failure of the first node/sensor does not cease the information delivery/collection. In contrast, network disintegration typically causes severe impact in the information delivery. This motivates us to employ the time until the remaining network becomes disconnected as our network lifetime definition. Using this definition, we arguethat it is crucial to consider the network connectivity in designing an energy-aware routing algorithm. C. Problem Formulation The problem of maximizing the minimum residual energy of nodes in the network has been studied in [3], [], [2]. The time until the first node in the network dies can be found using the following linear program Maximize T s.t. )f (c) (i, j) 0, i N, j S i, c C, 2) j S i e t(i, j) c C f (c) (i, j)+ j:i S j e r(j, i) c C f (c) (j, i) E i, i N, 3) j:i S j f (c) (j, i)+tq (c) i = j S i f (c) (i, j), i N, c C, where f (c) (i, j) is the amount of information of commodity c that is transmitted from node v i to its neighbor node v j S i until time T, S i is the set of v i s neighboring nodes. Each commodity c C has certain source node O (c) and destination node D (c),whereo (c) is the origin/source of commodity c and D (c) being the destination/drain of commodity c. e(i, j) is the energy required to guarantee successful transmission denotes the informationgeneration rates at node v i of commodity c.thefirst constraint indicates that the number of packets transmitted between any two nodes is nonnegative. The second constraint implies that the total energy used for packet transmission and reception at one particular node should be less that the battery capacity in that node. And this applies for all nodes. The last constraint indicates the flow conservation at each nodes in the network, from node v i to node v j,andq (c) i we note that Q (c) i > 0 for v i O (c), Q (c) i < 0 for v i D (c), and Q (c) i =0, otherwise. The notation j : i S j denotes the summation of the flow from all nodes v j whose neighbor is node v i, therefore, j:i S j f (c) (j, i) denotes the total amount of information going into (inflow to) node v i until time T. Similarly, j S i f (c) (i, j) indicates the total amount of information going from (outflow from) node v i until time T. The flow conservation simply states that the total flow coming into node v i plus any information rate generated from node v i is equal to the total flow going out from node v i. We note that the above formulation has two assumptions; first, the formulation requires the knowledge of all commodities when performing the optimization. This implies that the information generation rate on sources in all commodities ()

3 3906 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 during the whole duration of network lifetime is required to solve the linear program. Second, the above formulation does not reflect the sequences of the commodities. We know that the performance of online routing algorithm depends on the sequences of the commodities []. By sequences of commodities, we mean the sequences in which the traffic from different commodities appears in the network. These two assumptions imply that the above formulation is only suitable for off-line optimization, which has a limited use in practical scenario. In practice, the routing decision has to be made online; the routing decision is done upon the packet arrival. And it will be very hard, if not impossible, to know the information generation rate of all commodities a priori. The qualitative performance comparison of online and offline algorithm for routing algorithm is given in []. They show that there is no online algorithm for message routing that has a constant competitive ratio in terms of network lifetime, where the competitive ratio is defined as the ratio of the solution to online algorithm with respect to the optimal off-line solution. This implies that the performance of online algorithm is worse compared to the off-line algorithm. Moreover, the formulation in () is only suitable for the time until the first node dies. For the above reasons, we focus on designing a robust online algorithm by taking into account the connectivity of the remaining network in making the routing decision. The robustness of our proposed scheme comes from the fact that when the information generation is not known a priori and the routing decision is made on the fly, employing the connectivity weight in the routing decision keeps the remaining network connected as long as possible. III. FACTS FROM SPECTRAL GRAPH THEORY Before we describe our proposed solution, we briefly summarize some important facts from spectral graph theory [5] [7], [3]. We will only state the lemmas since they provide insight for understanding the proposed scheme. And we will use some of these lemmas to prove several properties of the proposed scheme. The complete proofs of the lemmas can be found in the above literatures. A. Eigenvalues of Laplacian Matrix In this subsection, we briefly discuss the definition of a Laplacian matrix, its eigenvalues and the relationship between the eigenvalues of Laplacian matrix and the connectivity of the associated graph. The following notations will be used throughout the paper: G(V,E) is the graph with set of vertices V and set of edges E. We recall the number of vertices as V = n and the number of edges as E = m. Moreover, we define G vi as a graph resulted from removing vertex v i and all its adjacent edges from the original graph G. Intherest of the paper, we will use vertex and node, interchangeably. Similarly, we will use link and edge, interchangeably. The Laplacian matrix associated with a graph is defined as follow. Definition (Laplacian matrix associated with a graph): In a graph G(V,E), the Laplacian matrix associated with a graph, L(G) is an n by n matrix defined as follows: d vi if v i = v j, L(i, j) = if (v i,v j ) E, (2) 0 otherwise where i, j {,,n} are the indices of the nodes. Equivalently, the Laplacian matrix L(G) can be expressed as: L(G) =T (G) A(G), (3) where T (G) is an n by n diagonal matrix associated with graph G with the (i,i)-th entry having value d vi, which represents the number of the neighboring nodes. A(G) is a n by n adjacent matrix associated with graph G. The eigenvalues of the Laplacian matrix, L(G), (λ 0 (G) λ (G) λ n (G)) are usually referred to as the graph spectra. The following lemma describes the relationship between the graph spectra and the connectivity of a graph [5]. Lemma : Let s denote 0 = λ 0 (G) λ (G) λ n (G) as the eigenvalues of the Laplacian matrix L(G). If G is connected, then λ (G) > 0. Moreover,ifλ i (G) =0 and λ i+ (G) 0,thenG has exactly i+ disjoint connected components. It is easy to observe that the Laplacian matrix L(G) has all zeros row sums, therefore, matrix L(G) has an eigenvalue 0 with the corresponding eigenvector (,, ) T. Moreover, since L(G) is real, symmetric and nonnegative semi-definite, thus all the eigenvalues of L(G) should be real and nonnegative. It is obvious that the smallest eigenvalue of Laplacian matrix L(G) is zero. The above lemma indicates that if G is strongly connected (there exists a simple path from any initial node v i to the terminal node v j,wherei j) then the Laplacian matrix L has simple eigenvalue 0 (the eigenvalue 0 has the multiplicity of ). Moreover, if the eigenvalue 0 of the Laplacian matrix L(G) has multiplicity n, then there are n disconnected components. Since we always work with the connected graph, we will focus on the second smallest eigenvalue of the Laplacian matrix, L(G) for the rest of this paper. B. Fiedler value and vector Let s denote the eigenvalues of the Laplacian matrix, L(G) associated with G(V,E) as λ 0 (G),,λ n (G) and the corresponding eigenvectors as ν 0 (G),, ν n (G). Obviously, ν 0 (G) =e =(,, ) T. Suppose that the graph G(V,E) is strongly connected (the second smallest eigenvalue is strictly larger than zero, λ (G) > 0). This second smallest eigenvalue can be represented as (Courant-Fisher Theorem [4]) λ (G) = min x T L(G)x (4) x T x=,x T ν 0(G)=0 The second smallest eigenvalue of the Laplacian matrix is always referred to as the algebraic connectivity of the graph G [6]. It is also called as Fiedler value of a graph. The reason for calling the second smallest eigenvalue as the algebraic connectivity of a graph G comes from the following lemmas [6]. Lemma 2: If G and G 2 are edge-disjoint graphs with the same vertices, then λ (G )+λ (G 2 ) λ (G G 2 ). Lemma 3: The Fiedler value λ (G) is non-decreasing for graphs with the same set of vertices, i.e. λ (G ) λ (G), if G (V,E ), G(V,E), ande E. We observe that G and G have the same number of vertices. Since G has fewer edges compared to G and E E, this implies that G is less connected compared to G.

4 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 3907 From Lemma 3, we have that the Fiedler value corresponding to G is smaller than G, λ (G ) λ (G). Itisinthissense that the Fiedler value represents the degree of connectivity in a graph. Finally, the relation of Fiedler value for graph obtained from removing a vertex and all its adjacent edges is given by the following lemma [6]. Lemma 4: Let G vi be a graph obtained from removing vertex v i from G and all the adjacent edges. Then λ (G vi ) λ (G). The following two lemmas give some upper and lower bounds for the Fiedler value. These lemmas will be used to prove some properties of the proposed algorithm in Section V. Lemma 5: Let G(V,E), d vi be the degree of node v i and V = n be the number of vertices, then [ n λ (G) n ] min v i d vi. (5) Lemma 6: Let ε(g) be the edge connectivity of the graph G (the minimal number of edges whose removal would result in losing connectivity of the graph G). Then, we have [ ( )] π λ (G) 2ε(G) cos, (6) n where n is the number of vertices V = n. IV. KEEP-CONNECT ALGORITHMS In this section, we use the facts of spectral graph theory described in the previous section to develop online routing algorithms to maximize the network lifetime, which is defined as the time until the network becomes disconnected. Before describing the routing algorithms, let s first consider how to measure the degree of connectivity of the remaining graph in the routing metric. We propose to quantify the connectivity of the remaining graph based on the Fiedler value. Recall from Section III-B, the Fiedler value qualitatively represents the connectivity of a graph in the sense that the larger the Fiedler value is, the more connected the graph will be. The degree of connectivity of the remaining graph can be quantified by the Fiedler value of the graph resulted from removing that particular node and all the edges connected to that node from the original graph. We design the connectivity weight of each node as /λ (G vi ),whereg vi denotes a graph resulted from removing node v i and all its adjacent edges from the original graph G. In this way, the node that causes severe reduction in the remaining algebraic network connectivity will be avoided when performing the routing decision. This is due to the fact that λ (G vi ) measures the degree of connectivity of the graph after removing node v i and all the edges connected to that node. We note that if v i is an articulation node [5], (i.e. node after its removal results in disconnected network), then λ (G vi ) theoretically equals to zero. This will cause numerical problem when we embed the weight in the routing algorithm. To avoid this problem, we introduce a small threshold, ɛ. Ifλ (G vi ) <= ɛ, weset λ (G vi )=ɛ. The detail of the algorithm is given in Table I. In short, the proposed algorithm avoids using the articulation nodes to keep the remaining network connected. We set the threshold as ɛ =0 5 throughout this paper. TABLE I KEEP-CONNECT USING FIEDLER VALUE Let G(V, E) be the original graph. Let s define the graph obtained after removing node v i and all its adjacent edges as G vi ({V v i },E vi ). Let also denote ɛ as small threshold.. Initialization: Set nodes weights as zeros W (v i )=0, v i V 2. For each node v i : a. Form the Laplacian matrix L(G vi ) of graph as (2). b. Find the Fiedler value λ (G vi ) as (4). c. Set the weight of node as: W (v i )=/λ (G vi ),ifλ (G vi ) >ɛ. Else, set W (v i )=/ɛ. End for We note that in order to maintain the remaining algebraic connectivity of a network as long as possible in the routing algorithm, route metric should reflect the effect of selecting one particular route on the remaining (algebraic) connectivity of the network. In other words, the severity effect of the selection of one particular route on the connectivity measure of a network should be quantified. Since the route metric is the sum of the links costs that constitute the route [3], [4], [6], the link cost should also reflect the reduction in the remaining connectivity of the network. In the following subsections, we present proper modification on the existing routing algorithms by incorporating the connectivity weight into the routing metrics. A. MMKC, MHKC and MMREKC routing algorithms In this subsection, we first review the idea of existing routing algorithms and proceed with the modification to include the keep-connect algorithm. The minimum hop routing is usually used in ad-hoc or wire-line network to minimize the delay in the packet delivery. The routing algorithm chooses the route with the minimum hop (MH) [6]. The link cost between two nodes is. To modify this algorithm to reflect the remaining connectivity in thenetwork,weembedtheconnectivity weight in the link cost between two nodes, so that it becomes c(u, v) = 2 [W (u)y +W (w) y ]. We call this algorithm minimum hop while keeping the connectivity, (MHKC(y)). Both minimum hop and MHKC routing algorithms can be computed using standard Dijkstra algorithm. The max-min residual energy (MMRE) algorithm [4] selects the route that maximizes the minimum residual energy of nodes in the route. The link cost for MMRE is represented as c(u, w) =min{e u (t), E w (t)}, (7) where E u (t) and E w (t) represent the residual energy at time t for transmitting node u and receiving node w, respectively. Another variant of MMRE is to maximize the minimum link cost that is represented as c(u, w) =E u (t)+e w (t). (8) The max-min route can be implemented using modified Dijkstra algorithm [4]. Following the same way as before, we modify the variant of MMRC by embedding the connectivity into the link cost as c(u, w) =E u (t)w (u) y + E w (t)w (w) y, (9)

5 3908 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 TABLE II LINK COSTS FOR SEVERAL ROUTING ALGORITHMS WITH/OUT CONNECTIVITY CRITERION Algorithm / Method Link cost MH / Standard Dijkstra c(u, w) = MHKC(y) / Standard Dijkstra MMRE / Modified Dijkstra c(u, w) = 2 [W (u)y + W (w) y ] c(u, w) =min{e u (t), E w (t)} Variant of MMRE / c(u, w) =E u (t)+e w (t) Modified Dijkstra MMREKC(y) / c(u, w) = Modified Dijkstra E u (t)w (u) y + E w (t)w (w) y MMKC(y) / Modified Dijkstra c(u, w) =W (u) y + W (w) y where W (u) and W (w) are the connectivity weights for node u and v, respectively. We name this algorithm as MM- REKC(y). This algorithm selects the routing decision based on the remaining energy in nodes and the connectivity criterion. We note that maximizing the minimum residual energy of nodes is not the same as avoiding articulation nodes. Using MMREKC(y), one hopes to balance the choices of avoiding minimum residual energy of nodes and avoiding articulation nodes. If we set the residual energy of nodes in MMRECK(y) as one, we will get the max-min remaining connectivity (MMKC(y)) algorithm. We note that the MMKC is similar to MHKC except that the MMKC uses the modified dijkstra algorithm and MHKC uses the standard dijkstra algorithm. MMKC algorithm only avoids overusing some articulation nodes. Precisely, the routing algorithms that use connectivity criterion avoid using nodes that result in severe reduction in the remaining connectivity after they failed. In all the keep-connect algorithms, y determines how important the connectivity weight should affect the routing cost. Since in MMREKC(y) and MHKC(y) the routing metrics try to simultaneously two different objectives (e.g., both minimum hop and connectivity in the MHKC algorithm), this variable y controls the importance of connectivity criterion in the routing metric. The link cost for the exiting algorithms and the proposed modifications are summarized in Table II. The connectivity weight, W ( ), in the algorithms is calculated using the keep-connect algorithm (Table I). During the routing computation, if there are more than one nodes having weights of /ɛ, the route metric will be determined by the link cost of the rest of nodes in the route. In the case where there is a tie in route metric, any tie breaker rule is sufficient. B. Minimum total energy while keeping connectivity (MTEKC) routing In previous subsection, the routing metrics are determined either by the number of hops (minimum hop routing/mh) or minimum residual energy of nodes (MMRE), with/without the connectivity criterion. The minimum total energy (MTE) algorithm minimizes the total energy used for packet transmission and reception along the route [4]. By embedding the connectivity weights of nodes to the MTE algorithm, we obtain MTEKC(y). In particular, the link cost of MTEKC(y) is represented by c(u, w) =e t (u, w) W (u) y + e r (u, w) W (w) y, (0) where e t (u, w) and e r (u, w) are the transmit and receive energy for delivering a packet from node u to w. The TABLE III MTEKC(Y). For any source-destination pairs, find the minimum total energy path with edge cost as: e t(v i,v j )W (v i ) y + e r(v i,v j )W (v j ) y for v i V, v j S vi,wheree t(v i,v j ) and e r(v i,v j ) are the transmit and receive energy for delivering a packet from node v i to v j. S vi denotes the neighbors of node v i. W (v i ) is the weight of node v i. 2. If node dies, recompute the alive nodes weight using Keep-Connect algorithm. Redo step. MTEKC(y) minimizes the total transmit energy while trying to keep the remaining network as connected as possible. The complete algorithm for MTEKC is shown in Table III. One of the purposes of introducing the variable y is to limit the energy consumption of the MTEKC. In principle, the algorithm will never achieve both the best energy efficient and the most robust connectivity route. Introducing y provides a way to trade-off between the two objective functions. This is particularly true when the network is large as we will prove the bound of the energy consumption in the MTEKC algorithm in Section V. This bound can be easily controlled by the variable y. Similar to the previous keep-connect algorithms, in the case when there are more than one nodes having weights of /ɛ, the rest of the nodes in these different routes will determine which overall routing metrics are smaller. In the tie situation, any tie breaker rule is sufficient. It is important to emphasize the MTE algorithm is less effective compared to MMRE type algorithms only when one considers the first node dies as the network lifetime definition. In our case, since we employ the time until the network becomes disconnected as the network lifetime, the above argument is not necessary true in general. In fact this phenomena can be observed in [4]. In [4], after more than one node dies, the expiration time (network lifetime) for the MTE is higher than MMRE. We will further discuss this phenomenon in the Section VII. In our simulation, we typically observe that more than one node fails before the network becomes disconnected (cf. Figure 4). V. UPPER BOUND ON THE ENERGY CONSUMPTION OF MTEKC(Y) ALGORITHM In this section, we continue to show the upper bound on the energy consumption of the MTEKC(y) algorithm. It is well-known from [] that there is a tradeoff between the energy consumption in the route and avoiding overuses of popular nodes in achieving the maximum lifetime/total delivered packets before the network becomes disconnected. Similarly, there is a trade-off between the energy efficiency and the connectivity robustness. Hence, it is important to show the bound on the energy consumption of our proposed algorithm. In this section, we first derive some properties of our proposed algorithm and we employ these properties to prove the upper bound of the energy consumption. For simplicity we only include the transmit energy between two nodes in calculating the energy of the route. We denote r as the minimum total energy route connecting any fixed source node v 0 and destination node v d. Equivalently, the MTE route

6 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 3909 is represented as r =arg min d(r) r R(v 0,v d ), () where R(v 0,v d ) is the set of all routes connecting source node v 0 and destination node v d, d(r) is the number of hops in the route. Furthermore, we denote r as the MTEKC(y) route obtained using Fiedler value and this route satisfies r =arg min r R(v 0,v d ) d(r) λ (G vi ) y. (2) In the following, we give some simple lemmas on the lower bound and upper bound of the metric used in the keep-connect algorithm. Lemma 7: [Lower bound of MTEKC(y) metric] For each route, the MTEKC(y) employing the Fiedler value metric has the following property d ( ) y d e t(v i,v i+)w (v i) y n 2 e t(v i,v i+), n min i d vi (G) i=0 i=0 (3) ( ) y d (n 2)n e t(v i,v i+), (4) 2(n )m where d vi is the degree of node v i in the graph, n is the number of vertices in the graph, and m is the number of edges in the graph. Proof: To prove these inequalities, we require the upper bound of the Fiedler value as follows (stated in Lemma 5). Consider G(V,E) and let d vi (G) be the degree of node v i in graph G. Then, 0 <λ (G) n n min d vi (G). (5) i Now, consider the graph G vi obtained from graph G by removing node v i and all edges connecting to node v i. Obviously, λ (G vi ) n n 2 min d vi (G vi ) n i n 2 min d vi (G), i (6) since the minimum degree of graph G vi is smaller or equal to minimum degree of graph G. Now, using keep-connect algorithm with Fiedler value, we have d i=0 i=0 e t(v i,v i+ ) W (v i ) y = d i=0 ( n 2 (n ) min i d vi (G) e t(v i,v i+) λ (G vi ) y ) y d i=0 e t(v i,v i+ ). (7) Since n min i d vi i d v i =2m, wehave ( ) y ( ) y n. (8) min i d vi (G) 2m Then, we obtain the second inequality ( ) y d n 2 (n ) min i d vi (G) i=0 ( ) y d (n 2)n. (9) 2(n )m i=0 Lemma 8: [Upper bound of MTEKC(y) metric] For each route, the MTEKC(y) employing the Fiedler value metric has the following property d i=0 e t(v i,v i+ ) W (v i ) y [ ( 2 ε(g) )( cos ( π n ))] y d i=0 e t(v i,v i+ ), (20) where ε(g) is the edge-cut or edge connectivity of the graph. The edge-cut/edge connectivity is defined as the minimal number of edges whose removal would result in disconnected graph. n is the number of vertices in the graph. Proof: Similar to Lemma 7, we use the lower bound of Fiedler value in Lemma 6. Consider G(V,E) and let ε(g) be the edge-cut of the graph. Then, λ (G) 2ε(G)[ cos(π/n)]. Now, consider the graph G vi obtained from graph G by removing node v i and all edges connecting to node v i. From upper bound of Fiedler value, we have λ (G vi ) 2ε(G vi )[ cos(π/(n ))]. (2) Since, ε(g vi ) ε(g). Hence, λ (G vi ) 2[ε(G) ][ cos(π/(n ))]. (22) Follow the similar proof in Lemma 7, we can obtain d i=0 e t(v i,v i+ ) W (v i ) y = d e t(v i,v i+) i=0 λ (G vi ) [ ] y y 2[ε(G) ][ cos( π n )] d i=0 e t(v i,v i+ ). Lemma 9: [Complete Graph] For a complete graph, the MTEKC(y) route employing the Fiedler value metric is the same as the MTE route. Proof: From the definitions of MTE route and MTEKC(y) with Fiedler value route, we have the following inequalities d(r ) d(r ) λ (G vi ) y d(r ) d(r ) (23) λ (G vi ) y (24) We note that removing one nodes from a complete graph with n nodes results in another complete graph with n nodes. Therefore, we have λ (G vi )=n. Simplifying (24), we have d(r ) Combining with (23), we have d(r ) = d(r ) d(r ). (25). (26) Since it is impossible to have two different routes with the same total transmit energy in random network deployment for fixed source node v 0 and destination node v d, we conclude that r = r. Now we are ready to develop the upper bound on the energy consumed in the MTEKC(y) algorithm with Fiedler value. The

7 390 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 following theorem gives the upper bound on the consumed energy Theorem : The energy consumed in the MTEKC(y) using Fiedler value satisfies the following upper bound d(r ) e t(v i,v i+) [ ] y d(r (n )m ) e t(v i,v i+) n(n 2)(ε(G) )( cos(π/(n ))) (27) Proof: From inequality (24), Lemma 7, and Lemma 8, we have ( ) y d(r (n 2)n ) 2(n )m (n 2)n 2(n )m d(r ) e t(v i,v i+) λ (G vi ) y d(r ) λ (G vi ) y (28) [ ] y d(r ) 2[ε(G) ][ cos( π n )] (29) Combining the above two inequalities and (24), we have [ ] y d(r ) [ [ 2(ε(G) )( cos( π m(n ) n(n 2)(ε(G) )( cos( π ] y d(r ) n )) (30) d(r ) ] y d(r ) n )) (3) The above theorem gives the upper bound on the energy consumed in MTEKC(y) route compared to the minimum total energy for routing the packet. The following theorem gives the bound on the ratio of energy consumed by MTEKC(y) using Fiedler value when the number of nodes becomes large. Theorem 2: Suppose that the network generated satisfies m = a n and ε(g) = a 2, where a and a 2 are some constants. Then the upper bound on the ratio of energy consumed can be presented as follow d(r ) d(r ) = O((n2 ) y ) (32) Proof: Using the assumption of the theorem, we have d(r ) ( ) y a n(n ) d(r ) a 2 n(n 2)( cos( π d(r ) d(r ) n )) ( n(n )( + cos( π n C )) ) y n(n 2) sin 2 ( π n ), (33) where C =(a /a 2 ) y.asn,wehave d(r ) ( ) (n ) 2 y d(r ) C π 2, (34) Fig.. Exchange and Update Q-value where we have used small angle approximation in sinusoidal function, sin(θ) θ, asθ. Hence, we have (32). From this theorem, we see that when the network is very large, the ratio of energy increases as a quadratic function of number of nodes, compared to the minimum energy used to route a packet. This ratio of energy can be easily controlled by the parameter y, for instance if y =/2, then the ratio of consumed energy increases as a linear function of number of nodes in the network. In the extreme case, setting y = O(/n) makes the proposed algorithm approaching to MTE as n. VI. DISTRIBUTED IMPLEMENTATION AND LEARNING ALGORITHM In this section, we present the distributed implementation of the proposed robust connectivity routing algorithm. The method is based on the distributed reinforcement learning routing algorithm [7]. We note that the reinforcement learning algorithm has been shown to be an effective online decision making procedure in sensor network application [8]. The resulting algorithm can be characterized as a version of distributed Bellman-Ford algorithm that performs its path relaxation step asynchronously and online with the edge cost defined as weighted energy required to transmit packet in that hop [6]. Each node learns the routing decision by itself and maintains the best packet delivery cost to its destinations. In particular, each node v i maintains a table of Q-values Q vi (v j,v d ),forv j S vi,wherev j is in the set of node v i neighbors, S vi, and node v d is the destination. The Q vi (v j,v d ) has the interpretation of node v i s best estimated cost that a packet would incur to reach its destination node v d from node v i when the packet is sent via node v i s neighbor node v j. Whenever node v i wants to send packets to node v d,itsimply sends the packets to its neighbor that satisfies v j =arg min v j S vi Q vi (v j,v d ). (35) The value in the Q-table will be exchanged between node v i and v j, whenever a packet is sent from node v i to node v j, and vice versa. The exchange mechanism is illustrated as in Figure. Whenever node v i transmits a packet P to node v j, node v j feeds back Q vj (v k,v d)= min Q vj (v k,v d ) (36) v k S vj to node v i asshowninthefigure. We note that Q vj (v k,v d) represents the best estimated cost incurred when a packet is

8 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 39 sent from node v j to the destination node v d. The node i uses this value to update its own Q-value as follow Q vi (v j,v d )=( δ)q vi (v j,v d )+δ[q vj (vk,v d )+c(v i,v j )], (37) where c(v i,v j ) is the cost for sending packet from node v i to node v j,andδ [0, ] is the learning rate for the algorithm. It is important to point out that the storage requirement for the distributed algorithm is O(d vi (n )), since each node requires to store at most d vi (n ) Q-values, where d vi is the number of neighbors, node v i has. Moreover, the distributed algorithm has O() computational complexity per packet transmission, since for every packet transmission, the sender and the receiver update their corresponding Q-values according to (37). The cost of sending a packet between node v i and node v j is related to the energy consumption for sending the packet. In particular, the costs of sending a packet for MTE and MTEKC routing algorithms are [MTE]: c(v i,v j)=e t(v i,v j)+e r(v i,v j), [MTEKC(y)]: c(v i,v j)=e(v i,v j)w (v i) y + e r(v i,v j)w (v j) y, For MTE, Q vi (v j,v d ) represents the total energy consumption used to guarantee delivery a packet from node v i to node v d via node v i s neighbor node, v j. In contrast, Q vi (v j,v d ) in MTEKC represents the total energy consumption in delivering a packet from node v i to v d via v j, while considering the connectivity of the remaining network. The procedure for implementing the MTEKC is summarized in Table IV. We note that when δ =.0, the algorithm becomes the distributed Bellman-Ford iterations [6]. A. Improvement on distributed algorithm In previous description, it is obvious that the quality of the routing decision depends on the accuracy of the Q-table. The more accurate the Q-table represents the network state, the better the routing decision will be. Moreover, the quality of the Q-table depends on how frequently it is updated. In the distributed scheme described in previous subsection, the Q- value is updated whenever a packet is transmitted from the transmitter to the receiver. This existing distributed algorithm can be enhanced by improving the accuracy of the Q-table. The accuracy of the Q-table can be improved by updating the Q-values not only when there is a packet transmission between two nodes, but nodes periodically send their neighboring nodes small packets containing the request on updating Q vi (v j,v d ). This can be straightforwardly done by updating Q vi (N(v i ),v d ) as if there are packets to be sent from node v i to nodes N(v i ). This approach is similar to the use of packet radio organization packet (PROP) [9]. We note that the overhead encountered in each request on updating Q vi (v j,v d ) is only Q vj (vk,v d) as in (36), which will be fed back by node v j. This overhead depends on how many bits the Q-value is represented. Typically, 8 bits will be sufficient. It is important also to note that this small packet consumes negligible energy compared to the energy for the packet transmission. Therefore, these small packets can be sent more frequently to improve the learning speed of the Q-table. In a real application, there is a trade-off between how fast the link cost changes and how frequently the small packets should be exchanged. We will show in the simulation that the improvement in the distributed implementation achieves the near-centralized solution. B. Distributed weight computation and discussion on the scalability In the network initialization, each node will power up and start to broadcast control/organization packet similar to the DARPA packet radio network protocol [9]. This packet contains information about its neighboring nodes (nodes that are connected/reachable from this node). This packet will be broadcasted to all of its neighbors. The nodes that are one hop away from the origin node will also ripple this information outward in the tiered fashion by appending its own neighbor information. This is similar to the method of building tier table in [9]. Therefore, all the nodes have the correct view of the network topology. Before performing routing task, each node calculates its connectivity weight based on this topology view as in Table I. Precisely, the final topology view consists of the adjacency matrix of the graph. Since, nodes stay in their original places for their entire lifetime in the sensor network, the flooding of information occurs rarely. This event occurs whenever there are nodes joint the network or there are nodes that have failed. We assume that this event rarely happens after the network initialization. In practical network, maintaining the algebraic connectivity of the whole large network may not be necessary. Based on different applications of sensor network, the sensor network typically can be decomposed into several smaller subnetworks which are interconnected by access points (APs). These APs may not have power limitation. Each AP is responsible for its own subnetwork. In order to exchange information between nodes within each subnetwork, it is very important to maintain the connectivity within each subnetwork. It is within this smaller subnetworks, the keep-connect algorithm is applied. In this way, the neighboring nodes information is only disseminated over the smaller subnetworks and the improved distributed algorithm can be done on the fly. Using this hierarchical structure, the routing across subnetworks can be handled via the APs. And, the proposed method can be used in this way in a very large network where the connectivity of the whole network is maintained via combination of local connectivity and the APs. VII. SIMULATION RESULTS We simulate the routing algorithms in a packet level discrete-event simulator. The simulator initially deploys nodes in the network. All events are time-stamped and queued. The most current event will be dequeued and some task will be performed according to the type of the event. There are three types of events; the packet arrival event, the reporting event, and the sending event. In packet arrival event, packets are injected from source node to destination node. We assume the packet arrival follows the Poisson arrival process with mean μ. The reporting event occurs periodically to retrieve the simulation parameters such as the average delivery delay per packet, average hops per packet transmission, energy consumed per one delivered packet, and the number of packets

9 392 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 TABLE IV DISTRIBUTED ASYNCHRONOUS MTEKC(Y). Network Initialization: forward the adjacent neighbors information to all nodes in the network 2. Each node computes its own connectivity weight 3. Whenever node v i sends a packet to node v j : a. Node v j informs v i its minimum cost transmitting a packet to the destination v d, Q vj (v k,v d) b. Node v i updates its metric as: Q vi (v j,v d )=( δ)q vi (v j,v d )+δ[q vj (v k,v d)+c(v i,v j )] c. Node v i leaves other estimates unchanged. 4. When a node dies, the neighboring nodes flood this information to the rest of the network and repeat step 2 Normalized network lifetime Normalized transmit energy per packet Normalized packet delivery time Normalized total delivered packets MMRC MH MHKC(y=) Fig. 2. Comparison of normalized metric for MMRC, MH, and MHKC(y=) w.r.t. MTE algorithm, when the packet arrival follows the Poisson process with mean μ =.0. delivered in this report interval. All events that are neither the packet arrival events nor reporting events are the sending events. In the sending event, a packet is sent to the next hop. The next hop is determined based on the routing algorithm used. Whenever a packet arrives at a node, it is queued in the node s buffer and will be sent in the next transmission time. Whenever a packet reaches its destination, the number of delivered packets is incremented and the event associated with that packet is freed. The channel between two adjacent nodes in the network is modeled as fading process that attenuates the transmit signal proportionally to the distance, d α. We note that the main focus of this paper is on the static sensor network. In the static sensor network, the effect of fast fading is negligible, since practically the Doppler shift experienced by the receivers is zero. Therefore, this model is general enough to describe both the free space propagation and the two ray ground propagation model, which has been typically used in many ad-hoc simulators [20]. In this simulation section, we first generate 0 uniform random networks in the area of 00m by 00m with 36 nodes. The networks have average edges per node from 4.4 to 7.5. All nodes in the network initially have the same amount of energy. The source and destination are selected uniformly from the alive nodes. We use the network lifetime (time before the remaining network becomes disconnected), packet delivery time, average transmit energy per packet, and total delivered packet before the remaining network becomes disconnected as our performance metrics. In Figure 2, we compare the performance of the normalized max-min residual connectivity (MMRC) routing, normalized the minimum hop (MH) routing, and normalized minimum hop while keeping connectivity (MHKC(y=)) routing. All metrics are normalized with respect to the performance of MTE algorithm. From the figure, we observe that all the algorithms have lower network lifetime compared to the MTE algorithm. This also verifies that MTE is not less effective than MMRE type algorithm when one defines the network lifetime as the time before the remaining network becomes disconnected as discussed in Section IV. This is partly due to the fact that MH, MHKC, and MMRE do not use the transmission energy to guide the routing decision, hence the route is not energy efficient at all. This can be observed from the figure that all the algorithms consume 20% to 90% more energy per packet. However, the MH and MHKC take only 50% less time to deliver one packet, this is clear because MH based algorithms select route that has the minimum hops from source to destination, hence the resulting packet delivery delay is the smallest. Since considering the energy efficient route is very important, we will focus on MTE type algorithm for the rest of

10 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 393 Normalized network lifetime Normalized TxRx Energy Normalized routing time Normalized successful delivered packets load μ=.0 load μ=2.0 load μ= Fig. 3. Comparison of normalized metrics for MTEKC(y=) w.r.t. to MTE for different packet arrival rate. TABLE V SIMULATION PARAMETERS SET 2 Network Nodes Algebraic network connectivity the simulation section. Next, we simulate both 2D Poisson network and uniform random network. The nodes in the 2D Poisson network are deployed/generated based on 2D Poisson process, while the nodes in uniform random network are generated based on uniform distribution. We also consider 2 types of source and destination pairs, namely converge-cast traffic and random traffic. The converge-cast traffic refers to the pattern where the sources are generated from alive nodes to a fixed destination (sink). And, the random traffic refers to the pattern where the source and destination are selected randomly from the nodes that have not failed. We generate 5 networks, where nodes are generated using 2D Poisson point process with the following parameters. Networks with averaged 00 nodes are generated using 2D Poisson point process in the area of 00m by00m. The transmit energy per packet between two nodes is equivalent to d 2, where d is the distance between the nodes. The maximum transmit power equals to.4 Watt per packet, receive power equals to 0.7 Watt per packet. This implies that the farthest node that can be reached by a node is about 2.602m away. The initial energy of all nodes is equal to energy unit. We also show the algebraic network connectivity of the original graph in Table V. Figure 3 shows the simulation results for normalized MTEKC(y=) w.r.t. MTE algorithm for the generated networks. As before, this figure shows the normalized performance metrics with respect to MTE algorithm. These metrics are collected from the initialization of the network until the network becomes disconnected. The x-axes of these sub-figures denote the network number shown in Table V. From this figure, we observe that the improvement of keep-connect algorithm w.r.t MTE is about 20% for load (μ =.0) innetwork2. In summary, one can observe that the proposed algorithm achieves from 0% to 53% increase in the network lifetime for broad network loads. Similarly, the proposed algorithm achieves around 0% 36% improvement in the total delivered packets. We note that the keepconnect algorithm is about 20% less energy efficient compared to MTE. Besides being able to have longer lifetime and larger total delivered packets, our proposed algorithm is also more robust in terms of the algebraic network connectivity. This can be seen from Figure 4. This figure shows the decrease in the algebraic network connectivity whenever a node dies. The x-axis is the number of failed nodes before the network becomes disconnected and the y-axis is the algebraic network connectivity. We note that the zero algebraic network connectivity implies the disconnected network. It is obvious that our algorithm is more robust in terms of algebraic network connectivity. As shown in Figure 4, there is typically more than one node dies before the network becomes disconnected. We also simulate converge-cast traffic innetwork of Table V and run the simulation for 00 realizations. We randomly select the destination to be node 94 and assume that it has infinite energy. Figure 5 shows that our proposed algorithm constantly outperforms the MTE algorithm. Next, we show the performance of the distributed imple-

11 394 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 The robustness of our proposed algorithm in terms of network connectivity 0.7 MTE network MTEKCFiedler network 0.6 MTE network 2 MTEKCFiedler network 2 Algebraic network connectivity Robustness of our proposed algorithm in terms of network connec- Fig. 4. tivity Network lifetime Delivered packets Number of dead nodes 4.6 x 04 Comparison for MTEKCFiedler() and MTE algorithm for converge cast traffic load= x Network realization Network realization MTE MTEKCFiedler() Fig. 5. Comparison of normalized metrics for MTEKC(y=) w.r.t. to MTE for converge cast traffic. mentation presented in Section VI in Figure 6. The distributed solution sends around 0 small packets in between sending the actual packets to facilitate better learning result. If one node dies, each node increases the sending of small packets to quickly learn the correct route. The implementation is straightforward. The learning process for the distributed solution for the converge cast traffic innetwork is shown in Figure 6. This figure shows that the improved distributed solution can nearly achieve the centralized solution. Finally, in order to get statistics of improvement of keepconnect algorithm, we study the effect of routing algorithm on random network where nodes are randomly placed uniformly in an area of 00m by00m with random traffic. For each number of nodes (00 to 200 nodes), we generated 20 networks, we averaged the normalized network lifetime for MTEKC(y =, 2, 3) with respect to MTE algorithm. The results are plotted in Figure 7 to 9. In these Figures we show both the mean, m and unbiased standard deviation, s of the performance metrics given below by means of error bar, m = N x n, N n= s = N (x n μ) N 2, n= where N =20represents 20 network realizations. x n denotes the normalized network lifetime and normalized number of nodes failed before the network becomes disconnected in the left and right sub-figures of Figures 7 to 9, respectively. We note that Figure 7 shows the case where the attenuation is α =2.0 and load is μ =.0. In contrast, we also simulate the case where α =4.0 and μ =.0 in Figure 9. Comparing these figures, we conclude that the proposed algorithm is not sensitive to the choice of attenuation coefficient α. From all these figure, we show that the proposed algorithm achieves about 0% to 20% performance improvement compared to MTE algorithm. Moreover, the proposed algorithm is more robust in terms of algebraic network connectivity, which may be a much more important criterion to be considered in the network protocol design. This can be seen from these Figures, there is more number of nodes failed before the network becomes disconnected in our proposed algorithm. VIII. CONCLUSIONS In this paper, we employ the time before the network becomes disconnected as our definition of network lifetime. Using this definition, we propose a class of routing algorithms called keep-connect algorithms that use computable measures of network connectivity in determining how to route packets. The algorithm embeds the importance of a node when making the routing decision. The importance of a node is quantified by the Fiedler value of the remaining network when that particular node dies. The proposed algorithm achieves 0 20% better network lifetime (time before the network becomes disconnected) and total delivered packets compared to MTE algorithm in low to high arrival rate. More importantly, the proposed algorithm is more robust in terms of algebraic network connectivity. We also present a distributed implementation that nearly achieves the centralized solution. REFERENCES [] Tech. Review, 0 emerging technologies that will change the world, Technology Review, pp , Feb [2] C.-Y. Chong and S. P. Kumar, Sensor networks: evolution, opportunities, and challenges, IEEE Wireless Commun., vol. 9, no. 8, pp , Aug [3] J.-H. Chang and L. Tassiulas, Maximum lifetime routing in wireless sensor networks, IEEE/ACM Trans. Networking, vol. 2, no. 4, pp , Aug [4] C.-K. Toh, Maximum battery life routing to support ubiquitous mobile computing in wireless ad hoc networks, IEEE Commun. Mag., vol. 39, no. 6, pp , June 200. [5] F. R. K. Chung, Spectral Graph Theory. CBMS Regional Conference Series in Mathematics, no. 92, American Mathematical Society, 997. [6] M. Fiedler, Algebraic connectivity of graphs, Czechoslovak Mathematical J., vol. 23, pp , 973. [7], A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory, Czechoslovak Mathematical J., vol. 25, pp , 975.

12 PANDANA and LIU: ROBUST CONNECTIVITY-AWARE ENERGY-EFFICIENT ROUTING FOR WIRELESS SENSOR NETWORKS 395 TxRxEnergy Comparison of improved distributed solution for centralized and distributed MTEKCFiedler() centralized improved distributed Simulation time x 0 4 Routing time Simulation time x 0 4 Delivered packets Simulation time x 0 4 Fig. 6. Performance of Tx and Rx Energy, Routing time, and the delivered packets for improved distributed solution when μ =.0..3 Normalized lifetime in uniform network with random traffic load=.0 Normalized performance in uniform network with random traffic load= Normalized lifetime y= y=2 y= Number of nodes Normalized number of nodes failed before disconnected y= y=2 y= Number of nodes Fig. 7. Normalized performance in random network with random traffic when the packet arrival load is μ =.0 and α =2.0 Normalized lifetime Normalized lifetime in uniform network with random traffic load= y= y=2 y= Number of nodes Normalized number of nodes failed before disconnected Normalized performance in uniform network with random traffic load= y= y=2 y= Number of nodes Fig. 8. Normalized performance in random network with random traffic when the packet arrival load is μ =3.0 and α =2.0

13 396 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 7, NO. 0, OCTOBER 2008 Normalized lifetime Normalized lifetime in uniform network with random traffic load=.0 y= 0.6 y=2 y= Number of nodes Normalized number of nodes failed before network is disconnected Normalized performance in uniform network with random traffic load= Number of nodes y= y=2 y=3 Fig. 9. Normalized performance in random network with random traffic when the packet arrival load is μ =.0 and α =4.0 [8] T. S. Rappaport, Wireless Communications: Principle and Practice, 2nd ed. Upper Saddle River, NJ: Prentice Hall, 999. [9] A. Chandrakasan, W. Heinzelman, and H. Balakrishnan, Energyefficient communication protocol for wireless microsensor networks, in Proc. 33rd Annual Hawaii International Conference on System Sciences, pp. 0, Jan [0] D. M. Blough and P. Santi, Investigating upper bounds on network lifetime extension for cell-based energy conservation techniques in stationary ad hoc networks, in Proc. 8th Annual International Conference on Mobile Computing and Networking, pp , [] Q. Li, J. Aslam, and D. Rus, Online power-aware routing in wireless ad hoc networks, in Proc. 4th Annual IEEE/ACM MOBICOM, Rome, Italy, pp , July 200. [2] A. Sankar and Z. Liu, Maximum lifetime routing in wireless ad-hoc networks, in Proc. IEEE INFOCOM, pp , Mar [3] B. Mohar, Some applications of Laplace eigenvalues of graphs, Hahn and G. Sabidussi, eds., Graph Symmetry: Algebraic Methods and Applications, NATO ASI Series C, Kluwer, Dordrecht, vol. 497, pp , 997. [4] R. A. Horn and C. R. Johnson, Matrix Analysis, st ed. Cambridge, UK.: Cambridge University Press, 985. [5] R. Diestel, Graph Theory, 3rd ed. Springer-Verlag, [6] D. P. Bertsekas, Network Optimization: Continuous and Discrete Models. Belmont, MA.: Athena Scientific, 999. [7] M. L. Littman and J. A. Boyan, A distributed reinforcement learning scheme for network routing, Advances in Neural Infromation Processing Systems, vol. 6, pp , 993. [8] C. Pandana and K. J. R. Liu, Near-optimal reinforcement learning framework for energy-aware sensor communications, IEEE J. Select. Areas Commun., vol. 23, no. 4, pp , Apr [9] J. Jubin and J. D. Tornow, The darpa packet radio network protocols, Proc. IEEE, vol. 95, no., pp. 2 34, Jan [20] Y. Xu, J. Heidemann, and D. Estrin, Geography-informed energy conservation for ad-hoc routing, in Proc. MOBICOM, pp , July 200. societies. Charles Pandana received his B.S. degree and M. S. degree in electronics engineering from the National Chiao Tung University, Hsinchu, Taiwan, in 998 and 2000, and the Ph.D. degree in electrical and computer engineering from University of Maryland in He is currently working as a system research engineer in ArrayComm LLC. His research interests include stochastic modeling/learning and system level/network performance analysis for next generation wireless networks. He is a member of the IEEE Signal Processing and Communication K. J. Ray Liu (F-03) is Professor and Associate Chair, Graduate Studies and Research, of Electrical and Computer Engineering Department, and Distinguished Scholar-Teacher of University of Maryland, College Park. He leads the Maryland Signals and Information Group conducting research encompassing broad aspects of information technology including signal processing, communications, networking, information forensics and security, biomedical and bioinformatics. Dr. Liu is the recipient of numerous honors and awards including best paper awards from IEEE Signal Processing Society (twice), IEEE Vehicular Technology Society, and EURASIP; IEEE Signal Processing Society Distinguished Lecturer, EURASIP Meritorious Service Award, and National Science Foundation Young Investigator Award. He also received various teaching and research recognitions from University of Maryland including university-level Invention of the Year Award and collegelevel Poole and Kent Company Senior Faculty Teaching Award. Dr. Liu is Vice President Publications and on the Board of Governor of IEEE Signal Processing Society. He was the Editor-in-Chief of IEEE SIGNAL PROCESSING MAGAZINE and the founding Editor-in-Chief of EURASIP JOURNAL ON APPLIED SIGNAL PROCESSING. His recent books include Cooperative Communications and Networking, Cambridge University Press, 2008; Resource Allocation for Wireless Networks: Basics, Techniques, and Applications, Cambridge University Press, 2008; Ultra-Wideband Communication Systems: The Multiband OFDM Approach, IEEE-Wiley, 2007; Network-Aware Security for Group Communications, Springer, 2007; Multimedia Fingerprinting Forensics for Traitor Tracing, Hindawi, 2005.