Abstract. Index Terms

Transcription

1 Clustering in Distributed Incremental Estimation in Wireless Sensor Networks Sung-Hyun Son, Mung Chiang, Sanjeev R. Kulkarni, Stuart C. Schwartz Abstract Energy efficiency, low latency, high estimation accuracy, and fast convergence are important goals in distributed incremental estimation algorithms for sensor networks. One approach that adds flexibility in achieving these goals is clustering. In this paper, the framework of distributed incremental estimation is extended by allowing clustering amongst the nodes. Among the observations made is that a scaling law exists where the estimation accuracy increases proportionally with the number of clusters. The distributed parameter estimation problem is posed as a convex optimization problem involving a social cost function and data from the sensor nodes. An in-cluster algorithm is then derived using the incremental subgradient method. Sensors in each cluster successively update a cluster parameter estimate based on local data, which is then passed on to a fusion center for further processing. We prove convergence results for the distributed in-cluster algorithm, and provide simulations that demonstrate the benefits clustering for least squares and robust estimation in sensor networks. Index Terms Distributed estimation, optimization, incremental subgradient method, clustering, wireless sensor networks. This research was supported in part by the ONR under grant N , the Army Research Office under grant DAAD , Draper Laboratory under IR&D 600 grant DL-H-54663, and the National Science Foundation under grant CCR Portions of this work were presented at the 005 IEEE International Conference on Wireless Networks, Communications, and Mobile Computing, Maui, HI, USA, June 3-7, 005. The authors are with the Department of Electrical Engineering, Princeton University, Princeton, NJ USA. {sungson, chiangm, kulkarni, stuart}@princeton.edu.

2 I. INTRODUCTION A wireless sensor network (WSN) is comprised of a fusion center and a set of geographically distributed sensor nodes. The fusion center provides a central point in the network to consolidate the sensor data while the sensor nodes collect information about a state of nature. In our scenario, the fundamental objective of a WSN is to reconstruct that state of nature, e.g., estimation of a parameter, given the sensor observations. Depending on the application and the resources of the WSN, many possible algorithms exist that solve this parameter estimation problem. One failsafe approach that accomplishes this objective is the centralized approach in which all sensor nodes send their observations to the fusion center and allow the fusion center to make the parameter estimation. The centralized scheme allows the most information to be present when making the inference. However, the main drawback is the drainage of energy resources from each sensor of the WSN []. In an energy constrained WSN, the energy expenditure of transmitting all the observations to the fusion center might be too costly, thus making the method highly energy inefficient. In our application, the purpose of a WSN is to make an inference, not collect all the sensor observations. Another approach avoids the fusion center altogether and allows the sensors to collaboratively make the inference. This approach is referred to as the distributed in-network scheme, recently proposed by Rabbat and Nowak []. First, consider a path that passes through all the sensor nodes and visits each node only once. The path hops from one neighbor node to another until all the sensor nodes are covered. Instead of passing the data along the sensor node path, a parameter estimate is passed from node to node. As the parameter estimate passes through each node, each node updates the parameter estimate with its own local observations. The distributed in-network approach significantly reduces the transmission energy for communication required by the network. However, this approach has drawbacks in terms of latency, accuracy, and convergence. While the centralized approach takes one iteration to have access to all data, the distributed approach takes n iterations to have seen all the data captured by the network, where n is the number of sensors. Also, the parameter estimate of the distributed in-cluster algorithm is less accurate when com-

3 3 pared to the parameter estimate of the centralized algorithm. In terms of the number of iterations, the distributed in-network scheme converges slower than the centralized scheme. The distributed in-network scheme remedies the issue of energy inefficiency, but suffers in terms of these other performance parameters. In this paper, we consider a hybrid form of the two aforementioned approaches. While the former approach relies heavily on the fusion center and the latter approach eliminates the fusion center altogether, we allow the fusion center to minimally interact with the sensor nodes. We formulate a distributed in-cluster approach where the nodes are clustered and there exists a path within each cluster that passes through each node only once as shown in Fig.. While the precise mathematical formulation of the algorithm is stated in Section IV, roughly speaking, each cluster operates similarly to the distributed in-network scheme. Within each cluster, every sensor node updates its own parameter estimate based on its own local observations. Hence, each cluster has its own parameter estimate. The sensor node that initiates the algorithm is designated to be the cluster head. After completion of all the iterations within each cluster, the parameter estimate of each cluster is then passed to the fusion center and averaged. Then, the fusion center announces the average parameter value back to the cluster heads to start another set of cluster iterations if necessary. The purpose of clustering is to address the inherent inflexibility of both the centralized and the in-network algorithms. For example, if the WSN application calls for the most accurate estimate regardless of the communication costs, then the centralized algorithm would suffice. If the WSN demands the most energy efficient algorithm irrespective of the other performance parameters like latency, accuracy, and convergence speed, then the distributed in-network algorithm would be most suitable. However, given a WSN application with specific accuracy demands or energy constraints, are we able to develop an algorithm that is tailored to those desired performance levels? With the distributed in-cluster algorithm, it is more feasible since the number of clusters, or equivalently the size of the clusters, adds another dimension to the algorithm development process. We are also able to adjust the cluster size to accommodate the WSN requirements. Throughout the rest of the paper, we consider the following criteria in comparing the distributed

4 4 in-cluster, distributed in-network and centralized algorithms: Energy efficiency Latency Estimation accuracy Flexible tradeoff among accuracy, robustness, and speed of convergence We show that the proposed distributed in-cluster algorithm adds a flexible tradeoff among all the aforementioned criteria. Specifically, due to clustering, we are able to control the estimation accuracy since the residual error scales as a function of the number of clusters. The inclusion of clusters improves the scaling behavior of the estimation accuracy and latency. We use the centralized and the distributed in-network algorithms as extreme cases of maximal and minimal energy usage, respectively. For the special case where the WSN has n clusters with each cluster having n sensors, we show that the transport cost of the distributed in-cluster algorithm has the same order of magnitude as the distributed in-network algorithm. However, the latency and accuracy improve by factor of n under this specific clustering situation. The organization of the paper is as follows. Previous work in the areas of distributed incremental estimation and clustering is discussed in Section II. We formulate the problem and provide two concrete applications in Section III. The distributed in-cluster algorithm is precisely formulated in Section IV and convergence analysis is discussed in Section V. Section VI surveys the benefits of clustering on the performance parameters. Analytical results are verified in Section VII by simulations that involve two applications, least-squares and robust estimation, followed by the conclusion in Section VIII. II. PREVIOUS WORK The ideas of incremental subgradient methods and clustering applied to distributed estimation are quite prevalent in the literature. Incremental subgradient methods were first studied by Kibardin [3] and then, more recently, by Nedić and Bertsekas [4]. Then, Rabbat and Nowak [] applied the framework used in [4] to handle the issue of energy consumption in distributed estimation in WSNs. To further save on energy expenditure, Rabbat and Nowak [5] also implemented a

5 5 quantization scheme for distributed estimation, in which they showed that quantization does not effect the rate of convergence for the incremental algorithms. Clustering schemes have been implemented throughout the area of WSNs to provide a hierarchal structure to minimize the energy spent in the system (see [6] and references therein). The purpose of clustering is either to minimize the number of hops the data needs to arrive at a destination or to provide a fusion point within the network to consolidate the amount of data sent. In our work, clustering is applied to the framework of distributed incremental estimation to provide a better scaling law for estimation accuracy and latency in relation to the number of clusters. III. PROBLEM FORMULATION Consider a WSN with n sensors with each sensor taking m measurements. The parameter estimation objective of the WSN can be viewed as a convex optimization problem if the distortion measure between the parameter estimate and the data is convex. The problem is minimize f(x, θ) subject to θ Θ () where f : R nm+ R is a convex cost function, x R nm is a vector of all the observations collected by the WSN, θ is a scalar, and Θ is a nonempty, closed and convex subset of R. Note that x are the constants while θ is the variable. One method to decompose Problem () into a distributed optimization problem is to assume that the cost function has an additive structure. The additive property states that the social cost function given all the WSN data, f(x, θ), can be expressed as the normalized sum of individual cost functions given only individual sensor data, f i (x i, θ). Hence, the problem becomes minimize f(x, θ) = n n i= f i(x i, θ) subject to θ Θ () where f i (x i, θ) : R m+ R is a convex local cost function for the ith sensor only using its own measurement data x i R m.

6 6 Although the additive property does not hold for general cost functions, two important applications in estimation satisfy this property: least squares estimation and robust estimation with the Huber loss function. A. Least Squares Estimation The simplest estimation procedure is least-squares estimation. For the classical least squares estimation problem, the distortion measure is f(x, θ) = x θ, where is the Euclidean norm. Clearly, the least-squares distortion measure is convex and additive. Hence, the optimization problem is formulated as follows, minimize n n i= subject to θ Θ { m m p= xp i θ } (3) where f i (x i, θ) = m m p= xp i θ and x p i denotes the pth entry in the vector of observations from sensor node i. The beauty of least-squares estimation lies in its simplicity, but the technique is prone to suffer greatly in terms of accuracy if some of the measurements are not as accurate as the others. If some measurements have higher variances than other measurements, the least-squares inference procedure does not take this effect into account. Thus, the least squares procedure is highly sensitive to large deviations. To make the inference procedure more robust to these types of deviations, the following robust estimation procedure is often used. B. Robust Estimation Another practical application is the robust estimation problem, which has the following form, minimize n n i= subject to θ Θ { m m p= f H(x p i, θ) } (4) where x p f H (x p i, θ) = i θ /, if x p i θ γ (5) γ x p i θ γ /, if x p i θ > γ

7 7 and γ 0 is the Huber loss function constant [7]. Note, f i (x i, θ) = m m p= f H(x p i, θ). The purpose behind robust estimation is to introduce a new distortion measure that puts more weight on good measurements, and less weight on, or even discards, bad measurements. The parameter γ sets the threshold for the measurement values around the parameter estimate θ, i.e., the values within a γ range of θ are considered good measurements and the values outside a γ range of θ are considered bad measurements. As γ, robust estimation reduces to least-squares estimation. IV. DISTRIBUTED IN-CLUSTER ALGORITHM To solve a convex optimization problem like (), the most common method used is a gradient descent method. Given any starting point ˆθ dom f, update ˆθ by descending along the gradient of f. To formalize this procedure, let ˆθ new = ˆθ old + α ˆθ, (6) where α is the step size and ˆθ = f. The convexity of the function f guarantees that a local minimum will be a global minimum. However, if the function f is not differentiable, then a subgradient can be used. A subgradient of any convex function f(x) at a point y is any vector g such that f(x) f(y) + (x y) T g, x. For a differentiable function, the subgradient is just the gradient. Along with convexity, if the cost function has an additive structure, a variant of the subgradient method can be used. This method is called the incremental subgradient method [8]. The key idea of the incremental subgradient algorithm is to sequentially take steps along the subgradients of the marginal function f i (x i, θ) instead of taking one large step along the subgradient of f(x, θ). In doing so, the parameter estimate can be adjusted by the subgradient of each individual cost function given only its individual observations. Following this procedure leads to the distributed in-network algorithm []. The convergence results for the algorithm follow directly from the incremental subgradient method. We now describe the in-cluster algorithm. Consider a WSN with clusters and n S sensors per cluster, where n S = n. Assume that n S and are factors of n. Note that the distributed

8 8 in-network algorithm can be viewed as a special case of the distributed in-cluster algorithm where each sensor is its own cluster, = and n S = n. We use i = to index sensor nodes, j to index clusters, and k to index the iteration number. Let i = 0 represent the cluster head. Let f i,j (x i,j, θ) and φ i,j,k denote the local cost function and the parameter estimate at node i in cluster j during iteration k, respectively. For conciseness, we suppress the dependency of f on the parameters in the notation and let f i,j (x i,j, θ) = f i,j (θ) and f(x, θ) = f(θ). Also, let θ k be the estimate maintained by the fusion center during iteration k. We need to update θ k for k =,,..., by local updates of φ i,j,k. Each iteration k of the distributed in-cluster algorithm proceeds as follows: ) Fusion center passes the current estimate θ k to the cluster heads in all clusters. Cluster heads initialize φ 0,j,k = θ k, j. ) Incremental update is conducted in parallel in all the clusters. Within each cluster, the updates are conducted through update paths that traverse all the nodes in each cluster: φ i,j,k = φ i,j,k α k g i,j,k n where α k is the step size and g i,j,k is a subgradient of f i,j (7) using the last estimate φ i,j,k and the local measurement data x i,j. This is denoted as g i,j,k δf i,j (φ i,j,k ). If φ i,j,k / Θ, then project φ i,j,k onto a nearest point in the a priori constraint set Θ. 3) All clusters pass the last in-cluster estimate φ ns,j,k to the fusion center, which takes the average to produce the next estimate θ k+ = nc φ ns,j,k. 4) Repeat. In step 3 of the distributed in-cluster algorithm, the fusion center may process the in-cluster estimates {φ ns,j,k} using a variety of methods, e.g. a weighted average depending on the signal to noise ratio of the observations. By involving the fusion center, this allows more flexibility in the

9 9 algorithm development. In the convergence proofs, we will consider the case where an average of the in-cluster estimates was performed. V. CONVERGENCE Following the approach for the distributed incremental subgradient algorithm in [8], we show convergence for the distributed in-cluster approach. The main difference in our proofs is the emergence of the clustering values, and n S. In these proofs, we make reasonable assumptions that the optimal solution exists and the subgradient is bounded as shown in the following statements. Let the true underlying state of the environment be the (finite) minimizer, θ, of the cost function. Also, assume there exists scalars C i,j 0 such that C i,j g i,j,k for all i =,..., n S, j =,...,, and k. We start with the following lemma that is true for each cluster parameter estimate {φ i,j,k }. Lemma : Let {φ i,j,k } be the sequence of subiterations generated by Eq. (7). Then for all y Θ and for k 0, φ i,j,k y φ i,j,k y α k ( fi,j (φ i,j,k ) f i,j (y) ) + α k n n C i,j i, j. (8) See the Appendix for the proof. By summing all the inequalities in Eq. (8) over all i =,..., n S and j =,...,, we have the following lemma for each parameter estimate {θ k }. Lemma : Let {θ k } be a sequence generated by distributed in-cluster method. Then, for all y Θ and for k 0 θ k+ y θ k y α k ( f(θk ) f(y) ) + α k Ĉ, (9) n where Ĉ = { n S i= C i,j}. See the Appendix for the proof. Lemma guarantees that the sequence {θ k } k= gets smaller provided α k < n (f(θ k ) f(y)). This Ĉ key lemma is a crucial step in proving all the subsequent theorems.

10 0 Theorem : Let {θ k } be a sequence generated by the distributed in-cluster method. Then, for a fixed step-size, α k = α, and using Lemma, we have lim inf k where f(θ ) = inf θ Θ f(θ) and Ĉ = { n S i= C i,j}. See the Appendix for the proof. f(θ k) f(θ ) + αĉ n (0) If all the subgradients g i,j,k are bounded by one scalar C, we have the following corollary. Corollary : Let C 0 = max i,j C i,j. It is evident that C 0 g i,j,k for all i =,..., n S and j =,...,. Then, for a fixed step-size, α k = α, lim inf k f(θ k) f(θ ) + αc 0. () Since the incremental subgradient method is a primal feasible method, a lower bound of f(θ ) is always satisfied. Therefore, the sequence of estimates f(θ k ) will eventually be trapped between f(θ ) and f(θ ) + αc 0. The fluctuation around the equilibrium, R = αc 0, () is the residue estimation error due to the fact that a constant step size is used for subgradient methods. Comparing Corollary with both the standard results in [8] and the result used by Rabbat and Nowak in [], we observe that in our case, we have a smaller threshold tolerance. Using the same assumptions as iorollary, in the distributed in-network case, lim inf k f(θ k) f(θ ) + αc 0. (3) Thus, as k, the distributed in-network algorithm converges to a ( αc 0 - suboptimal) solution, whereas, the distributed in-cluster algorithm converges to a ( αc 0 - suboptimal) solution. We observe a key advantage of the in-cluster approach: estimation accuracy is tighter by a factor of. Even for medium scale sensor networks, a factor of can be an order of magnitude improvement. The next theorem provides the necessary number of iterations, K, to achieve a certain desired estimation accuracy.

11 Theorem : Let {θ k } be a sequence generated by the distributed in-cluster method. Then, for a fixed step-size α and for any positive scalar ɛ, min 0 k K f(θ k) f(θ ) + ( ) αĉ + ɛ n (4) where K is given by See the Appendix for the proof. nc θ 0 θ K = αɛ. (5) Along the same lines as Theorem and Corollary, we have the following corollary. Corollary : Let C 0 = max i,j C i,j. Then, for a fixed step-size α and for any positive scalar ɛ where K is given by Eq. (5). min f(θ k) f(θ ) + ( αc 0 0 k K ) + ɛ By observing Eq. (6), we see that the index k refers to the parameter estimate obtained at the end of each cluster cycle. Since each cluster has n S iterations, the total number of iterations required for an accuracy of αc ( ) 0 + ɛ is n nc θ 0 θ S. In comparison, for the dis- αɛ tributed in-network case, the total number of iterations required for an accuracy of (αc 0 + ɛ) is n θ0 θ. Therefore, for both algorithms, the total number of iterations necessary is the αɛ same order of magnitude, while the benefit of distributed in-cluster algorithm over the distributed in-network is an accuracy improvement by a factor of. Another natural extension is varying the step-size. For a fixed step-size, complete convergence can not be achieved. The parameter estimates, {θ k }, enter a limit cycle after an arbitrary number of iterations. To force convergence to the optimal value, f(θ ), the step-size can be set to diminish at a rate inversely proportional to the number of iterations, α k = α. More generally, we have the k following theorem. Theorem 3: Let {θ k } be a sequence generated by the distributed in-cluster method. Also, assume that the step-size α k satifies (6) lim α k = 0 k and α k =, k=0

12 then, lim inf k f(θ k) = f(θ ). See the Appendix for the proof. A. Energy Efficiency VI. BENEFITS OF CLUSTERING ON PERFORMANCE PARAMETERS The main expenditure of energy for a WSN is in the cost of communication. This entails transporting bits either from sensor to sensor or from sensor to fusion center. So, the transport cost would be a good measure of the energy usage for a WSN. For example, consider our original WSN consisting of n sensors where each sensor has m measurements. The sensors are distributed randomly (uniformly) over one square meter. We use bit-meters as the metric to measure the transport cost in the transmission of data. In the centralized setting, all n sensors send their m observations to the fusion center, requiring O(mn) bits to be transmitted over an average distance of O() meters. In total, the transport cost is O(mn) bit-meters. In the distributed in-network setting, all n sensors use their m observations to update the parameter estimate and pass the parameter estimate along the path that contains all the sensor nodes. The distributed in-network method needs O(n) bits to be transmitted over an average distance of O( n ). In total, the transport cost is O( n). In the distributed in-cluster setting, the sensor network forms clusters with n S nodes per cluster. This method requires O(n) bits to be transmitted over an average distance of O( n ) meters which accounts for the sensor to sensor transport cost and O( ) bits to be transmitted over an average distance of O() meters which accounts for the sensor to fusion center transport cost. Thus, the transport cost is O( n + ) bit-meters. An interesting result arises when the cluster size and the number of sensors per cluster are equal, = n S = n. For this case, the total transport cost of the distributed in-cluster algorithm becomes O( n). The distributed in-cluster algorithm and the distributed in-network algorithm have the same magnitude in terms of transport cost when = n S = n.

13 3 B. Latency Latency is defined as the number of iterations needed to see all the data captured by the network. For the centralized case, only one iteration is needed while for the in-network case, n iterations are needed. However, the in-cluster algorithm, the latency of the WSN can be adjusted by the size of the cluster. The latency for the in-cluster case reduces to n shown in Table I as shown in Table I. or more simply, n S iterations as C. Estimation Accuracy By forming clusters in a WSN, estimation accuracy can be improved. For the fixed step-size case, the estimation accuracy is reduced by a factor of when compared to the distributed innetwork case as shown in Table I. The accuracy improvement by a factor of holds for both cases where k tends toward infinity and k is finite as shown iorollary and Corollary, respectively. VII. SIMULATIONS, VARIATIONS, AND EXTENSIONS Consider a WSN with 00 sensors uniformly distributed over a region, each taking 0 measurements each. The observations are independent and identically distributed from measurement to measurement, observations are independent from sensor to sensor. If the sensors are working properly, the measurements are distributed by a Gaussian distribution with mean 0 and variance and if the sensors are defective, the measurements are distributed by a Gaussian distribution with mean 0 and variance 00. This application can be viewed as a deterministic mean-location parameter estimation problem. The simulations assume that 0% of the sensor nodes are damaged and a fixed step-size of α k = 0.4 is used. As summarized in this section, a variety of simulations are conducted to verify the theorems and characterize other properties and tradeoffs in the proposed distributed in-cluster algorithm. A. Basic Simulations Least squares estimation and Huber robust estimation are simulated, and the resulting convergence behavior of the residual value is shown in Figs. and 3, respectively. In both figures, the

14 4 distributed in-network method and the distributed in-cluster method are shown by a solid line while the centralized method is shown by a dashed line. Since a fixed step size of α k = 0.4 is used, there are residue fluctuations around the equilibrium. In both estimation procedures, an increase in the number of clusters causes a decrease in fluctuations. The precise data points confirm the theoretical prediction: the distributed in-cluster method fluctuation is smaller than the distributed in-network fluctuation. By observing the least squares estimation plots in Fig., when = 4 and = 0, the fluctuations are smaller by a factor of 4 and 0, respectively, compared to the plot when =. In the robust estimation example of Fig. 3, we use a Huber parameter of γ =. The distributed in-cluster method fluctuation again shows narrower fluctuations and is almost indistinguishable from the centralized estimation curve. B. Accuracy, Robustness and Speed Tradeoff To determine the tolerance bounds for the robust estimation procedure, the gradient of the Huber loss function is calculated. Since f i (θ) = m m p= f H(x p i, θ), it is clear that f i (θ) γ by differentiation, while f i (θ) C 0 by definition. Hence, C 0 can be set to equal γ to provide an upper bound for the gradient. This gives the following results that analytically characterize the tradeoff among three competing criteria: accuracy, robustness, and speed of convergence for incremental estimation. Combining Eq. () with the relation that C 0 = γ, we have the following formula. For a given network size n, the tradeoff among estimation error bound R, Huber robustness parameter γ, and constant step size α is characterized by: R = αγ. (7) For example, to maintain a desired level of robustness γ, tighter convergence bounds (smaller R) implies slower convergence speed (smaller α). As another example, to get tighter convergence bounds, we would like R to be small. This can be achieved by either reducing α, which means

15 5 smaller step size and slower convergence speed, or reducing γ, which means accepting less reliable data for Huber estimation and reducing the robustness as well as the speed of convergence, since less reliable data are used for estimation. An illustrative example is shown in Fig. 4, where we reduce γ by a factor of (cf. Figs. 4a and 4b). This reduction of γ reduces the estimation error by about a factor of 4 but also increases the convergence time by roughly a factor of. The term in the tradeoff characterization in Eq. (7) again highlights an advantage of the in-cluster approach: more clusters help achieve a more efficient tradeoff. VIII. CONCLUSION We have presented a distributed in-cluster scheme for a WSN that uses the incremental subgradient method. By incorporating clustering within the sensor network, we have created a degree of freedom which allows us to tune the algorithm for energy efficiency, estimation accuracy, convergence speed and latency. Specifically, in terms of estimation accuracy, we have shown that a different scaling law applies to the clustered algorithm: the residual error is inversely proportional to the number of clusters. Also, for the special case where a WSN with n sensors forms n clusters, we are able to maintain the same transport cost as the distributed-in-network scheme, while increasing both accuracy of the estimate and convergence speed, and reducing latency. Simulations have been provided for both least squares and robust estimation. We plan to extend our work by relaxing the independence assumption of sensor to sensor observations. In particular, in future work, we will consider a WSN scenario where the data within each cluster are spatially correlated, while the data from cluster to cluster are independent. REFERENCES [] G. J. Pottie and W. J. Kaiser, Wireless integrated network sensors, Communications of the ACM, vol. 43, no. 5, pp. 5 58, May 000. [] M. Rabbat and R. Nowak, Distributed optimization in sensor networks, in Proceedings of the Third International Symposium on Information Processing in Sensor Networks (IPSN 04), Berkeley, CA, 004. [3] V. M. Kibardin, Decomposition into functions in the minimization problem, Automation and Remote Control, vol. 40, pp. 3 33, 980.

16 6 [4] A. Nedić and D. P. Bertsekas, Incremental subgradient methods for nondifferentiable optimization, Tech. Rep., Massachusetts Institute of Technology, Cambridge, MA, 999. [5] M. Rabbat and R. Nowak, Quantized incremental algorithms for distributed optimization, IEEE Journal on Selected Areas iommunications, vol. 3, no. 4, pp , April 005. [6] S. Bandyopadhyay and E. Coyle, An energy efficient hierarchical clustering algorithm for wireless sensor networks, in Proceedings of the nd Annual Joint Conference of the IEEE Computer and Communications Societies (Infocom 003), San Francisco, CA, 003. [7] P. J. Huber, Robust Statistics, John Wiley & Sons, New York, 98. [8] D. P. Bertsekas, Convex Analysis and Optimization, Athena Scientific, Belmont, MA, April 003. A. Proof of Lemma Proof: φ i,j,k y = φ i,j,k α k g i,j,k n APPENDIX y φ i,j,k y α k n (φ i,j,k y)g i,j,k + α k n g i,j,k φ i,j,k y α k n (f i,j(φ i,j,k ) f i,j (y)) + α k n C i,j. The last line of the proof uses the fact that g i,j,k is a subgradient of the convex function f i,j at φ i,j,k. Thus, f i,j (φ i,j,k ) f i,j (y) + (φ i,j,k y)g i,j,k. B. Proof of Lemma Proof: θ k+ y = = ( φns,j,k ) y ( φns,j,k y ) φ ns,j,k y { φ ns,j,k y α k n ( fns,j(φ ns,j,k) f ns,j(y) ) } + α k n C i,j

17 7 where in the third and fourth lines of the proof, we used the Quadratic Mean-Arithmetic Mean inequality and Lemma, respectively. After recursively decomposing φ i,j,k y n S times, we get θ k+ y Then, using the fact that { φ 0,j,k y α k n = θ k y α k n = θ k y α k n + α k n the expression simplifies to n S i= θ k+ y = θ k y α k Then, + α k n n S i= n S C i,j. f(θ k ) = n n S i= { θ k+ y θ k y α k + α k n i= n S i= ( fi,j (φ i,j,k ) f i,j (y) ) + α k n ( fi,j (φ i,j,k ) f i,j (y) ) + α k n } n S Ci,j i= n S Ci,j i= ( fi,j (φ i,j,k ) f i,j (y) + f i,j (θ k ) f i,j (θ k ) ) n S i= f i,j (θ k ), f(θ k ) f(y) + n C i,j. { n S Ci,j i= θ k y α k + α k n { n S Ci,j i= f(θ k ) f(y) + n f(θ k ) f(y) + n n S i= = θ k y α k ( f(θk ) f(y) ) n { C { + α n S i } k C n i,j C m,j + i= m= = θ k y α k ( f(θk ) f(y) ) + α k n ( fi,j (φ i,j,k ) f i,j (θ k ) )} n S i= n S i= C i,j φ i,j,k θ k C i,j n S Ci,j i= } α k n i m= { ns } C i,j i= } C m,j }

18 8 where in the second and third inequalities, we used the fact that f i,j (θ k ) f i,j (φ i,j,k ) C i,j φ i,j,k θ k and respectively. φ i,j,k θ k α k n i C m,j, i =,..., n S, m= C. Proof of Theorem Proof: Proof is by contradiction. If Thm. is not true, then there exists an ɛ > 0 such that Let z Θ be the value so that lim inf k lim inf k f(θ k) > f(θ ) + αĉ n + ɛ. f(θ k) f(z) + αĉ n + ɛ, and let k 0 be sufficiently large so that for all k k 0, we have f(θ k ) lim inf k f(θ k) ɛ. Then, by combining the above two relations with Lemma and setting y = z, we have, for all k k 0 θ k+ z θ k z αɛ. Therefore, which cannot hold for sufficiently large k. θ k+ z θ k z αɛ θ k z 4αɛ. θ k0 z (k + k 0)αɛ

19 9 D. Proof of Theorem Proof: Proof is by contradiction. Assume that for all k with 0 k K, we have ( ) f(θ k ) > f(θ ) + αĉ + ɛ. n By setting α k = α and y = θ in Lemma and by combining that with the above relation, we have for all k with 0 k K, θ k+ θ θ k θ α (f(θ k ) f(θ )) + α Ĉ n n ( ( )) C θ k θ α αĉ + ɛ α Ĉ n n = θ k θ αɛ. If we sum the above inequalities over k for k = 0,,..., K, we have θ K+ θ θ 0 θ (K + ) αɛ. Thus, it is evident that θ 0 θ (K + ) αɛ 0, which contradicts the definition of K. E. Proof of Theorem 3 Proof: Proof is by contradiction. If Thm. 3 does not hold, then there exists an ɛ > 0 such that lim inf k f(θ k) ɛ > f(θ ). Then, using the convexity of f and Θ, there exists a point z Θ such that lim inf k f(θ k) ɛ f(z) > f(θ ). Let, there be a k 0 large enough that for all k k 0, we have f(θ k ) lim inf k f(θ k) ɛ.

20 0 Then, by combining the above two relations, we have for all k k 0, f(θ k ) f(z) ɛ. By setting y = z in Lemma and by combining that with the above relation, we obtain for all k k 0, = θ k z α k α kĉ θ k+ z θ k z α k ɛ + n n ( C ) Since α k 0, we may assume that k 0 is large enough so that ɛ α kĉ n. Thus, for all k k 0, we have which cannot hold for sufficiently large k. ɛ α kĉ n ɛ, k k 0. θ k+ z θ k z α kɛ θ k z (α k + α k )ɛ. θ k0 z ɛ k j=k 0 α j,

21 q k Fusioenter Fig.. Illustration of a sensor network implementing the distributed in-cluster algorithm. The dash-dotted lines represent the borders of the clusters. The shaded nodes represent the cluster heads that communicate with the fusion center. All clusters run the algorithm in parallel, although in the schematic only the lower-right cluster is shown running the incremental subgradient algorithm.

22 Energy Efficiency Latency Estimation Accuracy (bit-meters) (iterations) (residual error) centralized O(mn) 0 distributed in-network O( n) n αc 0 distributed in-cluster O( n + ) n αc 0 special case: ( = n) O( n) n αc 0 n TABLE I SUMMARY OF PERFORMANCE TRADEOFFS AMONG DIFFERENT ALGORITHMS IS SHOWN. NOTE, IN THE SPECIAL CASE WHERE = n, THE DISTRIBUTED IN-CLUSTER CASE AND IN-NETWORK ALGORITHMS HAVE THE SAVE TRANSPORT COST, BUT THE LATENCY AND ESTIMATION ACCURACY IS IMPROVED BY A FACTOR OF / n.

23 3 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in network 3 Least Squares Residual Value Total Number of Iterations (a) Least squares estimation ( = and n S = 00) 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster 3 Least Squares Residual Value Total Number of Iterations (b) Least squares estimation ( = 4 and n S = 5) 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster 3 Least Squares Residual Value Total Number of Iterations (c) Least squares estimation ( = 0 and n S = 0) Fig.. Plots of least squares residual value vs. total number of iterations for three different clustering scenarios are shown. (a) Distributed in-network algorithm, (b) Distributed in-cluster algorithm with = 4 and n S = 5, (c) Distributed in-cluster algorithm with = 0 and n S = 0.

24 4 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in network Robust Residual Value Total Number of Iterations (a) Robust estimation ( = and n S = 00) 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster Robust Residual Value Total Number of Iterations (b) Robust estimation ( = 4 and n S = 5) 5 4 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster Robust Residual Value Total Number of Iterations (c) Robust estimation ( = 0 and n S = 0) Fig. 3. Plots of robust residual value vs. total number of iterations for three different clustering scenarios are shown. (a) Distributed in-network algorithm, (b) Distributed in-cluster algorithm with = 4 and n S = 5, (c) Distributed in-cluster algorithm with = 0 and n S = 0.

25 5 0.8 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster Robust Residual Value Total Number of Iterations (a) Robust Estimation with γ =. 0.8 Fixed Stepsize, Percent Damaged = 0 centralized distributed in cluster Robust Residual Value Total Number of Iterations (b) Robust Estimation with γ = 0.5 Fig. 4. Plots of robust residual value vs. total number of iterations for the case where = 0 and n S = 0 are shown. The plots are rescaled for clarity. (a) Robust estimation with γ =, (b) Robust estimation with γ = 0.5.