Distributed non-convex optimization

Size: px

Start display at page:

Download "Distributed non-convex optimization"

Tiffany Glenn
5 years ago
Views:

1 Distributed non-convex optimization Behrouz Touri Assistant Professor Department of Electrical and Computer Engineering University of California San Diego/University of Colorado Boulder AFOSR Computational Math Program Review 2017

2 Outline What is distributed optimization? A motivating example Main results Sketch of the proof Discussions Behrouz Touri Distributed non-convex optimization 2 / 21

3 Distributed Optimization problem Standard (unconstrained) optimization problem: minimize x R d f(x) f : R d R is a nice (convex, differentiable, etc.) function Behrouz Touri Distributed non-convex optimization 3 / 21

4 Distributed optimization A network of n agents (processors, robots, etc.) Behrouz Touri Distributed non-convex optimization 4 / 21

5 Distributed optimization f 2 (x) 2 f 1 (x) 1 3 f 3 (x) A network of n agents (processors, robots, etc.) Agent i has a local cost function f i : R d R Behrouz Touri Distributed non-convex optimization 4 / 21

6 Distributed optimization f 2 (x) 2 f 1 (x) 1 3 f 3 (x) A network of n agents (processors, robots, etc.) Agent i has a local cost function f i : R d R Agents are not willing to communicate f i s Behrouz Touri Distributed non-convex optimization 4 / 21

7 Distributed optimization 2 f 2 (x) f 1 (x) 1 3 f 3 (x) A network of n agents (processors, robots, etc.) Agent i has a local cost function f i : R d R Agents are not willing to communicate f i s Global objective: each agent aims to n minimize x R d f i (x) i=1 Behrouz Touri Distributed non-convex optimization 4 / 21

8 Why distributed optimization? Application 1: distributed estimation Standard distributed optimization problem: minimize x R d n f i (x), i=1 A network of n sensors [n] := {1,..., n} Behrouz Touri Distributed non-convex optimization 5 / 21

9 Why distributed optimization? Application 1: distributed estimation Standard distributed optimization problem: minimize x R d n f i (x), i=1 A network of n sensors [n] := {1,..., n} Sensor i measures y i = θ + η i, with η i N (0, σ 2 ) Behrouz Touri Distributed non-convex optimization 5 / 21

10 Why distributed optimization? Application 1: distributed estimation Standard distributed optimization problem: minimize x R d n f i (x), i=1 A network of n sensors [n] := {1,..., n} Sensor i measures y i = θ + η i, with η i N (0, σ 2 ) ML decoder for θ: ˆθ argmin x R n i=1 (x y i ) 2 (1) Behrouz Touri Distributed non-convex optimization 5 / 21

11 Why distributed optimization? Application 1: distributed estimation Standard distributed optimization problem: minimize x R d n f i (x), i=1 A network of n sensors [n] := {1,..., n} Sensor i measures y i = θ + η i, with η i N (0, σ 2 ) ML decoder for θ: ˆθ argmin x R n i=1 (x y i ) 2 (1) What if there is no central computing unit but the sensors can communicate? Behrouz Touri Distributed non-convex optimization 5 / 21

12 Why distributed optimization? Application 1: distributed estimation Standard distributed optimization problem: minimize x R d n f i (x), i=1 A network of n sensors [n] := {1,..., n} Sensor i measures y i = θ + η i, with η i N (0, σ 2 ) ML decoder for θ: ˆθ argmin x R n i=1 (x y i ) 2 (1) What if there is no central computing unit but the sensors can communicate? Applications: source localization, distributed machine learning, control of swarm robots, distributed resource allocation, etc. Behrouz Touri Distributed non-convex optimization 5 / 21

13 Distributed optimization: setup Unconstrained distributed optimization problem: minimize x R d i. f i : agent i s local cost n f i (x) (2) i=1 Behrouz Touri Distributed non-convex optimization 6 / 21

14 Distributed optimization: setup Unconstrained distributed optimization problem: n minimize x R d f i (x) (2) i=1 i. f i : agent i s local cost ii. Assumption: agents are not sharing local cost functions Behrouz Touri Distributed non-convex optimization 6 / 21

15 Distributed optimization: setup Unconstrained distributed optimization problem: n minimize x R d f i (x) (2) i=1 i. f i : agent i s local cost ii. Assumption: agents are not sharing local cost functions iii. At time t 0, agent i can share information with a set of neighbors N + i (t) [n] and receives information from N i (t) [n] Behrouz Touri Distributed non-convex optimization 6 / 21

16 Distributed optimization: setup Unconstrained distributed optimization problem: n minimize x R d f i (x) (2) i=1 i. f i : agent i s local cost ii. Assumption: agents are not sharing local cost functions iii. At time t 0, agent i can share information with a set of neighbors N + i (t) [n] and receives information from N i (t) [n] iv. Denote the directed graph by G(t) = ([n], E(t)) Behrouz Touri Distributed non-convex optimization 6 / 21

17 Distributed optimization: setup Unconstrained distributed optimization problem: n minimize x R d f i (x) (2) i=1 i. f i : agent i s local cost ii. Assumption: agents are not sharing local cost functions iii. At time t 0, agent i can share information with a set of neighbors N + i (t) [n] and receives information from N i (t) [n] iv. Denote the directed graph by G(t) = ([n], E(t)) Main question: How to solve problem (2) subject to the above information constraint? How much communication is needed? Behrouz Touri Distributed non-convex optimization 6 / 21

18 Distributed optimization: past works - Nedić, A. and Ozdaglar, A., Distributed subgradient methods for multi-agent optimization. IEEE Transactions on Automatic Control, 54(1), pp Nedić, A., Ozdaglar, A. and Parrilo, P.A., Constrained consensus and optimization in multi-agent networks. IEEE Transactions on Automatic Control, 55(4), pp Wang, J. and Elia, N., 2010, September. Control approach to distributed optimization. In Communication, Control, and Computing (Allerton), th Annual Allerton Conference on (pp ). IEEE. - Lobel, I. and Ozdaglar, A., Distributed subgradient methods for convex optimization over random networks. IEEE Transactions on Automatic Control, 56(6), pp Chen, J. and Sayed, A.H., Diffusion adaptation strategies for distributed optimization and learning over networks. IEEE Transactions on Signal Processing, 60(8), pp Tsianos, K.I., Lawlor, S. and Rabbat, M.G., 2012, December. Push-sum distributed dual averaging for convex optimization. In 2012 IEEE 51st IEEE Conference on Decision and Control (CDC) (pp ). IEEE. - Li, N. and Marden, J.R., Designing games for distributed optimization. IEEE Journal of Selected Topics in Signal Processing, 7(2), pp Bianchi, P. and Jakubowicz, J., Convergence of a multi-agent projected stochastic gradient algorithm for non-convex optimization. IEEE Transactions on Automatic Control, 58(2), pp Gharesifard, B. and Cortés, J., Distributed continuous-time convex optimization on weight-balanced digraphs. IEEE Transactions on Automatic Control, 59(3), pp Kia, S.S., Cortés, J. and Martinez, S., Distributed convex optimization via continuous-time coordination algorithms with discrete-time communication. Automatica, 55, pp Nedić, A. and Olshevsky, A., Distributed optimization over time-varying directed graphs. IEEE Transactions on Automatic Control, 60(3), pp Behrouz Touri Distributed non-convex optimization 7 / 21

19 Notations, definitions, and assumptions Graph G(t): representing who is communicating to whom at t, directed, for this talk: unweighted 2 G: Behrouz Touri Distributed non-convex optimization 8 / 21

20 Notations, definitions, and assumptions Graph G(t): representing who is communicating to whom at t, directed, for this talk: unweighted 2 G: 1 3 Adjacency matrix A(t): A ij (t) = 1 if i j in G(t) otherwise A ij (t) = A = Behrouz Touri Distributed non-convex optimization 8 / 21

21 Notations, definitions, and assumptions G = ([n], E) is strongly connected if: there exists a directed path from any node i to any node j Behrouz Touri Distributed non-convex optimization 9 / 21

22 Notations, definitions, and assumptions G = ([n], E) is strongly connected if: there exists a directed path from any node i to any node j Behrouz Touri Distributed non-convex optimization 9 / 21

23 Notations, definitions, and assumptions Reminder: objective is to minimize x R d n i=1 f i(x) Behrouz Touri Distributed non-convex optimization 10 / 21

24 Notations, definitions, and assumptions Reminder: objective is to minimize x R d n i=1 f i(x) previously assumed: local cost functions f i : R d R s are convex Behrouz Touri Distributed non-convex optimization 10 / 21

25 Notations, definitions, and assumptions Reminder: objective is to minimize x R d n i=1 f i(x) previously assumed: local cost functions f i : R d R s are convex Behrouz Touri Distributed non-convex optimization 10 / 21

26 An existing method Push-sum based method: minimize x R d n f i (x), i=1 Agent i maintains estimate x i (t) R d of an optimizer and auxiliary variables w i (t), z i (t) Behrouz Touri Distributed non-convex optimization 11 / 21

27 An existing method Push-sum based method: minimize x R d n f i (x), i=1 Agent i maintains estimate x i (t) R d of an optimizer and auxiliary variables w i (t), z i (t) Agents update rule: x i (t + 1) = 1 N + j (t) x j(t) α(t) f i (z i (t)) w(t + 1) = j N i (t) j N i (t) z i (t) = x i(t) w i (t) 1 N + j (t) w j(t) where x, z, f i (z i (t)) (R d ) n, w R n and w(0) = 1 Behrouz Touri Distributed non-convex optimization 11 / 21

28 Existing methods: method #2 Push-sum based method-main result: Proposition Suppose that 1 {G(t)} is a graph sequence such that for some B > 0, G(t : t + B) = ([n], E(t) E(t + B 1)) is strongly connected for any t 0, 2 the step-size α(t) > 0 is non-increasing with t=0 α(t) = and t=0 α2 (t) <, 3 f i s are convex with bounded gradients and the underlying distributed optimization problem has a non-empty solution set X. Then, for any initial condition x(0) (R d ) n, lim t z i (t) = x for some x X and all i [n]. Behrouz Touri Distributed non-convex optimization 12 / 21

29 Existing methods: why it works? Agents update rule: x(t + 1) = A T (t)x(t) α(t) f z(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t) Behrouz Touri Distributed non-convex optimization 13 / 21

30 Existing methods: why it works? Agents update rule: Why it works? x(t + 1) = A T (t)x(t) α(t) f z(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t) Behrouz Touri Distributed non-convex optimization 13 / 21

31 Existing methods: why it works? Agents update rule: x(t + 1) = A T (t)x(t) α(t) f z(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t) Why it works? - average observer: under the connectivity assumptions, in long run z i(t) x(t) for all i Behrouz Touri Distributed non-convex optimization 13 / 21

32 Existing methods: why it works? Agents update rule: x(t + 1) = A T (t)x(t) α(t) f z(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t) Why it works? - average observer: under the connectivity assumptions, in long run z i(t) x(t) for all i Behrouz Touri Distributed non-convex optimization 13 / 21

33 Existing methods: why it works? Agents update rule: x(t + 1) = A T (t)x(t) α(t) f z(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t) Why it works? - average observer: under the connectivity assumptions, in long run z i(t) x(t) for all i - flow tracking: we have x(t + 1) = x(t) α(t) n n i=1 f i(z i(t)) x(t) α(t) n n f i( x(t)) i=1 Behrouz Touri Distributed non-convex optimization 13 / 21

34 Non-convex Distributed Optimization Convexity allows to quantify a decrease for x(t + 1) x 2 for some x X But regardless, one can think of the dynamics on x(t) as a perturbed gradient descent Behrouz Touri Distributed non-convex optimization 14 / 21

35 Main Result Still one can show that under suitable conditions: lim z i(t) x(t) = 0 for all i (push-sum protocol) t Behrouz Touri Distributed non-convex optimization 15 / 21

36 Main Result Still one can show that under suitable conditions: lim z i(t) x(t) = 0 for all i (push-sum protocol) t Theorem (Convergence to critical points) Let 1 {G(t)} is a graph sequence such that for some B > 0, G = ([n], E(t) E(t + B 1)) is strongly connected for any t 0, 2 the step-size α(t) > 0 is non-increasing with t=0 α(t) = and t=0 α2 (t) <, and 3 F (x) is bounded from bellow (existence of a global minimizer), F (x) = n i=1 f i(x) is coercive, and f i (x) is Lipschitz continuous for all i. Then, lim t x(t) = x exists and belongs to the set of critical points of F (x). Furthermore, lim t z i (t) = x. Behrouz Touri Distributed non-convex optimization 15 / 21

37 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

38 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) LV (t, x) = E[V (t + 1, x(t + 1)) x(t) = x] V (t, x). V (t, x) = V (x) = F (x) + C 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

39 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) LV (t, x) = E[V (t + 1, x(t + 1)) x(t) = x] V (t, x). V (t, x) = V (x) = F (x) + C V ( x(t)) is a nonnegative martingale 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

40 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) LV (t, x) = E[V (t + 1, x(t + 1)) x(t) = x] V (t, x). V (t, x) = V (x) = F (x) + C V ( x(t)) is a nonnegative martingale there exists a finite limit of V ( x(t)) a.s. as t 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

41 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) LV (t, x) = E[V (t + 1, x(t + 1)) x(t) = x] V (t, x). V (t, x) = V (x) = F (x) + C V ( x(t)) is a nonnegative martingale there exists a finite limit of V ( x(t)) a.s. as t Since, F is coercive, x(t) is bounded a.s. 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

42 Sketch of the Proof Idea: Stochastic Approximation Approach 1 to analyze x(t) LV (t, x) = E[V (t + 1, x(t + 1)) x(t) = x] V (t, x). V (t, x) = V (x) = F (x) + C V ( x(t)) is a nonnegative martingale there exists a finite limit of V ( x(t)) a.s. as t Since, F is coercive, x(t) is bounded a.s. α(t) and Assumptions x(t) critical point of F (boundary of a connected component) 1 M.B. Nevelson and R. Z. Khasminskii. Stochastic approximation and recursive estimation, 1973 Behrouz Touri Distributed non-convex optimization 16 / 21

43 Non-convex Distributed Optimization How to get out of the local maximas? Behrouz Touri Distributed non-convex optimization 17 / 21

44 Non-convex Distributed Optimization How to get out of the local maximas? We consider: x(t + 1) = A T (t)x(t) α(t) f z(t) + α(t)k(t) w(t + 1) = A T (t)w(t) z i (t) = x i(t) w i (t), where K(t) is a noise process with K i (t) a random i.i.d. vector zero-mean coordinates and variance one Behrouz Touri Distributed non-convex optimization 17 / 21

45 Main Result 2 With the new term, we have x(t + 1) = x(t) a(t) n n f i (z i (t)) + α(t) K(t), i=1 lim z i(t) x(t) = 0 almost surely for all i (push-sum protocol) t Behrouz Touri Distributed non-convex optimization 18 / 21

46 Main Result 2 With the new term, we have x(t + 1) = x(t) a(t) n n f i (z i (t)) + α(t) K(t), i=1 lim z i(t) x(t) = 0 almost surely for all i (push-sum protocol) t Theorem (Convergence to critical points) Let 1 {G(t)} is a graph sequence such that for some B > 0, G = ([n], E(t) E(t + B 1)) is strongly connected for any t 0, 2 the step-size α(t) > 0 is non-increasing with t=0 α(t) = and t=0 α2 (t) <, and 3 F (x) is bounded from bellow (existence of a global minimizer), F (x) = n i=1 f i(x) is coercive, and f i (x) is Lipschitz continuous for all i. Then, lim t x(t) = x is a local maxima with probability zero. Behrouz Touri Distributed non-convex optimization 18 / 21

47 Sketch of the Proof Theorem Let x be a local maxima, and D: x D. Assume there exists a function V (t, x), bounded below on x D for all t, such that: (1) LV (t, x) γ(t) for t 0 and x D, where t=1 γ(t) <, (2) lim t V (t, x(t)) =, if lim t x(t) = x. Then Pr{lim t x(t) = x} = 0 independently on x(0). V (t, x) chosen intelligently to reflect the infinite push needed to climb the local maxima. Behrouz Touri Distributed non-convex optimization 19 / 21

48 Further considerations Getting rid of saddle points? Behrouz Touri Distributed non-convex optimization 20 / 21

49 Further considerations Getting rid of saddle points? Other distributed optimization methods? Behrouz Touri Distributed non-convex optimization 20 / 21

50 Further considerations Getting rid of saddle points? Other distributed optimization methods? Rate of convergence result? Behrouz Touri Distributed non-convex optimization 20 / 21

51 Further considerations Getting rid of saddle points? Other distributed optimization methods? Rate of convergence result? Payoff based methods? Results are reported at: T. Tatarenko and BT, Non-convex distributed optimization. IEEE Transactions on Automatic Control (2017). Behrouz Touri Distributed non-convex optimization 20 / 21

52 Publications Journal: T. Tatarenko and BT, Non-convex distributed optimization. IEEE Transactions on Automatic Control (2017). BT and B. Gharesifard, Structural approach to distributed optimization, submitted. BT and B. Gharesifard, Saddle-point dynamics for distributed convex optimization on general directed graphs, submitted. BT and B. Gharesifard, Average consensus on random chains, submitted. Conference: T. Tatarenko and BT, On local analysis of distributed optimization. American Control Conference (ACC), Boston, July BT and B. Gharesifard, Continuous-time distributed convex optimization on time-varying directed networks. IEEE 54th Annual Conference on Decision and Control (CDC), Japan, Dec BT and B. Gharesifard, A Distributed Observer for Distributed Control Systems, Allerton Conference on Communication, Control, and Computing Behrouz Touri Distributed non-convex optimization 21 / 21

Random Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization

Random Walk Distributed Dual Averaging Method For Decentralized Consensus Optimization Cun Mu, Asim Kadav, Erik Kruus, Donald Goldfarb, Martin Renqiang Min Machine Learning Group, NEC Laboratories America