Getting multi-agent systems to cooperate. Luca Schenato University of Padova OptHySys 2017

Transcription

1 Getting multi-agent systems to cooperate Luca Schenato University of Padova OptHySys 2017

2 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

3 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

4 Networked Control Systems SMART GRIDS WIRELESS SENSOR NETWORKS INTELLIGENT TRAFFIC SYSTEMS Physically distributed dynamical systems interconnected by a communication network SMART BUILDINGS SWARM ROBOTICS SMART CITIES

5 Target applications: MAgIC Lab. at University of Padova Wireless Sensor Actuator Networks Smart Camera Networks Robotic Networks Smart Energy Grids

6 Joint work with Colleagues at Univ. of Padova Ruggero Carli Angelo Cenedese Alessandro Chiuso Gianluigi Pillonetto Sandro Zampieri Former/current students: International collaborators: Andrea Carron Marco Todescato Simone Del Favero Sinan Yildirim Ege Univ., Turkey Fabio Fagnani Turin Politech, Italy Lara Brinon-Arranz IST, Portugal Damiano Varagnolo Saverio Bolognani Univ. of Lulea, Swedenn MIT, USA Filippo Zanella Sellf Inc. Alexandra von Meier UC. Berkeley, USA Reza Argandeh CIEE, Berkeley, USA Kameshwar Poolla UC. Berkeley, USA

7 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

8 Challenges Unreliable (wireless) communication: Random delay, packet loss, limited communication range Scalability: Complexity (CPU, memory, communication) per agent must be constant Robustness: Mild performance degradation when local failures Architecture: Centralized vs hierarchical vs distributed vs decentralized Cooperative vs competitive

9 Challenges: a personal experience Prototyping time Leader-based/hierarchical algorithms too complex to code Debugging time Few LEDs for visual inspection Ex-post analysis of dozens of agent data logs after a failure Rapid peer-to-peer communication Wi-Fi, bluetooth, zigbee not suitable for peer-to-peer Courtesy of Antonio Franchi, CNRS, Toulouse Need for simple asynchronous peer-to-peer algorithms

10 Some working complex systems INTERNET Cell phones networks

11 A leading paradigm: ISO layers with few primitives Application layer Communication layers

12 Multi-agent systems: an ISO-like paradigm? Smart Power Grids Intelligent transportation What should be the right ISO-model? Need to seamlessly integrate: Communication network(s) Sensing and control Physical constraints (conservation mass/energy) Markets

13 ISO for multi-agent systems Application Sensor Map layer Time-synch??? calibration building Communication layer Point-to-point Broadcast Multi-cast???

14 Consensus algorithm: a primitive for cooperation Application layer Time-synch Sensor calibration Map building??? Cooperation layer Average consensus Consensus??? Communication layer Point-to-point Broadcast Multi-cast???

15 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

16 The consensus problem Main idea Having a set of agents to agree upon a certain value (usually global function) using only local information exchange (local interaction) Also known as: Agreement problem (economics, social networks) Load balancing (Computer Science & communications) Synchronization (statistical mechanics) Rendezvous and flocking (robotics) Old problem: Markov Chains (60 s), Load balancing (70 s), Distributed decision making (80 s), flocking(00 s)

17 Network of N agents Communication graph: i-th node neighbors: Multi-agent modeling Local variable: node i store

18 Recursive Distributed Algorithms DEFINITION: Recursive Distributed Algorithm consistent with the graph G: Any recursive algorithm where the i-th node s update law depends only on the local variables of i and its neighbors

19 Consensus definitions DEFINITION: A Recursive Distributed Algorithm consistent with the graph G is said to asymptotically achieve consensus if x i DEFINITION: A Recursive Distributed Algorithm consistent with the graph G is said to asymptotically achieve average consensus if x i Consensus Iteration Consensus Iteration

20 A robotics example: the rendezvous problem 2 Receiving node: 1 Other nodes: 4 3

21 A robotics example: the rendezvous problem 2 Receiving node: Other nodes:

22 A robotics example: the rendezvous problem Receiving node:

23 The linear consensus algorithm PROPERTIES OF P(t) (Stochastic Matrix) Consistent with the graph: Component-wise non-negative: Row-sum unitary: P(t) doubly stochastic if also column-sum unitary:

24 Constant matrix P Synchronous communication: At each time all nodes communicate according to the communication graph and update their local variables (Laplacian weigths)

25 Time varying P(t): broadcast Broadcast communication: At each time one node wakes up and broadcasts its information to all its neighbors

26 Time varying P(t): symmetric gossip Symmetric gossip communication: At each time one node wakes up and choses one of its neighbors. These two nodes exchange their local variables

27 Asynchronous consensus: convergence Standard Consensus (Broadcast) Graph rooted on average Self-loops, i.e. P(t) with positive diagonal P(t) row-stochastic Average Consensus (Gossip) Graph connected on average Self-loops, i.e. P(t) with positive diagonal P(t) doubly stochastic

28 Convergence for time-varying communication union

29 Asynchronous consensus: communication burden Broadcast-based Consensus Achieves consensus updates per 1 sent message No ACK message required Gossip-based Consensus Achieves average consensus 2 updates per (at least) 3 sent messages Non-trivial communication protocol

30 Average consensus: the (broadcast) ratio consensus Standard Transmitter node Receiver nodes Ratio Transmitter node Receiver nodes Other nodes: Other nodes: Row stochastic Column stochastic

31 Average consensus: the ratio consensus Standard Ratio D. Kempe, A. Dobra, and J. Gehrke, 2003 M. Alighanbari and J. How, 2008 F. Benezit, V. Blondel, P. Thiran, J. Tsitsiklis, M. Vetterli, 2010

32 Realistic scenarios Ideal scenario Collisions Packet losses

33 Packet loss: Broadcast consensus Standard Ratio Transmitter node Transmitter node Receiver nodes Receiver nodes Other nodes: Other nodes: Row stochastic Column sub-stochastic

34 Packet losses: symmetric gossip consensus Gossip nodes Other nodes Row stochastic

35 Standard Consensus (broadcast) Guaranteed (slower) convergence Average Consensus (gossip) Guaranteed (slower) convergence, but loss of average Under randomized communication: (Fagnani-Zampieri 2009, Frasca-Hendrickx 2013) Ratio Consensus (broadcast) No convergence Asynchronous consensus: packet loss and random delay Robust Ratio Consensus (broadcast) Guaranteed average consensus Additional local variables required (Dominguez Garcia-Hadjicostis-Vaidya, 2014)

36 Consensus algorithm: a primitive for multi-agent systems Application layer Time-synch Sensor calibration Distributed optimization??? Robust asynchronous broadcast-based and relatively simple implementations available Cooperation layer Average consensus Communication layer Consensus??? Point-to-point Broadcast Multi-cast???

37 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

38 Wireless Sensor Actuator Networks Consensus-based applications Smart Camera Networks Robotic Networks Sensor Calibration RF indoor tracking Clock Synchronization Cardinality estimation Perimeter patrolling Smart Energy Grids Rendez-vous Map building Localization Source-seeking Multi-area state estimation

39 Sensor calibration issues in RF-based localization Systematic calibration errors i d i k d k d j j

40 WSN sensor calibration Ideally: Estimate : Use offset: to compensate the Calibrated measurement What we propose is: All nodes overestimate or underestimate the distance similarly. The errors, in the triangulation process, cancel out partially.

41 Calibration as consensus problem Define we want Recalling: update equation Steady state

42 Experimental Testbed 25 Tmote-Sky nodes with Chipcon CC2420 RF transceiver randomly placed inside a single conference room x [m] Network topology and nodes displacement: Edge if packet loss probability <25% y [m]

43 Experimental results: Broadcast consensus Links divided into 2 categories: Training links (black) Validation links (gray) x [m] number of edges Error distribution after before ôi [dbm] y [m] 500/25=20 oscillations Prx ij Prx ji [dbm] Number of consensus iterations

44 Estimation from noisy relative measurements Synchronous implementations: Barooah 2007 Asynchronous implementations: P. Barooah and J. P. Hespanha, 2005 A. Giridhar and P. R. Kumar, 2006 N. M. Freris and A. Zouzias, 2012 C. Ravazzi, P. Frasca, H. Ishii, and R. Tempo, 2013 Asynchronous implementation robust to packet losses and random delays M. Todescato, A. Carron, R. Carli, L. Schenato, 2014

45 Clock Synchronization in WSN Low Power TDMA communication for battery powered nodes sensor node BASE STATION ON OFF Node j Node i transmission

46 Clock Synchronization: cascade consensus Hardware clocks Virtual reference clock Software clock Goal: Skew/Drift Offset

47 Clock Synchronization: cascade consensus i j Drift compensation Solis, Borkar, Kumar, 2006 Sommer, Wattenhofer, 2009 Fiorentin, Schenato 2011 Liao, Barooha 2013 Offset compensation

48 Clock Synchronization: PI consensus steady state error Hardware clocks Virtual reference clock Software clock PI consensus: Cascade consensus: Carli, Chiuso, Schenato, Zampieri 2006 Yildirim, Carli, Schenato, 2014

49 Clock Synch in WSN: experiments Actual code PI synch Sommer lines Complexity FTSP PulseSync GTSP PISync CPU Overhead (ticks) ' 5440 ' 5440 ' 5610 ' 145 Message Length (bytes) Main Memory Overhead (bytes) * N Flash Memory Requirements (bytes) x50 faster x4-20 less RAM memory

50 Clock Synch in WSN: video Courtesy of Sinan Yildirim, Ege University, Turkey

51 Map-building in robotic networks x j χ 1 0 x i ! Scenarios!!! Each robot collects local data Local communication with robot Patrolled area dynamically change x χ

52 Map-building as least-squares regression Model class: Noisy measurements: Goal: minimize sum of squares of residues Xiao-Boyd-Lall, 2005 Bolognani-Del Favero-Schenato-Varagnolo, 2010

53 Consensus-based Map-building: gossip communication j i x x χ

54 Consensus-based map-building: robust broadcast ratio consensus Courtesy of Damiano Varagnolo, University of Lulea, Sweden

55 Cooperative distributed optimization Agents cooperate to find the minimizer of the network cost: Global estimation: Each node wants a copy of the global minimizer Machine learning, map building,. Local estimation: each node just wants Calibration, localization,. : number of agents : state dimension ( )

56 On going work: Newton-Raphson Consensus Distributed optimization very popular research area: Augmented Lagrangians (ADMM) Sub-gradient methods Asynchronous and robust distributed optimization still very open and practically relevant Our recent effort in merging Newton-Raphson and consensus ideas together

57 Outline Motivations and target applications Challenges The consensus algorithm Application of consensus Conclusions and open vistas

58 Conclusions Consensus as a building block for cooperative multiagent applications Effort is in casting general problems as consensus Time-varying higher order consensus is still an open problem PI consensus (clock synch) PD consensus (fast consensus, diffusive algorithms) PID (?) Self-tuning: adaptive tuning of parameters/gains in distributed algorithms

59 Open vistas (1) Architecture: Multi-agent/complex systems still an open challenge Smart Power Grids

60 Open vistas (2) Computation: Asynchronous distributed algorithms robust to unreliable communication Parallel computing (old paradigm) Cloud computing (new paradigm)

61 Open vistas (3) Data Tsunami ( Big data): most data is timeseries. Time and causality must be treated differently than usually done in machine learning Cooperative multi-agent algorithms will be a necessity

62 Q&A URL:

63 Consensus: References (1) F. Garin, L. Schenato. A survey on distributed estimation and control applications using linear consensus algorithms. Networked Control Systems. vol. 406,pp , 2011 Sensor calibration: S. Bolognani, S. Del Favero, L. Schenato, D. Varagnolo. Consensus-based distributed sensor calibration and least-square parameter identification in WSNs. International Journal of Robust and Nonlinear Control, vol. 20(2), 2010 M. Todescato, A. Carron, R. Carli, L. Schenato. Distributed Localization from Relative Noisy Measurements: a Robust Gradient Based Approach. IEEE Conference on Decision and Control (CDC14), submitted Clock synchronization: L. Schenato, F. Fiorentin. Average TimeSynch: a consensus-based protocol for time synchronization in wireless sensor networks. Automatica, vol. 47(9), pp , 2011 K. Yildirim, R. Carli, L. Schenato. Proportional-Integral Synchronization In Wireless Sensor Networks. ACM Transactions on Sensor Networks (submitted)

64 Map Building: References (2) A. Carron, M. Todescato, R. Carli, L. Schenato, G. Pillonetto. Multi-agents adaptive estimation and coverage control using Gaussian regression. IEEE Conference on Decision and Control (CDC14), submitted Distributed optimization: D. Varagnolo, F. Zanella, A. Cenedese, G. Pillonetto, L. Schenato. Newton- Raphson Consensus for Distributed Convex Optimization. IEEE Transactions on Automatic Control (submitted) R. Carli, G. Notarstefano, L. Schenato, D. Varagnolo. Asynchronous Newton-Raphson Consensus for Robust Distributed Convex Optimization. IEEE Conference on Decision and Control (CDC14), submitted