11/7/2011. Networked embedded systems. The Vision for WSANs. Embedded systems

Transcription

1 Networked embedded systems Principles of distributed computing for design of scalable and robust sensor actuator networks Vinod Kulathumani Dept. of Computer Science and Electrical Engineering West Virginia University Currently Embedded processors -part of a larger system Application known apriori Little flexibility in programming What if? embedded processors were connected preferably wireless? there was greater flexibility in programming? sensing and actuation capabilities were included? That s the vision for Sensor Actuator Networks Networked Embedded Systems Laboratory - Fundamentally a network of embedded systems Embedded systems The Vision for WSANs Found in variety of devices Aircraft, radar systems, nuclear and chemical plants Vehicles, TVs, camcorders, elevators > 90% of CPUs used for embedded devices Combine wireless networks with sensing / actuation Ubiquitous computing / pervasive computing Fine-grained monitoring and control of environment Network and interact with billions of embedded computers Reasons Wireless communication -no need for infrastructure setup Drop and play Nodes are built using off-the-shelf cheap components Feasible to deploy nodes densely 1

2 Enabling technology Emerging applications Powerful microprocessors Small form factor Low energy consumption Micro-sensors (MEMS, Materials, Circuits acceleration, vibration, gyroscope, tilt, motion magnetic, heat, pressure, temp, light, moisture, humidity, barometric chemical (CO, CO 2, radon, biological, micro-radar actuators (mirrors, motors, smart surfaces, micro-robots Combination of sensors with mobile devices Social networking Participatory urban sensing Assisted living health monitoring Communication short range, low bit-rate, CMOS radios Vehicular networks with variety of sensors Application category Monitoring type Challenges in monitoring based sensor networks Energy constraint : Nodes are battery powered Unreliable communication : Wireless, limited bandwidth Environmental monitoring Object tracking Unreliable sensors : False positives, negatives Ad hoc deployment : Pre-configuration inapplicable Infrastructure monitoring Body sensor networks Large scale networks : Algorithms should scale well Distributed execution : Difficult to debug & get it right Perimeter security Camera sensor networks Ease of use : All Scientists not programmers 2

3 Sensor networks for control applications Not simply monitoring events, objects Combined with actuation Traditional control applications Decouple information availability Control assumes information is instantaneously available What if information is transmitted over a sensor network? Losses, delays in information New tools needed for programming, reasoning about such systems Building blocks for Cyber-physical systems - recent buzzword! Example sensor actuator networks Robotic systems Self-configuring structures Robotic surgery Self-configuring table Autonomic vehicular platoons Use in UAV swarms Autonomous driving Google Car! Distributed vibration control Distributed illumination control, irrigation, process control Smart power grid Sensor networks for control applications We saw all these challenges for sensor networks Not simply monitoring events, objects Combined with actuation Traditional control applications Note Decouple information availability Applying control theory for network Control assumes information is instantaneously available systems has existed before (example: TCP congestion What if information is transmitted over a sensor network? Losses, delays in information This is control systems designed on top of networks New tools needed for programming, reasoning about such systems Building blocks for Cyber-physical systems - recent buzzword! Energy constraint Unreliable communication Unreliable sensors Ad hoc deployment Large scale networks Distributed execution Ease of use : Nodes are battery powered : Wireless, limited bandwidth, bursty traffic : False positives, negatives : Pre-configuration inapplicable : Algorithms should scale well : Difficult to debug & get it right : All Scientists not programmers 3

4 Add to these... Energy constraint Unreliable communication : Nodes are battery powered : Wireless, limited bandwidth, bursty traffic Unreliable sensors : False positives, negatives. A control application that sits on top Ad hoc Requires deployment information : guarantees Pre-configuration from network inapplicable below! 1. Fault-tolerant, Self stabilizing network services Large scale networks Distributed execution Ease of use : Algorithms should scale well : Difficult to debug & get it right : All Scientists not programmers Role of middleware Why self-stabilization Application Middleware Network Specifications: Control error Convergence time Scale: node systems Distributed computing services that bridge the gap Broadcast nature: Interference Collisions Resource-constrained Faults will happen Messages will be lost Nodes may fail Variables may be corrupted Nodes restarting in arbitrary state Low on battery arbitrary behavior! Mobility nodes move around Once a fault stops system should recover Return to a good state And stay in good states Common examples Discovering alternate routes Electing alternate aggregating nodes 4

5 Why self-stabilization Reasoning Faults will happen Messages will be lost Note Nodes may fail Variables may This be corrupted is different than masking fault-tolerance Nodes restarting in arbitrary state Low on battery In masking, arbitrary up behavior! to a certain number of faults can be Mobility nodes handled move without aroundfalling out of correct state Example: have redundant routes always Once a fault stops system should recover Return to a good More state expensive And stay in good states Common examples Discovering alternate routes Electing alternate aggregating nodes Fault model A set of faults that can lead to program moving out of invariant states Example, node fault, network faults Self-stabilization Show that irrespective of initial condition, protocol converges to invariant If no more faults, protocol stays within invariant Invariant states Faulty states Fixed point Reasoning about self-stabilization Invariant A state predicate that continues to hold when the actions are executed in any process in any order Proving safety Find an invariant that satisfies correctness and show that it holds for the protocol Fault-local self-stabilization A self-stabilizing program is fault-local self-stabilizing if the time and number of messages bounded by perturbation size Not dependent on the network size. Biological analogy Blood clots around a wound Fixed point A predicate that belongs to invariant and is a terminating condition [no more actions are enabled] Proving progress Show that program eventually terminates Sometimes shown using a variant function 5

6 Simple example: leader election Self-stabilizing leader election Given a set of moteswithin a 2 hop neighborhood, write a distributed program which ensures that a unique leader is appointed. The program should stabilize when nodes (including the leader are added or removed. Programs that use node ids are not recommended as they will re-elect a leader every time a new node is added. One hop network: All nodes within hearing range of each other Two hop network: Diameter = 2 At least one node common to range of any two nodes Leader election protocol [2 hops] Process j Variables: Status [either idle,candidate, leader, follower] Cluster_id [id of leader] Actions Timeout I(j.idle -> j.status = candidate; bcast [cand_msg(j] Timeout II((j.follower or j.leader ^ recv[cand_msg(i] -> bcast[conflict_msg(j, j.cluster_id] Timeout III(j.candidate -> j.status=leader; j.cluster_id=j; bcast[leader_msg(j] (J.idle or J.candidate ^ recv[conflic_msg(i, m] -> J.status=follower; j.cluster_id = m; (J.idle or J.candidate ^ recv[leader_msg(i] -> J.status = follower; j.cluster_id = i; Solution strategy Example 2: Solid-disc clustering Exploit Atomic broadcast property of wireless network Simultaneous reception if successful Randomization and CSMA 4 states at each node Idle, candidate, leader, follower Discussion Why clustering? Act as information aggregators Act as distributed controllers Solid-disc clustering: All nodes within a unit distance of the clusterhead belong only to that cluster All clusters have a non-overlapping unit radius solid-disc Why? Reduces intra-cluster signal contention clusterheadis shielded at all sides with members, does not have to endure over-hearing nodes from other clusters Yields better spatial coverage with clusters aggregation at clusterhead is more meaningful since it is median of the cluster Results in a guaranteed upper bound on the number of clusters 6

7 Requirements How to handle? Solid-disc property Self-stabilizing to addition and deletion of nodes Handles node mobility No global propagation of re-clustering : local stabilization Not dependent on network size Relax clustering requirements a unique node is designated as a leader of each cluster all nodes in the i-band of each leader belong to that cluster nodes within o-band (= m*i-band radius may belong to a cluster each node belongs to a cluster no node belongs to multiple clusters Form clusters in O(1 time Not dependent on network size No assumptions of a starting node or a starting configuration Program converges from any state Challenges for solid-disc clustering Justification for stretch factor > 2 ( ( ( ( ( ( ( new node subsumed cascading ( ( ( ( A B new node For m 2 local healing is achieved: a new node is either subsumed by one of the existing clusters, or allowed to form its own cluster without disturbing neighboring clusters Equi-radiussolid-disc clustering with bounded overlaps is not achievable in a distributed and local manner ( ( ( ( ( ( ( ( ( new cluster 7

8 Key idea Nodes wait for random time and announce candidacy Candidate nodes receive conflict notification if another clusterhead exists in solid-disc Else they lock all nodes in solid disc into their cluster 2. Consistency of distributed control Program shown to locally stabilize from arbitrary states Even if solid-disc cluster property is violated for any reason Stabilization in O(1 time Sample clustering with FLOC Problem 1: serializability Actuator: camera, heat source, light source etc. C1 C2 Controller Sensor: heat, light, camera. Sensor / actuator pairs distributed across an area Controllers C1 and C2 have overlapping set of actuators Can cause read/write conflicts during concurrent transactions Inconsistency example: C1 reads current state; C2 reads current state; C1 updates; C2 updates 8

9 Solution atomicity enforcement Challenges in wireless networks Distributed mutual exclusion [Ricart Agarwala] Controllers first acquire locks from other controllers with shared actuators Requests ordered by timestamps [Logical clocks] plus priorities for ties Proceed only if all locks received Ordering ensures progress, fairness and prevents deadlocks Issues Synchronization may not be possible Strict ordering will reduce allowable parallelization Arrows indicate shared resources 4 1,4,5 can go in parallel But 4,5 may be blocked by 2,3 respectively Well-known that message reliability is a requirement for consensus Recall attacking generals problem But wireless messages likely to be lost upon interferences and occasionally by link failures May assume eventual delivery of messages But what commit strategies are most efficient? 2 PC with timeout or 3 PC Time out commit or timeout abort Use explicit acks(interference prone or only negative acks Problem 2: Atomic commit Controller initiates updates to actuator set A Ensure atomicity All commit Or all revert [If committed, cannot revert] 2 phase commit Controller requests Actuators acknowledge If all acks received, commit confirmed - send commit message Issues: coordinator dies before issuing commit, Some receive commit message, others don t and controller fails 3 phase commit Include a prepare-to-commit phase with a timeout recovery If no commit received revert, else commit If any one node commits, it can force others to commit even if controller fails Here is one strategy C1 C2 C1 broadcasts requests, atomically heard by all neighbors Shared actuators detect and report conflict when C2 requests At least one controller makes progress Abort is time triggered at all nodes If controller receives no conflict, it sends commit to all nodes If no commit received, actuators send inquiry If even one actuator commits, can assist others in committing Optimistic protocol for simultaneously enforcing serializabilityand atomicity! 9

10 Research questions Acksor nacks Acks have interference issue With Nacks Is there really no conflict? Or was the message not delivered? Upon timeout commit or abort Commit saves a message But if commit was wrong, abort is impossible How to choose timeout? Impact in multi-hops network Prolonged proposal and rejection phase? Conclusions Sensor actuator networks Large scale sensing combined with actuation Building blocks for cyber physical systems Can form ubiquitous intelligent systems Key distributed computing principles for sensor actuator networks Self-stabilizing, robust network programs Local self-healing Conflict serializability Distributed mutual exclusion Atomicity enforcement Exploit locality in application design Joint design of control and network Control system designed with network capabilities in mind Not greedy demands 10