Opportunistic Application Flows in Sensor-based Pervasive Environments

Opportunistic Application Flows in Sensor-based Pervasive Environments Nanyan Jiang, Cristina Schmidt, Vincent Matossian, and Manish Parashar ICPS 2004 1

Outline Introduction to pervasive sensor-based applications Meteor : A content-based middleware for decoupled interactions Content-based routing infrastructure Programming model for decoupled interactions Opportunistic application flows Implementation and evaluation July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 2/30

Motivation Growing ubiquity of sensor/actuator devices with embedded communication and computation capabilities Emergence of pervasive applications Sensors, actuators, services, resources interact and collaborate to satisfy application goals Examples: Fire management applications Emergency medical care July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 3/30

Sensor-based Pervasive Environments Ad hoc structures/behaviors Interactions form and stop in an ad hoc manner Dynamic Peers join and leave at any given time Unreliable No delivery or routing guarantees Loosely-consistent Global knowledge bound by scale of the system Heterogeneous Rich variety of platforms, systems, and protocols July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 4/30

ORBIT: An Experimental Wireless Network Testbed July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 5/30

Enabling pervasive applications Pervasive applications Ad hoc interactions between sensors and actuators Bridge wired and wireless networks Large, distributed, complex, heterogeneous, dynamic Need for Middleware architecture Scalable and self-managing Provide content-level addressing Support asynchronous and decoupled interactions Autonomic Programming model (self-*) Defines interaction & coordination paradigm Achieve global behavior without the need for global knowledge July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 6/30

Project AutoMate: Software Stack NeTS Applications (Autonomic Living, Ad hoc Control) Coordinated Flows Security: Meteor Middleware Stack Ontology, Taxonomy Ad Hoc Routing Self Configuration Autonomic Elements, Emergent Flows/Opportunistic Interactions Content Routing and Discovery, Associative Messaging Self Organizing Content Overlay Wireless/Wired Infrastructure Authorization Authentication Trust Programming Model Orbit Testbed July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 7/30

Meteor - Content-based Middleware Self-organizing overlay Chord P2P overlay, each peer is a rendezvous point Content-based routing Squid AR Messaging Profile Manager Matching engine Pervasive Application Associative Associative Rendezvous Rendezvous Messaging Messaging Content-based Content-based Routing, Routing, Discovery Discovery Self-organizing Self-organizing Overlay Overlay Wireless/Wired substrate e.g. sensors, actuators, internet Meteor Stack July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 8/30

Self-organizing Overlay (Chord) Pervasive Application Associative Rendezvous Messaging Associative Rendezvous Messaging Content-based Routing, Discovery Content-based Routing, Discovery Self-organizing Overlay Self-organizing Overlay Meteor Stack A self-organizing P2P ring overlay Nodes and data have unique identifiers (keys), from a circular key space (0 to 2 m ) Each node maintains a routing table, called finger table A key is stored at the first node whose identifier is greater or equal than the key The request is routed to the neighbor node closer to the destination Routes in O(log n) hops Finger = the successor of (this node id + 2 i-1 ) mod 2 m, 0 <= i <= m Routing from node 1 to node 6 0 + 1 1 0 + 2 3 0 + 4 5 6 7 5 4 0 1 3 2 1 + 1 3 1 + 2 3 1 + 4 5 3 + 1 5 3 + 2 5 3 + 4 0 Wireless/Wired substrate July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 9/30

Content-based Routing (Squid) Pervasive Application Associative Rendezvous Messaging Associative Rendezvous Messaging Content-based Routing, Discovery Content-based Routing, Discovery Self-organizing Overlay Self-organizing Overlay Meteor Stack Wireless/Wired substrate Chord can route based on unique data identifiers only Squid uses a dimension-reducing indexing scheme Can route based on keywords, partial keywords, wildcards and ranges Uses the Space-Filling Curves for mapping keywords to identifiers Squid offers guarantees: all destinations matched by the keywords will be identified July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 10/30

Hilbert Space-Filling Curve (SFC) f: N d N, recursive generation Properties: 01 10 00 11 0 1 00 01 10 11 Digital causality Locality preserving Clustering 1 0 0101 0110 1000 1001 0111 0100 1010 1011 1100 0011 0010 1101 1110 0000 0001 1111 11 10 01 00 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 11/30

Squid Definitions Keyword tuple (used to specify the destination) List of d keywords, wildcards and/or ranges Example: (temperature, celsius), (temp*, *), (*, 10, 20-25) Simple keyword tuple Contains only complete keywords Complex keyword tuple Contains wildcards and/or ranges Units Celsius Simple keyword tuple Sensor type Longitude Complex keyword tuple 20 25 Temperature Sensor type 10 Latitude July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 12/30

Squid Content-based Routing (1) Using simple keyword tuples Routing to a single destination equivalent to Chord lookup Longitude Simple keyword tuple Longitude 0 51 7 13 1 7 40 2 Latitude SFC index Latitude 29 Map the point (2, 1) to index 7 using the Hilbert Space Filling Curve (SFC). Route to node 13 (the successor of the index 7) July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 13/30

Squid Content-based Routing (2) Using complex keyword tuples Routing to multiple destinations the straightforward solution Longitude 5 1 28 31 11 6 Complex keyword tuple Route to the nodes that store the clusters 40 51 32 0 13 Destination nodes 2 3 Latitude Translate the query to relevant clusters on the SFC-based index space July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 14/30

Squid: Experimental Evaluation (1) Simulation Up to 5000 nodes, and up to 10 6 keyword tuples Metrics: Number of processing nodes Number of data nodes Keyword tuples: Keyword tuples with wildcards Keyword tuples with ranges Wildcard tuples Range tuples System size increases from 1000 to 5000 nodes, keys from 2*10 5 to 10 6 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 15/30

Squid: Experimental Evaluation (2) 0.12 Percentage of nodes 0.1 0.08 0.06 0.04 0.02 0 1000 2000 3000 4000 5000 Number of nodes (system size) Wildcard keyword tuples: fraction of nodes that process the query Wildcard keyword tuples: fraction of nodes that found matches Range keyword tuples: fraction of nodes that process the query Range keyword tuples: fraction of nodes that found matches July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 16/30

Associative Rendezvous (AR) Pervasive Application Associative Rendezvous Messaging Associative Rendezvous Messaging Content-based Routing, Discovery Content-based Routing, Discovery Self-organizing Overlay Self-organizing Overlay Meteor Stack Wireless/Wired substrate Content-based decoupled interactions: All interactions are based on content, rather than names or addresses The participants (e.g. senders and receivers) communicate through an intermediary, the rendezvous point The communication is asynchronous. The participants can be decoupled both in space and time. Programmable reactive behaviors: The reactive behaviors at the rendezvous points encapsulated within message => flexibility, expressivity, and multiple interaction semantics July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 17/30

Associative Rendezvous: Interaction Model Messages: (header, action, data) Symmetric post primitive Associative selection Reactive behavior Header Action [Data] profile credentials message context TTL (time to live) store retrieve notify delete Profile = list of (attribute, value) pairs: Example: <(sensor_type, temperature), (latitude, 10), (longitude, 20)> AR message post (header, action, data) Profile Manager Matching Engine Action Dispatcher profile action data action data execute action Associative Selection Reactive Behavior July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 18/30

Associative Rendezvous: An Illustrative Example AR: Associative Rendezvous post (<p 1, p 2 >, notify_interest (C1)) C1 notify_interest(c1) post(<p 1, *>, notify_data(c2) ) C2 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 19/30

Associative Rendezvous: An Illustrative Example AR: Associative Rendezvous post (<p 1, p 2 >, store, data) <p 1, *> notify_data(c2) C1 notify(c2) C2 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 20/30

Associative Rendezvous: An Illustrative Example AR: Associative Rendezvous <p 1, p 2 > store data C1 retrieve(c2, data) post(<p 1, *>, retrieve (C2)) C2 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 21/30

Associative Rendezvous: An Illustrative Example AR: Associative Rendezvous post (<p 1, p 2 >, delete_data (C1)) <p 1, p 2 > store data <p 1, *> retrieve (C2) C1 post(<p 1, *>, delete_interest (C2)) C2 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 22/30

Cascading Local Behaviors (CLB) Associative Rendezvous (AR) Cascading Local Behaviors (CLB) An abstraction for content-based decoupled interaction with programmable reactive behaviors A programming model, where the behaviors of individual application elements are locally defined in terms of local state, and context and content events, and result in data and interest messages being produced. Opportunistic Flows Application flows emerge as a consequence of the cascading effect of local behaviors, without having to be explicitly programmed July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 23/30

Cascading Local Behaviors Example T Temperature Sensor if (temp > 80) then publish temp Th Thermostat if (temp > 85) then temp_control = on; publish temp_control W Window Actuator if (temp_control == on) then close windows post (temp = 91, store) T Meteor post (temp > 85, notify_data) Th notify (thermostat) Th turn temp_control on Th post (temp_control, store) Meteor post (temp_control, retrieve_data) W retrieve (temp_control) W Time if (temp_control = on) then Close window July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 24/30

Meteor: Implementation Overview Current implementation builds on JXTA Chord, Squid and AR Messaging layers are implemented as event-driven JXTA services Each layer exposes an API to the upper layer Chord uses JXTA discovery mechanism to find other nodes already in the group JXTA resolver service to send messages between peers July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 25/30

Meteor: Experimental Evaluation (1) Current Deployment 64 node Linux cluster, 1.6 GHz Pentium IV, each node running at least one peer (rendezvous node) 100 Mbps Ethernet interconnection network Experimental evaluation: Chord lookup overhead as a function of system size Content-based routing overhead Associative Messaging overhead at each RP node: querying the database associative selection constructing the notification messages reactive behavior July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 26/30

Meteor: Experimental Evaluation (2) Time (milliseconds) 80 75 70 65 60 55 50 45 40 2 16 32 48 64 80 96 Number of Peers Overlay network lookup overhead (Chord) Time (milliseconds) 500 400 300 200 100 0 16 32 48 64 80 96 Wildcard queries Range queries Wildcard and range queries Content-based routing overhead (Squid) Number of Peers July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 27/30

Meteor: Experimental Evaluation (3) 1600 1400 Time (milliseconds) 1200 1000 800 600 400 200 Matching overhead at a single RP 0 0.1 0.5 1 5 10 20 30 Number of matches 100 profiles 1000 profiles 10000 profiles 10000 profiles Number of profiles stored locally July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 28/30

Conclusions Content-based, decoupled programming model is suited to address the challenges of pervasive applications A P2P infrastructure is a natural solution to implement the Associative Rendezvous abstraction JXTA is a convenient platform to develop decoupled, content-based middleware July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 29/30

More Information Meteor project page: http://www.caip.rutgers.edu/tassl/projects/meteor/ Squid project page: http://www.caip.rutgers.edu/~cristins/research.html Recent report: http://www.jxta.org/universities/rutgers.html July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 30/30

Appendix July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 31/30

Load Balancing (LB) Why Necessary? The node identifiers (Chord) are uniformly distributed in the identifier space The data to be stored in the system is not uniformly distributed in the d-dimensional space The SFC preserves locality => The SFC index is not uniformly distributed, and we need load balancing July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 32/30

Load Balancing - Discussion Assume that all nodes have the same storage and computational capabilities The LB algorithms used balance the storage load, not the computational one Future directions: Better LB algorithms, that take into consideration The nodes heterogeneity The hot-spots some nodes will store more popular information, and they will have lots of requests July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 33/30

Load balancing Algorithms Load balancing at node join: generate more than one ID for the new node, send join requests in the network and join with the ID that places the node in the most crowded part of the network Load balancing at runtime: run a local load balancing algorithm between neighbors (from time to time), and redistribute the load use virtual nodes that can migrate to less loaded physical nodes July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 34/30

Load balancing Simulation Results Number of keys 2100 1800 1500 1200 900 600 300 0 1 501 1001 1501 2001 2501 3001 3501 4001 4501 The distribution of the keys in the index space. The index space was partitioned into 5000 intervals. The Y-axis represents the number of keys per interval. The index space (intervals) Number of keys 2100 1800 1500 1200 900 600 300 0 1 501 1001 1501 2001 2501 3001 3501 4001 4501 Nodes in the system Number of keys 2100 1800 1500 1200 900 600 300 0 1 501 1001 1501 2001 2501 3001 3501 4001 4501 Nodes in the system The distribution of the keys when using only the load balancing at node join technique. The distribution of the keys when using both the load balancing at node join technique, and the local load balancing. July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 35/30

Squid: Routing Optimization More than one cluster are typically stored at a node Not all clusters that are generated for a query exist in the network => optimize! SFC generation recursive => clusters generation is recursive => the process of cluster generation can be viewed as a tree Optimization: embed the tree into the overlay, and prune nodes during the construction phase July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 36/30

Squid: Routing Optimization - Example Binary complex keyword tuple (011, *) Longitude 1 01 10 11 10 0101 0110 1001 1010 0100 1011 0111 1000 111 110 101 100 (011, *) 0 00 0 11 Latitude 1 01 00 0011 0010 1101 1100 0000 0001 1110 1111 00 01 10 11 011 010 001 000 000 001 010 011 100 101 110 111 00 000000 0 111000 0 01 0001 000100 00, 01 100001 0001, 0010 0110, 0111 011 001111 001001 000101, 000110 001001, 001010 011111 011010, 011011, 011100 July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 37/30

Squid vs. Existing P2P Systems P2P System Pros & Cons Examples Unstructured Pro: overlay easy to maintain, supports complex queries Con: no search or cost guarantees Gnutella like systems Hybrid Data-lookup Structured Keyword Search Pro: supports complex queries Con: not scalable Pro: efficient lookup with guarantees Con: - complex queries not supported - high structure overlay maintenance cost Pro: supports more complex queries (e.g. keywords, SFC-based systems support even partial keywords, wildcards and ranges) Con: high structure overlay maintenance cost Napster, Morpheus Chord, CAN, Tapestry, Pastry Inverted Indices, PeerSearch, SFC-based systems (HP Labs, Squid) July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 38/30

Implementation Overview (cont d) Squid uses Chord s lookup primitive, to route clusters to their destination Squid s API consists of one primitive: post (AR_Message) routes based on keywords extracted from the message s profile AR Messaging Layer API consists of a single function: post (header, action, data) It currently uses a MySQL database as storage for profiles, and as a matching engine July 22nd 2004 Opportunistic flows in pervasive environments ICPS 2004 39/30