FROM PEER TO PEER...

Transcription

1 FROM PEER TO PEER... Dipartimento di Informatica, Università degli Studi di Pisa HPC LAB, ISTI CNR Pisa in collaboration with: Alessandro Lulli, Emanuele Carlini, Massimo Coppola, Patrizio Dazzi 2 nd HPC WORKSHOP, June 16 th

2 OUTLINE This presentation follows the line of the previous one: Where we came from? The next presentation: Where are we going? The past: a deep experience in defining complex P2P overlays, in this talk we will see overlays for multidimensional query resolution gossip-based overlays for Distributed Virtual Environments applying gossip-based community detection (Group, Affinity Propagation) overlays integrating P2P and clouds The future: applying our experience to distributed big data processing 2

3 OVERLAYS FOR MULTIDIMENSIONAL QUERY RESOLUTION the definition of a distributed query support for multidimensional range query has been widely studied in the past decade from the P2P community however, this service is required for several next generation distributed platform Big cloud-data centres Cyber-physical networks Distributed sensor networks Several proposals in the literature most based on extending a DHT support: based on delegation mapping each attribute of a multidimensional query separately on the DHT exploit locality preserving functions alternative approaches based on the definition of distributed tree structures 3

4 OVERLAYS FOR MULTIDIMENSIONAL QUERY RESOLUTION no solution proposed in the literature is fully satisfactory delegation implies large overhead in presence of dynamic data due to the publisher constantly updating the delegated copy tree-based proposals often require complex distributed protocols to maintain tree properties our proposal: DRAGON: a Distributed RAnge aggregation tree a light weight protocol for building distributed trees no delegation: objects published by peer P are stored on P support multi-attribute range queries multiple aggregation techniques tuning the precision of the query resolution with respect to the complexity of the aggregation techniques 4

5 A DRAGON TREE each peer manages exactly a leaf node, containing the data it publishes. it may in addition manage other internal nodes of the tree. an underlying DHT is exploited only to obtain uniform distribution of tree nodes to peers. a straightforward procedure for joining the tree different strategies to resolve contention on internal tree nodes 5

6 SUPPORTING MULTIDIMENSIONALITY space filling curves: bijective mapping between the multidimensional space of objects 2-dimension Z-curve a linear space of derived keys well known technique in the field of data base query processing In DRAGON compute the Z-derived key for multidimensional objects published by peers aggregate derived-key intervals in internal nodes of the tree intersect the query with the aggregate intervals 6

7 DRAGON AGGREGATION TECHNIQUES Each internal node stores a digest containing a summary of the keys stored in the subtree rooted at that node the digest guides the search of query matches in the tree different digest strategies: Bitvectors Bloom Filters Q-Digest Wavelet Tuning the complexity of the aggregation technique, with a larger precision in query resolution. 7

8 QUERY RESOLUTION Query resolution main design choices: avoid reaching the upper levels of the tree explore the parts of the tree that contain data values relevant to the query with higher probability Digest information at internal nodes is exploited to forward queries to a promising sub-tree only once a sub-tree is visited, the query climbs to the upper levels of the tree. 8

9 COMPARISON WITH MATCHTREE 9

10 GOSSIP-BASED OVERLAYS FOR DISTRIBUTED VIRTUAL ENVIRONMENTS Distributed Virtual Environments: Synthetic worlds where user can interact with other geographically distributed users through an alter-ego (avatar) The killer application: Massively Multi-Player Online Games (MMOGs). Large Scale Online Games are networked applications maintained by a (set of) server players are connected via clients MMOG: involve a large number of players synchronous, persistent virtual world interactive: time matters! 10

11 CURRENT PLATFORMS FOT MMOGS Server Clusters buy your own infrastructure to run a MMOG full control over the architecture They do not scale in cost, must be sized on the peak load: resources over-provisioning hardware, cooling, electricity Computational model: Clients: Send input to the server and maintain a partial replica of the game state Servers: Maintain the full replica of the game state, resolve conflicts and update clients Laura Ricci 11

12 MMORG CURRENT PLATFORMS Current Platforms: Cloud-based Pay-per-Use model No over-provisioning and full control over the architecture But... needs provisioning strategy can still be expensive for large instances: renting, bandwidth, storage Research proposal based on P2P architectures: player machines perform as servers and clients at the same time eliminates the costs, is inherently scalable But... hardly scales in QoS peer unreliability overhead for synchronization unpredictable latencies no trusted server for player authentication 12

13 OUR PROPOSAL: INTEGRATING P2P AND CLOUDS A proposal from PhD thesis of E. Carlini Functional partitioning two key components: State Action Manager (SAM) manages the state of the objects Positional Action Manager (PAM) manages the positions of the entities Each component employs a different combination of Cloud and P2P 13

14 OUR PROPOSAL: INTEGRATING P2P AND CLOUDS Positional Action Manager defines two services a backbone server, the PAM server a fully decentralized network, the PAM overlay the client periodically communicates with the server for state updates periodically queries the overlay to learn about the entities in its AOI. This reduces the number of communications with the server 14

15 THE PAM GOSSIP BASED OVERLAY Avatars have spatial relationship in the MMOG 15

16 THE PAM GOSSIP BASED OVERLAY Avatars have spatial relationship in the MMOG Clients maintain connections (overlay) according to these relationships Exchange position of nearby entities 16

17 THE PAM GOSSIP BASED OVERLAY Avatars have spatial relationship in the MMOG Clients maintain connections (overlay) according to these relationships Exchange position of nearby Entities Clients periodically communicate with the server to: input their position obtain a fresh update 17

18 THE PAM GOSSIP BASED OVERLAY Avatars have spatial relationship in the MMOG Clients maintain connections (overlay) according to these relationships Exchange position of nearby entities Clients communicate with the server to: input their position obtain a fresh update How to maintain the overlay: gossip peer sampling + coverage sampling 18

19 THE PAM GOSSIP BASED OVERLAY Defining the overlay for the dissemination of the entities position Exploit the concept of Area Of Interest: Visual/Interaction circular area centred on the avatar Each peer locally maintains neighbours that covers its AOI with their AOIs Peer Selection quadrant selection View Exchange max_coverage(receiver, myview); View Retention max_coverage(myself, myview + newview) 19

20 THE PAM GOSSIP BASED OVERLAY AOI Coverage: given a set N = {N 1...N n } of AOIs and an AOI P N, AOI-coverage(P,N) is the area of P overlapping with the areas in N goal: given a peer P, find a set of neighbours N such that AOI-coverage(P,N) is maximal several heuristics: greedy score based: favours the peer that guarantee exclusive coverage of some part of the AOI 20

21 THE PAM GOSSIP BASED OVERLAY 21

22 CONCLUSIONS We have investigated the definition of several techniques for building distributed overlays in the last years Tree-based overlays for multidimensional range query: ongoing work analyse different aggregation techniques define strategies to balance the load of peers for nodes management Gossip-based overlays for DVE: ongoing work integration with the GROUP protocol to detect peers communities integration with Affinity Propagation development of a CAN-based DHT to implement spatial queries Distributed graph processing: throwing the ball to Emanuele! 22