Using peer to peer. Marco Danelutto Dept. Computer Science University of Pisa

Transcription

1 Using peer to peer Marco Danelutto Dept. Computer Science University of Pisa Master Degree (Laurea Magistrale) in Computer Science and Networking Academic Year

2 Rationale Two common paradigms in distributed computing client server peer to peer Within the SPM course we used client server like RTS to implement different flavours of farm and data parallel patterns we assumed to have some p2p technique to implement resource discovery when recruiting workers in farm and data parallel patterns We want to focus a little bit better those paradigms 2

3 Client server architecture Well know and widely used a client asks a server for some service(params) the server is identified by a well known address a servers runs a well known protocol the server answers client requests serially or concurrently (multithreaded servers) managing to keep some consistent state or in a stateless way possibly interacting with other servers to provide results of service(params) to the client 3

4 Classical server architecture while(true) { accept a client request; fork concurrent activity to serve the request; } requestservice(client, params) { process request with params; compute service result; send back result to the client; } 4

5 Server sample WEB server well known address (IP:80) well known protocol : HTTP multithreaded implementation serversocket.accept() passed to thread in a thread pool the thread retrieves the document, compute the answer and sends back page (or answer) to the requesting client DNS server NFS server 5

6 Client/Server in SPM: Farm (1) Worker server E/C client W compute(task) E/C E/C W W compute(task) W compute(task) result compute(task) result 6

7 Client/Server in SPM: Farm (2) Worker client E/C server W result E/C W W E/C result result W task result task 7

8 Client/Server salient features Asymmetric communication schema client starts interaction, server runs infinite loop, reacting to requests reflected in TCP asymmetric connection schema Asymmetric rôle server has the information / computational power, it is always on client may ask server for service, may be off most of time Centralized architecture one (logical) server (the bottleneck) usually many clients 8

9 Client/Server & SPM concepts Server is usually the bottleneck Server Farm used to solve the problem A front-end node accepts service requests And schedules requests to one of the n (replicated) servers each capable to answer the service request directly delivering answer to clients C1 Server1 Front-end Cn ServerK Server Farm 9

10 Two tier server farms First tier: several farms spread in different places DNS rounds robin through the different farms when resolving farm address Second tier: single server farm request coming to a server farm is dispatched to an available server unless service answer is in cache in the front end this is more or less an SPM farm (but for caching) 10

11 Notable sample server farm: google 11

12 Notable sample server farm: google (2) 12

13 Notable sample server farm: google (3) Server farm works with a map reduce paradigm actually this boosted structured parallel/distributed programming although: 1) it is not recognized as a skeleton by google guys & 2) it is not structured the composable way we assumed here 13

14 Peer to peer Huge phenomenum usually associate to file sharing and, unfortunately, to illegal content sharing music, movies, books 14

15 Peer to peer No specific rôle all hosts are peers Services provided through cooperation of the peers using an infrastructure allowing peer to communicate peer to peer communication no server used to mediate peer interaction (in principle) 15

16 Peer to peer features Efficiency (w.r.t. resources) Exploit unused bandwidth, storage, processing power Scalability No communication and computation bottleneck Resources added in steps Reliability Distributed replicas No single point of failure Administration Node self-organization Fault tolerance, replication, and load balancing Anonymity Privacy much better than in a centralized system Dynamism highly dynamic environment ad-hoc communication and collaboration 16

17 Typical P2P applications File sharing e.g. napster, bittorrent, gnutella,... service provided : get a copy of a file P2P VOIP phone e.g. Skype service provided : interconnecting (voice and video, cyphered) computers to computers, computers to phones 17

18 P2P application taxonomy P2P apps Parallel Content and file management Collaboration Compute intensive Compone ntized Content exchange File system Filtering, mining Instant message Shared apps Games 18

19 Peer to peer scenario(s) A number of peers discover each other and talk peer to peer to implement a sharing protocol A number of peers discover each other, possibly using one or more shared servers, then talk peer to peer to implement a sharing protocol 19

20 The initial story... Peer to peer file sharing server vs. peer to peer Server: one server contacted to get a copy of the file performance bottleneck at the peer PeerToPeer: a number of peers hosts copies of file (or chunks of the original copy) and can be queried to get (the missing) chunks distributed performance bottleneck ( no bottleneck) PeerToPeer file sharing seemed to be much more efficient than server based file sharing 20

21 Some performance models Upper limit in the two cases Server max{ N peerfile dim Server B, File dim P eer B, } Peer to peer max{ File dim Server B, File dim P eer B, N peer File dim Server B + N p eer i=1 P eer Bi } 21

22 Some performance models Upper limit in the two cases Server Server is the bottleneck max{ N peerfile dim Server B Peer is the bottleneck, File dim P eer B, } Peer to peer max{ File dim Server B, File dim P eer B, N peer File dim Server B + N p eer i=1 P eer Bi } 21

23 Performance result (theoretical) P eer B = b, File dim b =1hour, Server B = 10b (Upload peer badwidth Download peer bandwidth) 22

24 Different approaches Problem to be solved how to find location of peers with FileX First step in the file sharing process Then: second step contact peers to obtain (portions of) the file 23

25 Approach one: centralized directory Server (or server farm!!!) host index of shared files file X is on hosts H1,..., Hn <H1,IP1>,...,..., <Hh,IPh> original Napster approach Peer: asks server contact peer hosts (choose the better one...) 24

26 Centralized approach: pros and cons Pros Low peer requirements (just server name(s)) Limited bandwidth required fast host discovery Good success rate of p2p queries Cons Single point of failure No scalability 25

27 Napster Application-level, client-server protocol over point-to-point TCP Four steps Connect to Napster server Upload file list Upload query Select best peer among those answered by the server 26

28 Approach two: query flooding Completely distributed approach (used by original Gnutella) Based on overlay network concept nodes are peers, arcs between connected peers Peer looking for File X queries the peers connected in the overlay network peer with the file: answer + propagate recursively the request peer without the file: simply propagate recursively the request Variant: request with TTL: not propagated after TTL hops 27

29 Query flooding pros and cons Pros completely distributed fault tolerant no bottelnecks Cons high bandwidth used large time required to locate items no guarantee of success (see next slide) load unbalance 28

30 Problem: how to build the overlay network Peer A entering the system tries to establish a contact with at least another peer B using a server to get the address of some peers, multicasting over UPD, contacting all the IP addresses in its cache,... Once connected to B sends to B a ping message with TTL = n B propagates the message to all neighbours in overlay network, unless TTL = 0 and answers with a message to A recursively... A tries to connect to some of the answering nodes, to connect to the overlay network 29

31 Third approach: hierarchical network Mixed approach (used by Kaaza & current version of Gnutella) Peers and super peers Super peers servers with index of all the connected peers in addition: connected with other super peers Peers just connect to one super peer Peer send request to super peer if locally solved done otherwise super peer flood other super peers 30

32 Sample P2P framework JXTA (Sun) Open platform for p2p cooperation Platform independent Languages: Java, C Operating systems: Unix/Linux/BSD, Windows Networks: TCP/IP on different network layer

33 JXTA (2) Objectives Find peers and resources Share files with anyone across the network Create a particular group of peers across different networks Communicate securely with peers across public networks Protocols Peer discovery protocol Peer resolver protocol Peer information protocol Rendezvous protocol Pipe binding protocol Endpoint routing protocol 32

34 JXTA overview JXTA community applications JXTA Shell JXTA community services Sun JXTA Services Peer commands Peer Groups Peer Pipes Peer Monitoring Security JXTA CORE Peer network 33

35 P2P & SPM P2P is unstructured in some sense naturally opposed to structured approaches? Actually some structured programming issues fit the model e.g. resource discovery other issues could be solved using alternative approaches e.g. master/worker implementation schema 34

36 Discovery Source code compiler structured distributed code requires a number of resources These resources may be discovered with P2P inspired approaches Muskel discovery message flooding emulated through UDP multicast nothing prevents setting up a real query flooding procedure Same techniques may be used when recruited resources fail to recruit new resources, possibly not appearing at appl start 35

37 P2P Discovery User host runs skeleton program figures out the resources needed send a message to known hosts those available to host a worker answer and propagate request those non available simply propagate the request request TTL decreased on propagation eventually, a list of hosts available to participate is known 36

38 P2P Discovery w.r.t. implementation model Macro data flow a number of un-distinct nodes is needed each used to run a macro data flow interpreter instance Implementation template a number of nodes is needed to be specialized by means of a process template most likely some of the nodes require special features Consequences MDF slightly more suitable due to un-distinct nodes used In both cases possibly more nodes needed to substitute failed ones 37

39 Peculiar support to structured models Peer discovery dynamic process can be used in static phases (such as initial configuration) perfectly suitable to dynamic re structuring of a computation to substitute faulty nodes, to improve parallelism degree, Peers leaving systems conveniently modelled as a fault more sever FT policies required 38

40 Implementation patterns with P2P Experience (in Pisa) JXTA implementation of a skeleton framework (task farm) by Mauro Morici april 2004 Actually a master/worker schema implementing MDF Peers discovered through JXTA mechanisms specialized to run workers on demand each time a task comes to be executed if idle worker exist assign task otherwise lookup new peer 39

41 JXTA/Muskel : Emitter structure 40

42 JXTA/Muskel : interactions 41

43 JXTA/Muskel : results 42

44 JXTA/Muskel : results 43

45 Sample code 44