Department of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing
|
|
- Mercy Andrea Curtis
- 5 years ago
- Views:
Transcription
1 Department of Computer Science Institute for System Architecture, Chair for Computer Networks File Sharing
2 What is file sharing? File sharing is the practice of making files available for other users to download over the internet and smaller networks Content: typically music, videos or software File-sharing utilities: Conventional networks: FTP, HTTP P2P-networks: Napster, Gnutella, Emule/Kademlia, BitTorrent, Mute, Freenet, Gnu-Net File sharing has made P2P popular Due to their scalability requirements, file sharing systems are highly interesting systems for educational purposes 2
3 Why does file sharing make P2P networking so popular? Conventional networks offer rather cost-intensive central mass store solutions The P2P model enables a decentralized data management Peers can work as clients and as servers Users can share their files with hundreds of thousands of users and access hundreds of millions of files Files can be accessed directly from the local hard disc of other peers Overlay structure End host / Peer IP router IP link Overlay link 3
4 Outline Generations of P2P file sharing networks. First generation: Centralized peer-to-peer Decentralized P2P 2. Second generation: Hybrid P2P 3. Third generation: Distributed Hash Tables based on Kademlia BitTorrent 4
5 First generation - Centralized P2P Central directory server is the most important communication entity It makes lists of files with their associated peers available Decentralized file transfer Central directory server. Peers register at central server, publish their IP address and a list of files to be shared 2. Peers send queries to central server, server returns a list of peers with the requested objects After pinging the selected peer the file transfer happens directly between peers Example: Napster 5
6 Strengths and Weaknesses - Centralized P2P Strengths: Consistent view of the network Central server always knows who is available and who is not Fast and efficient searching in small networks Central server makes all files offered by peers available to other peers Answer is guaranteed to be correct Weaknesses: Single point of failure If the central server crashes then the whole P2P network crashes Performance Bottleneck In a P2P network with hundreds of thousands of connected users, the central server needs enough computation power to handle thousands of queries per second -> Not scalable 6
7 First generation Decentralized P2P (/2) The decentralized network has only peers Peers called servents (server+client) CONNECT OK Joining the network via Bootstrapping Contact out-of-band channel, e.g. get the member IP address from a website All subsequent access operations can use the host cache The communication establishment happens via a connect message, if the participating peer accepts this connection request it answers with the OK message Example: Gnutella.4 7
8 First generation Decentralized P2P (2/2) Discovering new peers Sending a broadcast ping Connected peers answer with pong Locating specific content Query messages are sent to all neighbours Query messages contain TTL (time to life) TTL counter is decreased by hop TTL=2 Example: Gnutella.4 IF (neighbour owns the file) THEN answer with QueryHit message via query path to querying peer; ELSE neighbour forwards Query message to its neighbours; Download the file Directly from source node e.g. via HTTP 8
9 Strengths and Weaknesses - Decentralized P2P Strengths: Absolutely decentralized network No single point of failure No performance bottleneck Main part of communication is anonymous Weaknesses: High network traffic via ping or pong packet Not protected against fire queries (i.e. attacker broadcasts artificial queries) Establishes request loops (i.e. overheads caused by message cycles) QueryHit rate is reduced in large networks 9
10 Second generation Hybrid P2P (/2) Goal: Combine efficiency properties of centralized model with robustness of decentralized model Web Cache Web Cache Web Cache list of SuperNodes Super Node Super Node Super Node Leaf Leaf Leaf HostCache Dividing servents into SuperNodes and LeafNodes Client software makes servents to SuperNodes by means of specific criteria (i.e. connection speed) SuperNodes are temporary servers Every SuperNode can manage approx. 2-5 leafs Reduce message transfer by Flow Control and Pong caching Bootstrapping by WebCaches (directory servers) and HostCache
11 Second generation Hybrid P2P (2/2) Leaf file transfer Super Node QueryHit Leaf Query forwarding Query Super Node Super Node Leaf Leaf Leaf Locating specific content Query message is sent to the SuperNode SuperNode decides about forwarding the query to the leafs by RouteTable managed by SuperNode RouteTableUpdate message is sent to the SuperNode by the LeafNode and contains all keywords which describe the content shared by this LeafNode Dependent on TTL, SuperNode can forward the query to other connected SuperNodes Eventually direct file transfer between LeafNodes
12 Third generation Structured P2P Lookup problems in unstructured P2P networks Where to place and how to find data items in a distributed system with regard to scalability and efficiency? Centralized P2P: Fast and efficient searching but not scalable Retrieving a data item is O() Decentralized P2P: Broadcast mechanism is not scalable about 4% of total traffic is caused by file queries Linear increasing communication overhead - O(n) No guarantee for a result when searching with limited TTL Structured P2P networks Based on Distributed Hash Table (DHT) Guaranteed correct results Quick search O(logN) 2
13 Distributed Hash Table (DHT) DHT is a data structure in which (Key, Value) pairs are distributed over the node amount as constantly as possible Key = hashed object identifier (OID) Value = IP address, NodeID, port Example: address space = Logical view of the DHT 8 Key NodeID= NodeIDs and keys are hashed in a common address space Every node is responsible for its own address space Thereby every node is analog to a bucket of a hash table The address space is viewed as a circle (Chord), a binary tree (Kademlia) or a quadratic area (CAN) If a node searches for a Value of a key then it has to locate a NodeID in whose address space the Key is included Mapping on the real topology 3
14 Distributed Hash Table - a simple algorithm (/2) Key=H( my data ) (4,(IP,NodeID,Port)) Value Node 4 manages Key 4 H( my data ) = (4,(IP,Port)) A naive algorithm for circle address space: Each node n knows its next neighbour NodeID -> successor(n) Each node n manages the address space s = ((predecessor(n)+) to n) consisting of (Key,Value) pairs Indicated by arrow Locating a Value Initial node searches for a Value of a Key H( my data ) () Initial node checks itself IF Key not found THEN initial Node n sends the FindValue msg to its next node (2) IF this next node manages this Key THEN send the Value to its next node until initial node is reached ELSE forward the FindValue msg to its next node 4
15 Distributed Hash Table - a simple algorithm (2/2) After a new node joined the network, the responsible manager of its address space has to partition this address space New node gets succ(n) and its predecessor updates its succ(n) If a node leaves the network then its next neighbour (succ(n)) adds its whole address space Its predecessor updates its succ(n) 8 5 5
16 Kademlia a specific DHT Approach Wide propagation (Emule, BitTorrent, ) Basic Idea: NodeIDs or their prefixes are mapped onto leafs of a binary tree Thereby, a prefix represents a single node managing all identifiers in its subtree On searching every hop leads to a small subtree, of which any node can be accessed Prefix of NodeID=4 () NodeID=3 () step: step:2 step:3 Prefix of NodeID= () NodeID=2 () Distance of two binary NodeIDs are calculated by the XOR metric Example: -> Node 2 XOR -> Node 4 Max. number of steps = 8+4=2 ; between Node 2 and Node 4 = ld(2)=3,58=4 Nodes 6 Hint: In the following examples a simplified tree structure is used
17 Kademlia routing information k-bucket The structure of an internal routing table consists of a binary tree Every node knows at least one node in each of its k-buckets Every node keeps a list (k-buckets) of (IP address, UDP port, i NodeID) triples for nodes of distance between and originated from itself with i 6 Every k-bucket keeps a maximum of contact triples Default k=2 (in our example below, kmax=2 only) Contacts are kept sorted by time last seen Least-recently seen node at the head Most-recently seen at the tail 2 i 2 7
18 Update of the k-bucket Each received message updates the corresponding k-bucket Case differentiation Sender node is known Move this node to the tail of the k-bucket Sender node is unknown and the k-bucket is not full Insert new node to tail of the k-bucket Sender node is unknown and the k-bucket is full Ping the node at the head of the k-bucket IF this node does not respond THEN remove this node AND insert the new node ELSE the new node is dropped 8
19 Kademlia k-bucket structure example View from Node : Possible routing table of NodeID contacts k-bucket 3 k-bucket k-bucket 2 k-bucket Node : Distance -> between and -> inserted in k-bucket 3 Node : Distance 7 -> between and -> inserted in k-bucket 2 Node : Distance 2 -> between and -> inserted in k-bucket... k-bucket
20 Kademlia Operation (/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 NodeID e.g. n=2 k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 step NodeID is searching for NodeID max. number of Hops: ld(2)=3,58=4 Nodes dist distance Contacts/ NodeIDs 2
21 Kademlia Operation (2/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 2. Those n NodeIDs answer with their k closest nodes NodeID e.g. n=2 2 step 2 Answer with AND k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 Contacts/ NodeIDs 4 and 7 dist= 2 and 3 8 and 5 Contacts/ NodeIDs 2
22 Kademlia Operation (3/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 2. Those n NodeIDs answer with their k closest nodes 3. In recursive steps initiator selects the n closest NodeIDs from the response set and resends the FIND_NODE msg to them IF one node does not answer, THEN select another one of the set NodeID Contacts/ NodeIDs e.g. n=2 4 and 7 dist= 2 and 3 8 and 5 Contacts/ NodeIDs step 3 k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 22
23 Kademlia Operation (4/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 2. Those n NodeIDs answer with their k closest nodes 3. In recursive steps initiator selects the n closest NodeIDs from the response set and resends the FIND_NODE msg to them IF one node does not answer, THEN select another one of the set Recipient returns (IP-Address, UDP-Port, NodeID) NodeID Contacts/ NodeIDs step 3 e.g. n=2 Answer with 2 and 3 dist= dist= 4 and 7 8 and5 Contacts/ NodeIDs Ping 4 and 7 dist= 2 and 3 8 and 5 Contacts/ NodeIDs k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 23
24 Kademlia Operation (5/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 2. Those n NodeIDs answer with their k closest nodes 3. In recursive steps initiator selects the n closest NodeIDs from the response set and resends the FIND_NODE msg to them IF one node does not answer, THEN select another one of the set Recipient returns (IP-Address, UDP-Port, NodeID) NodeID Contacts/ NodeIDs e.g. n=2 Contact the node No reaction 4 and 7 dist= 2 and 3 8 and 5 Contacts/ NodeIDs 2 and 3 dist= dist= 4 and 7 8 and5 Contacts/ NodeIDs k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 24
25 Kademlia Operation (6/7) FIND_NODE Locating k closest NodeIDs to a NodeID:. Initiator picks the n closest NodeIDs of the matched k- bucket, THEN sends the FIND_NODE msg to n NodeIDs in parallel, default: n=3 2. Those n NodeIDs answer with their k closest nodes 3. In recursive steps initiator selects the n closest NodeIDs from the response set and resends the FIND_NODE msg to them IF one node does not answer, THEN select another one of the set Recipient returns (IP-Address, UDP-Port, NodeID) NodeID Contacts/ NodeIDs 3 e.g. n=2 2 Answer with AND Answer with No reaction Ping 4 and 7 dist= 2 and 3 8 and 5 Contacts/ NodeIDs 2 and 3 dist= dist= 4 and 7 8 and5 Contacts/ NodeIDs k-buckets (distance between 2 i i and 2 ) 8 and 5 4 and 7 dist= dist= 2 and 3 25
26 Kademlia Operation (7/7) FIND_VALUE Locating the Value of a Key The process is similar to FIND_NODE IF a requested node manages this Key, THEN answer with Key Value ELSE FIND_VALUE get k closest NodeIDs to the Key and ask them for the Key Value STORE Save (Key,Value) pair to a node Locating the k closest NodeIDs for the Key by FIND_NODE Sending STORE msg to these k closest NodeIDs These k nodes re-publish this (Key,Value) pair to other closer new nodes at hourly intervals After 24 hours the link will expire PING Node online identification 26
27 Third generation BitTorrent Third generation file sharing protocol and system architecture for the distribution of large amounts of data High amount of the world wide P2P traffic are caused by BitTorrent Based on 3 important components: A web server offering a.torrent file A tracker keeping a list of clients downloading a specific file A client program acting as peer Two classes of peers exist:. Leechers: - Users who download files - These users provide their downloaded chunks for the upload 2. Seeders: - Users who have downloaded the complete file and only provide the upload Based on swarming Strengths: Very efficient file distribution system Highly scalable due to swarming Weaknesses: Tracker is a single point of failure (BitTorrent extensions without a central tracker based on Kademlia avoid this problem) No explicit file search functionality 27
28 Chunk 2 Swarming To increase efficiency of downloads, clients implement specific strategies Used in BitTorent system in combination with Kademlia algorithm File is splitted into many chunks and can be retrieved by downloading necessary chunks from different peers Swarming: A file sharing client downloads a file from many sources (peers) at the same time A swarm is a set of clients downloading the same file While a peer is downloading a file it already offers downloaded chunks to other peers Goal: Quickly replicate chunks to a large number of clients file XYZ Chunk Chunk 2 Chunk 3 Chunk 2 Chunk 2 still missing Chunk Chunk 2 Chunk 3 Chunk Chunk 2 Chunk 3 Chunk Chunk 2 Chunk 3 28 file XYZ file XYZ file XYZ
29 .torrent file For files shared via BitTorrent, a.torrent file is created and usually published on some web site Contains URL of tracker (server) Name and description of the file Length of chunks (e.g. 52 kb) Amount of chunks SHA- hashes of each piece in file for reliability A user has to search (manually) for this file After the.torrent file has been retrieved by a user (usually by its URI), it is imported by a BitTorrent client for getting necessary information for downloading the associated shared file 29
30 BitTorrent Web Server GET.torrent file 2 Identify tracker 3 Get adresses of peers Tracker 4 Request list of chunks The.torrent file is fetched and interpreted by a BitTorrent client ( and 2 ) The tracker specified for the desired file is contacted and a list with peers (leechers and seeders) offering parts of this file are returned by the tracker ( 3 ) The peers are contacted and a list of chunks offered by the apropriate peer is requested ( 4 ) After a client has retrieved the list with chunks from every peer, it has to decide which chunk it will request from which peer 3
31 Chunk selection policy Rarest first download the rarest chunk first Each peer creates a statistic of rarest chunks, indicator is the frequency of communication of this chunk to other peers Increases likelihood that all pieces are still available even if original seed leaves before any one node has downloaded entire file Random first The first chunk is downloaded randomly Risk: A new node downloads rarest first and logs off without sharing this chunk -> block the download Endgame Mode To avoid slow download in the end, all peers are requested for the missing chunks Request for missing chunk Chunk Chunk 2 Chunk 3 Client has almost downloaded all of the chunks Chunk Chunk 2 Chunk 3 Chunk Chunk 2 Chunk 3 Chunk Chunk 2 Chunk 3 3
32 Third generation Freenet Decentralized, censorship-resistant distributed data store Each participant provides a part of his own hard disc to store files from other participants A user has no control or knowledge of what kind of files his node stores All files on each node are encrypted One node knows only neighbor nodes thus requests can be only sent to neighbors No global semantic search functionality Uses Globally Unique Identifier (GUI) keys for identifying shared files Several types of key generation mechanisms may be applied, most important type of key: Signed-subspace Key (SSK) Every node has an ID and stores files having a key hash similar to this ID 32
33 Key Management rn/lectures/ public Key Hash String Signed-subspace Keys (SSK). Public-private key pair and rn/lectures/mc/scripts/chapter.pdf symmetric key are generated private Key Symmetric key 2. File is encrypted using the file symmetric key and signed using encrypt the private key. signature Nodes do not store the symmetric key, only the public key part of the SSK as an index to the data, thus a node hosting the file can plausibly deny knowledge of the stored data. 3. SSK is build by the hash of the public key and the symmetric key. The hash of the public key acts as the index to the data for searching purposes. Furthermore, the public key is stored with the data thus nodes can verify the signature when a SSK file enters a node and clients can verify the signature when retrieving a file. The symmetric key is used by clients for file decryption. file Key stored file Structure: SSK@<hash of the public key>,<symmetric key>, <information about crypto mechanism><description - file version> Published via e.g. a website or mail Example:SSK@GB3wuHmtxN2wLc7g4yZVydkK6sOT-DuOsUoeHK35w,c63EzO7uBENpiUbHPkMcJYW7i7cOvG42CM3YDduXDs,AQABAAE/rn/lectures/mc/scripts/c hapter.pdf-4 33
34 Requesting files and data transfer A user gets a SSK (e.g. SSK@GB...AE/rn/lectures/mc/scripts/chapter.pdf-4) Client software extracts from this SSK the hash of the public key and the symmetric decryption Key Then sends the request message containing the hash of public key, a TTL value and a unique request ID () Initiator node checks its own file system (2) IF there is no match THEN send request message to the node with the closest ID until TTL expires (3) IF the file is found THEN send reply message to the requesting node (neighbour!) (4) Initiator decrypts the encrypted file Decryption ((encrypted file), decryption key) Decryption ((bla.bla), 3op7) = chapter.pdf Trans.id=453 Sourceadd=x.x.x.2 Des.add=x.x.x.3 hash = ah66 TTL=3 - abh6 search for chapter.pdf Data Request 3 Trans.id=453 Sourceadd=x.x.x.3 Des.add=x.x.x.4 hash = ah66 TTL=2 Trans.id=453 Sourceadd=x.x.x. Des.add=x.x.x.2 hash = ah66 TTL=4 Data Fail 2 - ah66 - ahg ai2u ahg5 Data Reply
35 Conclusion As decentralized as possible (anonymity/security), as centralized as necessary (efficient searching) and as fast as BitTorrent!? 35
36 References Ralf Steinmetz: Peer-to-Peer systems and applications, 25 Kademlia: A Peer-to-peer Information System Based on the XOR Metric by Petar Maymounkov and David Mazieres Freenet project: Freenet: A Distributed Anonymous Information Storage and Retrieval System by Ian Clarkem and Oskar Sandberg Petar Maymounkov and David Mazieres: Kademlia: A Peer-to-peer Information System Based on the XOR Metric, 36
Kademlia: A P2P Informa2on System Based on the XOR Metric
Kademlia: A P2P Informa2on System Based on the XOR Metric Today! By Petar Mayamounkov and David Mazières, presented at IPTPS 22 Next! Paper presentation and discussion Image from http://www.vs.inf.ethz.ch/about/zeit.jpg
More informationKademlia: A peer-to peer information system based on XOR. based on XOR Metric,by P. Maymounkov and D. Mazieres
: A peer-to peer information system based on XOR Metric,by P. Maymounkov and D. Mazieres March 10, 2009 : A peer-to peer information system based on XOR Features From past p2p experiences, it has been
More information08 Distributed Hash Tables
08 Distributed Hash Tables 2/59 Chord Lookup Algorithm Properties Interface: lookup(key) IP address Efficient: O(log N) messages per lookup N is the total number of servers Scalable: O(log N) state per
More informationOverlay networks. To do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. Turtles all the way down. q q q
Overlay networks To do q q q Overlay networks P2P evolution DHTs in general, Chord and Kademlia Turtles all the way down Overlay networks virtual networks Different applications with a wide range of needs
More informationDISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES
DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES Dr. Jack Lange Computer Science Department University of Pittsburgh Fall 2015 Outline System Architectural Design Issues Centralized Architectures Application
More informationCompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang
CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview Problem Evolving solutions IP multicast Proxy caching Content distribution networks
More informationinternet technologies and standards
Institute of Telecommunications Warsaw University of Technology 25 internet technologies and standards Piotr Gajowniczek Andrzej Bąk Michał Jarociński Internet application layer peer-to-peer systems overview
More informationScalable overlay Networks
overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent
More informationDISTRIBUTED SYSTEMS CSCI 4963/ /4/2015
1 DISTRIBUTED SYSTEMS CSCI 4963/6963 12/4/2015 2 Info Quiz 7 on Tuesday. Project 2 submission URL is posted on the web site Submit your source code and project report (PDF!!!) in a single zip file. If
More informationOverlay networks. T o do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. q q q. Turtles all the way down
Overlay networks T o do q q q Overlay networks P2P evolution DHTs in general, Chord and Kademlia Turtles all the way down Overlay networks virtual networks Different applications with a wide range of needs
More informationContent Overlays. Nick Feamster CS 7260 March 12, 2007
Content Overlays Nick Feamster CS 7260 March 12, 2007 Content Overlays Distributed content storage and retrieval Two primary approaches: Structured overlay Unstructured overlay Today s paper: Chord Not
More informationOverlay and P2P Networks. Unstructured networks. Prof. Sasu Tarkoma
Overlay and P2P Networks Unstructured networks Prof. Sasu Tarkoma 19.1.2015 Contents Unstructured networks Last week Napster Skype This week: Gnutella BitTorrent P2P Index It is crucial to be able to find
More informationBitTorrent. Masood Khosroshahy. July Tech. Report. Copyright 2009 Masood Khosroshahy, All rights reserved.
BitTorrent Masood Khosroshahy July 2009 Tech. Report Copyright 2009 Masood Khosroshahy, All rights reserved. www.masoodkh.com Contents Contents 1 Basic Concepts 1 2 Mechanics 3 2.1 Protocols: Tracker and
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2018 Networking Laboratory P2P Applications Traditional P2P applications: for file sharing BitTorrent, Emule
More informationScalable overlay Networks
overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent
More informationTelematics Chapter 9: Peer-to-Peer Networks
Telematics Chapter 9: Peer-to-Peer Networks Beispielbild User watching video clip Server with video clips Application Layer Presentation Layer Application Layer Presentation Layer Session Layer Session
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2017 Networking Laboratory Presentation Outline 2.1 Introduction 2.2 Client-Server Paradigm 2.3 Peer-To-Peer
More informationOverlay and P2P Networks. Unstructured networks. PhD. Samu Varjonen
Overlay and P2P Networks Unstructured networks PhD. Samu Varjonen 25.1.2016 Contents Unstructured networks Last week Napster Skype This week: Gnutella BitTorrent P2P Index It is crucial to be able to find
More informationLesson 9 Applications of DHT: Bittorrent Mainline DHT, the KAD network
Lesson 9 Applications of DHT: Bittorrent Mainline DHT, the KAD network 20/03/2017 1 IMPLEMENTATION OF KADEMLIA Kademlia is the protocol used by the largest public DHTs. Vuze ( before known as Azureus),
More informationFlooded Queries (Gnutella) Centralized Lookup (Napster) Routed Queries (Freenet, Chord, etc.) Overview N 2 N 1 N 3 N 4 N 8 N 9 N N 7 N 6 N 9
Peer-to-Peer Networks -: Computer Networking L-: PP Typically each member stores/provides access to content Has quickly grown in popularity Bulk of traffic from/to CMU is Kazaa! Basically a replication
More informationIntroduction to P2P Computing
Introduction to P2P Computing Nicola Dragoni Embedded Systems Engineering DTU Compute 1. Introduction A. Peer-to-Peer vs. Client/Server B. Overlay Networks 2. Common Topologies 3. Data Location 4. Gnutella
More informationOverlay and P2P Networks. Unstructured networks. Prof. Sasu Tarkoma
Overlay and P2P Networks Unstructured networks Prof. Sasu Tarkoma 20.1.2014 Contents P2P index revisited Unstructured networks Gnutella Bloom filters BitTorrent Freenet Summary of unstructured networks
More informationLecture 21 P2P. Napster. Centralized Index. Napster. Gnutella. Peer-to-Peer Model March 16, Overview:
PP Lecture 1 Peer-to-Peer Model March 16, 005 Overview: centralized database: Napster query flooding: Gnutella intelligent query flooding: KaZaA swarming: BitTorrent unstructured overlay routing: Freenet
More informationCS 3516: Advanced Computer Networks
Welcome to CS 3516: Advanced Computer Networks Prof. Yanhua Li Time: 9:00am 9:50am M, T, R, and F Location: Fuller 320 Fall 2017 A-term 1 Some slides are originally from the course materials of the textbook
More informationPeer-to-Peer (P2P) Systems
Peer-to-Peer (P2P) Systems What Does Peer-to-Peer Mean? A generic name for systems in which peers communicate directly and not through a server Characteristics: decentralized self-organizing distributed
More informationPeer-to-peer systems and overlay networks
Complex Adaptive Systems C.d.L. Informatica Università di Bologna Peer-to-peer systems and overlay networks Fabio Picconi Dipartimento di Scienze dell Informazione 1 Outline Introduction to P2P systems
More information15-744: Computer Networking P2P/DHT
15-744: Computer Networking P2P/DHT Overview P2P Lookup Overview Centralized/Flooded Lookups Routed Lookups Chord Comparison of DHTs 2 Peer-to-Peer Networks Typically each member stores/provides access
More informationCS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University
CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes
More informationOverlay and P2P Networks. Unstructured networks: Freenet. Dr. Samu Varjonen
Overlay and P2P Networks Unstructured networks: Freenet Dr. Samu Varjonen 1.2.2015 Contents Last week Napster Skype Gnutella BitTorrent and analysis This week: Freenet Introduction to structured networks
More informationChapter 2: Application layer
Chapter 2: Application layer 2. Principles of network applications app architectures app requirements 2.2 Web and HTTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 P2P applications 2.7 Socket programming
More informationPeer-to-Peer Protocols and Systems. TA: David Murray Spring /19/2006
Peer-to-Peer Protocols and Systems TA: David Murray 15-441 Spring 2006 4/19/2006 P2P - Outline What is P2P? P2P System Types 1) File-sharing 2) File distribution 3) Streaming Uses & Challenges 2 Problem:
More informationPeer-to-Peer Internet Applications: A Review
Peer-to-Peer Internet Applications: A Review Davide Quaglia 01/14/10 Introduction Key points Lookup task Outline Centralized (Napster) Query flooding (Gnutella) Distributed Hash Table (Chord) Simulation
More information12/5/16. Peer to Peer Systems. Peer-to-peer - definitions. Client-Server vs. Peer-to-peer. P2P use case file sharing. Topics
// Topics Peer to Peer Systems Introduction Client-server vs peer to peer Peer-to-peer networks Routing Overlays Structured vs unstructured Example PP Systems Skype login server Peer-to-peer - definitions
More informationResilient Networking. 6: Resilient Peer-to-Peer Networking Thorsten Strufe. What is P2P? Types of P2P Systems P2P Security
6: Resilient Peer-to-Peer Networking Thorsten Strufe What is P2P? Types of P2P Systems P2P Security How do you define peer-to-peer? 2 What do you consider to be the main problems? 3 What do you think are
More informationPage 1. How Did it Start?" Model" Main Challenge" CS162 Operating Systems and Systems Programming Lecture 24. Peer-to-Peer Networks"
How Did it Start?" CS162 Operating Systems and Systems Programming Lecture 24 Peer-to-Peer Networks" A killer application: Napster (1999) Free music over the Internet Key idea: share the storage and bandwidth
More informationLast Lecture SMTP. SUNY at Buffalo; CSE 489/589 Modern Networking Concepts; Fall 2010; Instructor: Hung Q. Ngo 1
Last Lecture SMTP SUNY at Buffalo; CSE 489/589 Modern Networking Concepts; Fall 2010; Instructor: Hung Q. Ngo 1 This Lecture Peer-to-Peer (P2P) Applications SUNY at Buffalo; CSE 489/589 Modern Networking
More informationENSC 835: HIGH-PERFORMANCE NETWORKS CMPT 885: SPECIAL TOPICS: HIGH-PERFORMANCE NETWORKS. Scalability and Robustness of the Gnutella Protocol
ENSC 835: HIGH-PERFORMANCE NETWORKS CMPT 885: SPECIAL TOPICS: HIGH-PERFORMANCE NETWORKS Scalability and Robustness of the Gnutella Protocol Spring 2006 Final course project report Eman Elghoneimy http://www.sfu.ca/~eelghone
More informationInternet Services & Protocols
Department of Computer Science Institute for System Architecture, Chair for Computer Networks Internet Services & Protocols Content Distribution Dr.-Ing. Stephan Groß Room: INF 3099 E-Mail: stephan.gross@tu-dresden.de
More informationPeer-to-Peer Networks
Peer-to-Peer Networks 14-740: Fundamentals of Computer Networks Bill Nace Material from Computer Networking: A Top Down Approach, 6 th edition. J.F. Kurose and K.W. Ross Administrivia Quiz #1 is next week
More informationMaking Gnutella-like P2P Systems Scalable
Making Gnutella-like P2P Systems Scalable Y. Chawathe, S. Ratnasamy, L. Breslau, N. Lanham, S. Shenker Presented by: Herman Li Mar 2, 2005 Outline What are peer-to-peer (P2P) systems? Early P2P systems
More informationDistributed Systems. 17. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2016
Distributed Systems 17. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2016 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More informationA Survey of Peer-to-Peer Content Distribution Technologies
A Survey of Peer-to-Peer Content Distribution Technologies Stephanos Androutsellis-Theotokis and Diomidis Spinellis ACM Computing Surveys, December 2004 Presenter: Seung-hwan Baek Ja-eun Choi Outline Overview
More informationIntroduction to Peer-to-Peer Systems
Introduction Introduction to Peer-to-Peer Systems Peer-to-peer (PP) systems have become extremely popular and contribute to vast amounts of Internet traffic PP basic definition: A PP system is a distributed
More informationTinyTorrent: Implementing a Kademlia Based DHT for File Sharing
1 TinyTorrent: Implementing a Kademlia Based DHT for File Sharing A CS244B Project Report By Sierra Kaplan-Nelson, Jestin Ma, Jake Rachleff {sierrakn, jestinm, jakerach}@cs.stanford.edu Abstract We implemented
More informationINF5070 media storage and distribution systems. to-peer Systems 10/
INF5070 Media Storage and Distribution Systems: Peer-to to-peer Systems 10/11 2003 Client-Server! Traditional distributed computing! Successful architecture, and will continue to be so (adding proxy servers)!
More informationPeer to Peer Systems and Probabilistic Protocols
Distributed Systems 600.437 Peer to Peer Systems & Probabilistic Protocols Department of Computer Science The Johns Hopkins University 1 Peer to Peer Systems and Probabilistic Protocols Lecture 11 Good
More informationEE 122: Peer-to-Peer Networks
EE 122: Peer-to-Peer Networks Ion Stoica (and Brighten Godfrey) TAs: Lucian Popa, David Zats and Ganesh Ananthanarayanan http://inst.eecs.berkeley.edu/~ee122/ (Materials with thanks to Vern Paxson, Jennifer
More informationP2P. 1 Introduction. 2 Napster. Alex S. 2.1 Client/Server. 2.2 Problems
P2P Alex S. 1 Introduction The systems we will examine are known as Peer-To-Peer, or P2P systems, meaning that in the network, the primary mode of communication is between equally capable peers. Basically
More informationCMSC 332 Computer Networks P2P and Sockets
CMSC 332 Computer Networks P2P and Sockets Professor Szajda Announcements Programming Assignment 1 is due Thursday Where are we? What sorts of problems are we having? 2 Recap SMTP is the language that
More informationP2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli
P2P Applications Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli Server-based Network Peer-to-peer networks A type of network
More informationPeer-to-Peer Systems. Chapter General Characteristics
Chapter 2 Peer-to-Peer Systems Abstract In this chapter, a basic overview is given of P2P systems, architectures, and search strategies in P2P systems. More specific concepts that are outlined include
More informationLecture 8: Application Layer P2P Applications and DHTs
Lecture 8: Application Layer P2P Applications and DHTs COMP 332, Spring 2018 Victoria Manfredi Acknowledgements: materials adapted from Computer Networking: A Top Down Approach 7 th edition: 1996-2016,
More informationCS 640 Introduction to Computer Networks. Today s lecture. What is P2P? Lecture30. Peer to peer applications
Introduction to Computer Networks Lecture30 Today s lecture Peer to peer applications Napster Gnutella KaZaA Chord What is P2P? Significant autonomy from central servers Exploits resources at the edges
More informationPeer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today
Network Science: Peer-to-Peer Systems Ozalp Babaoglu Dipartimento di Informatica Scienza e Ingegneria Università di Bologna www.cs.unibo.it/babaoglu/ Introduction Peer-to-peer (PP) systems have become
More informationExtreme Computing. BitTorrent and incentive-based overlay networks.
Extreme Computing BitTorrent and incentive-based overlay networks BitTorrent Today we will focus on BitTorrent The technology really has three aspects A standard that BitTorrent client systems follow Some
More informationPeer-to-Peer Applications Reading: 9.4
Peer-to-Peer Applications Reading: 9.4 Acknowledgments: Lecture slides are from Computer networks course thought by Jennifer Rexford at Princeton University. When slides are obtained from other sources,
More informationToday. Architectural Styles
Today Architectures for distributed systems (Chapter 2) Centralized, decentralized, hybrid Middleware Self-managing systems Lecture 2, page 1 Architectural Styles Important styles of architecture for distributed
More informationINF5071 Performance in distributed systems: Distribution Part III
INF5071 Performance in distributed systems: Distribution Part III 5 November 2010 Client-Server Traditional distributed computing Successful architecture, and will continue to be so (adding proxy servers)
More informationAdvanced Computer Networks
Advanced Computer Networks P2P Systems Jianping Pan Summer 2007 5/30/07 csc485b/586b/seng480b 1 C/S vs P2P Client-server server is well-known server may become a bottleneck Peer-to-peer everyone is a (potential)
More informationEE 122: Peer-to-Peer (P2P) Networks. Ion Stoica November 27, 2002
EE 122: Peer-to-Peer (P2P) Networks Ion Stoica November 27, 22 How Did it Start? A killer application: Naptser - Free music over the Internet Key idea: share the storage and bandwidth of individual (home)
More informationMarch 10, Distributed Hash-based Lookup. for Peer-to-Peer Systems. Sandeep Shelke Shrirang Shirodkar MTech I CSE
for for March 10, 2006 Agenda for Peer-to-Peer Sytems Initial approaches to Their Limitations CAN - Applications of CAN Design Details Benefits for Distributed and a decentralized architecture No centralized
More informationP2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza
P2P Applications Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Versione originale delle slides fornita da Dora Spenza e Marco Barbera P2P Paradigm Late 80
More informationArchitectures for distributed systems (Chapter 2)
Today Architectures for distributed systems (Chapter 2) Architectural styles Client-server architectures Decentralized and peer-to-peer architectures Lecture 2, page!1 Module 1: Architectural Styles Important
More informationUnit 8 Peer-to-Peer Networking
Unit 8 Peer-to-Peer Networking P2P Systems Use the vast resources of machines at the edge of the Internet to build a network that allows resource sharing without any central authority. Client/Server System
More informationAdvanced Distributed Systems. Peer to peer systems. Reference. Reference. What is P2P? Unstructured P2P Systems Structured P2P Systems
Advanced Distributed Systems Peer to peer systems Karl M. Göschka Karl.Goeschka@tuwien.ac.at http://www.infosys.tuwien.ac.at/teaching/courses/ AdvancedDistributedSystems/ What is P2P Unstructured P2P Systems
More informationEECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Overlay Networks: Motivations
EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks Ion Stoica Computer Science Division Department of Electrical Engineering and Computer Sciences University of California, Berkeley
More informationScaling Problem Millions of clients! server and network meltdown. Peer-to-Peer. P2P System Why p2p?
Scaling Problem Millions of clients! server and network meltdown Peer-to-Peer 15-441 2 P2P System Why p2p? Leverage the resources of client machines (peers) Computation, storage, bandwidth Scaling: Create
More informationOverview Computer Networking Lecture 16: Delivering Content: Peer to Peer and CDNs Peter Steenkiste
Overview 5-44 5-44 Computer Networking 5-64 Lecture 6: Delivering Content: Peer to Peer and CDNs Peter Steenkiste Web Consistent hashing Peer-to-peer Motivation Architectures Discussion CDN Video Fall
More informationToday. Architectural Styles
Today Architectures for distributed systems (Chapter 2) Centralized, decentralized, hybrid Middleware Self-managing systems Lecture 2, page 1 Architectural Styles Important styles of architecture for distributed
More informationWeb caches (proxy server) Applications (part 3) Applications (part 3) Caching example (1) More about Web caching
By the end of this lecture, you should be able to. Explain the idea of edge delivery Explain the operation of CDNs Explain the operation of P2P file sharing systems such as Napster and Gnutella Web caches
More informationDistributed Systems. 16. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2017
Distributed Systems 16. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2017 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can
More informationIntroduction to P P Networks
Introduction to P P Networks B Sc Florian Adamsky florianadamsky@iemthmde http://florianadamskyit/ cbd Internet Protocols and Applications SS B Sc Florian Adamsky IPA / Outline Introduction What is P P?
More informationSlides for Chapter 10: Peer-to-Peer Systems
Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore, Kindberg and Blair Distributed Systems: Concepts and Design Edition 5, Addison-Wesley 2012 Overview of Chapter Introduction Napster
More informationCS 3516: Computer Networks
Welcome to CS 3516: Computer Networks Prof. Yanhua Li Time: 9:00am 9:50am M, T, R, and F Location: AK219 Fall 2018 A-term 1 Some slides are originally from the course materials of the textbook Computer
More informationR/Kademlia: Recursive and Topology-aware Overlay Routing
R/Kademlia: Recursive and Topology-aware Overlay Routing Bernhard Heep ATNAC 2010, Auckland, New Zealand, 11/02/2010, KIT University of the State of Baden-Wuerttemberg and National Research Center of the
More informationMiddleware and Distributed Systems. Peer-to-Peer Systems. Peter Tröger
Middleware and Distributed Systems Peer-to-Peer Systems Peter Tröger Peer-to-Peer Systems (P2P) Concept of a decentralized large-scale distributed system Large number of networked computers (peers) Each
More informationFreenet. Goals & Properties
Freenet an adaptive peer-to-peer network application that permits the publication, replication, and retrieval of data while protecting the anonymity of both authors and readers. Clarke, et.al. Goals &
More informationEffects of Churn on Structured P2P Overlay Networks
International Conference on Automation, Control, Engineering and Computer Science (ACECS'14) Proceedings - Copyright IPCO-214, pp.164-17 ISSN 2356-568 Effects of Churn on Structured P2P Overlay Networks
More informationContent Overlays (continued) Nick Feamster CS 7260 March 26, 2007
Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007 Administrivia Quiz date Remaining lectures Interim report PS 3 Out Friday, 1-2 problems 2 Structured vs. Unstructured Overlays Structured
More informationSearching for Shared Resources: DHT in General
1 ELT-53206 Peer-to-Peer Networks Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationPeer 2 Peer Massively Multiuser VEs. Patrice Torguet Université Paul Sabatier
Peer 2 Peer Massively Multiuser VEs Patrice Torguet torguet@irit.fr Université Paul Sabatier Schedule part 1 P2P Introduction Problems of P2P overlays Protocol operations Bootstrapping the P2P overlay
More informationLecture 25: Security Issues in Structured Peer-to-Peer Networks. Lecture Notes on Computer and Network Security. by Avi Kak
Lecture 25: Security Issues in Structured Peer-to-Peer Networks Lecture Notes on Computer and Network Security by Avi Kak (kak@purdue.edu) April 18, 2017 3:23pm c 2017 Avinash Kak, Purdue University Goals:
More informationOverlay Networks: Motivations. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Motivations (cont d) Goals.
Overlay Networks: Motivations CS : Introduction to Computer Networks Overlay Networks and PP Networks Ion Stoica Computer Science Division Department of lectrical ngineering and Computer Sciences University
More informationContent Search. Unstructured P2P. Jukka K. Nurminen
Content Search Unstructured P2P Jukka K. Nurminen *Partly adapted from original slides provided by Rüdiger Schollmeier and Jörg Eberspächer (Technische Universität München) 1 V1-Filename.ppt / yyyy-mm-dd
More informationCSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing. March 8, 2016 Prof. George Porter
CSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing March 8, 2016 rof. George orter Outline Today: eer-to-peer networking Distributed hash tables Consistent hashing
More informationCIS 700/005 Networking Meets Databases
Announcements CIS / Networking Meets Databases Boon Thau Loo Spring Lecture Paper summaries due at noon today. Office hours: Wed - pm ( Levine) Project proposal: due Feb. Student presenter: rd Jan: A Scalable
More informationDefinition of a DS A collection of independent computers that appear to its user as a single coherent system.
Definition of a DS A collection of independent computers that appear to its user as a single coherent system. Challenges in DS Transparency, Heterogeneity, Failure Handling, Openness, Scalability and Openness
More informationMotivation for peer-to-peer
Peer-to-peer systems INF 5040 autumn 2015 lecturer: Roman Vitenberg INF5040, Frank Eliassen & Roman Vitenberg 1 Motivation for peer-to-peer Ø Inherent restrictions of the standard client/ server model
More informationScalability In Peer-to-Peer Systems. Presented by Stavros Nikolaou
Scalability In Peer-to-Peer Systems Presented by Stavros Nikolaou Background on Peer-to-Peer Systems Definition: Distributed systems/applications featuring: No centralized control, no hierarchical organization
More informationLecture 17: Peer-to-Peer System and BitTorrent
CSCI-351 Data communication and Networks Lecture 17: Peer-to-Peer System and BitTorrent (I swear I only use it for Linux ISOs) The slide is built with the help of Prof. Alan Mislove, Christo Wilson, and
More informationContent Search. Unstructured P2P
Content Search Unstructured P2P Prof. Jukka K. Nurminen Data Communications Software (DCS) Lab, Department of Computer Science and Engineering, Aalto University *Partly adapted from original slides provided
More informationOverlay networks. Today. l Overlays networks l P2P evolution l Pastry as a routing overlay example
Overlay networks Today l Overlays networks l P2P evolution l Pastry as a routing overlay eample Network virtualization and overlays " Different applications with a range of demands/needs network virtualization
More informationProject Guide for Kademlia
Project Guide for Kademlia Building Go's build tools depend on the value of the GOPATH environment variable. $GOPATH should be the project root: the absolute path of the directory containing {bin,pkg,src}.
More informationDistributed hash table - Wikipedia, the free encyclopedia
Page 1 sur 6 Distributed hash table From Wikipedia, the free encyclopedia Distributed hash tables (DHTs) are a class of decentralized distributed systems that provide a lookup service similar to a hash
More informationOverlay and P2P Networks. Structured Networks and DHTs. Prof. Sasu Tarkoma
Overlay and P2P Networks Structured Networks and DHTs Prof. Sasu Tarkoma 6.2.2014 Contents Today Semantic free indexing Consistent Hashing Distributed Hash Tables (DHTs) Thursday (Dr. Samu Varjonen) DHTs
More informationDistributed Systems Peer-to-Peer Systems
Distributed Systems Peer-to-Peer Systems [2] P2P Systems - Goals and Definition Goal: to enable sharing of data and resources on a very large scale by eliminating any requirement for separately-managed
More informationSearching for Shared Resources: DHT in General
1 ELT-53207 P2P & IoT Systems Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationBadri Nath Rutgers University
lookup services Badri Nath Rutgers University badri@cs.rutgers.edu 1. CAN: A scalable content addressable network, Sylvia Ratnasamy et.al. SIGCOMM 2001 2. Chord: A scalable peer-to-peer lookup protocol
More information(C) Chukiat Worasucheep 1
Slides for Chapter 10: Peer-to-Peer Systems From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 4, Pearson Education 2005 Contents Motivations & characteristics P2P
More informationCSE 486/586 Distributed Systems Peer-to-Peer Architectures
CSE 486/586 Distributed Systems eer-to-eer Architectures Steve Ko Computer Sciences and Engineering University at Buffalo CSE 486/586 Last Time Gossiping Multicast Failure detection Today s Question How
More information