Overlay Networks: Motivations. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Motivations (cont d) Goals.

Size: px
Start display at page:

Download "Overlay Networks: Motivations. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Motivations (cont d) Goals."

Transcription

1 Overlay Networks: Motivations CS : Introduction to Computer Networks Overlay Networks and PP Networks Ion Stoica Computer Science Division Department of lectrical ngineering and Computer Sciences University of California, Berkeley Berkeley, CA 9- Changes in the network happen very slowly Why? - Internet network is a shared infrastructure; need to achieve consensus (ITF) - Many of proposals require to change a large number of routers (e.g., IP Multicast, QoS); otherwise end-users won t benefit Proposed changes that haven t happened yet on large scale: - More Addresses (IPv 9) - Security (IPSC 9); Multicast (IP multicast 9) Motivations (cont d) Goals One size does not fit all Applications need different levels of - Reliability - Performance (latency) - Security - Access control (e.g., who is allowed to join a multicast group) - Make it easy to deploy new functionalities in the network accelerate the pace of innovation Allow users to customize their service Solution Deploy processing in the network Have packets processed as they traverse the network AS- IP Overview Resilient Overlay Network (RON) Overlay Multicast Peer-to-peer systems AS- Overlay Network (over IP)

2 Resilient Overlay Network (RON) xample Premise: by building application overlay network, can increase performance and reliability of routing Berkeley MIT Install N computers at different Internet locations Default IP path determined by BGP & OSPF ach computer acts as an overlay network router - Between each overlay router is an IP tunnel (logical link) - Logical overlay topology is all-to-all (N ) Computers actively measure each logical link in real time for - Packet loss rate, latency, throughput, etc Route overlay network traffic based on measured characteristics UCLA Reroute traffic using red alternative overlay network path, avoid congestion point Acts as overlay router Overview IP Multicast Problems Resilient Overlay Network (RON) Seventeen years of research, but still not widely deployed Overlay multicast Peer-to-peer systems Poor scalability - Routers need to maintain per-group or even per-group and persender state! - Multicast addresses cannot be aggregated Supporting higher level functionality is difficult - IP Multicast: best-effort multi-point delivery service - Reliability and congestion control for IP Multicast complicated No support for access control - Nor restriction on who can send easy to mount Denial of Service (Dos) attacks! 9 Overlay Approach Narada [Yang-hua et al, ] Provide IP multicast functionality above the IP layer application level multicast Challenge: do this efficiently Projects: - Narada - Overcast - Scattercast - Yoid - Coolstreaming (Roxbeam) - Rawflow Source Speific Trees Involves only end hosts Small group sizes <= hundreds of nodes Typical application: chat

3 Narada: nd System Multicast Properties CMU Gatech Stanford Stan Stan asier to deploy than IP Multicast - Don t have to modify every router on path asier to implement reliability than IP Multicast - Use hop-by-hop retransmissions Berk Gatech Overlay Tree Stan Berkeley Berk But - Consume more bandwidth than IP Multicast - Typically has higher latency than IP Multicast - Harder to scale CMU Stan Berk Berk Optimization: use IP Multicast where available Overview Resilient Overlay Network (RON) Overlay multicast How Did it Start? A killer application: Naptser - Free music over the Internet Key idea: share the storage and bandwidth of individual (home) users Peer-to-peer systems Internet Model Main Challenge ach user stores a subset of files ach user has access (can download) files from all users in the system Find where a particular file is stored - Note: problem similar to finding a particular page in web caching (see last lecture what are the differences?) F D A B C

4 Other Challenges Napster Scale: up to hundred of thousands or millions of machines Dynamicity: machines can come and go any time Assume a centralized index system that maps files (songs) to machines that are alive How to find a file (song) - Query the index system return a machine that stores the required file Ideally this is the closest/least-loaded machine - ftp the file Advantages: - Simplicity, easy to implement sophisticated search engines on top of the index system Disadvantages: - Robustness, scalability (?) 9 Napster: xample Gnutella m F m A m m m A m B m C m D m m F m B D m C m Distribute file location Idea: broadcast the request Hot to find a file: - Send request to all neighbors - Neighbors recursively multicast the request - ventually a machine that has the file receives the request, and it sends back the answer Advantages: - Totally decentralized, highly robust Disadvantages: - Not scalable; the entire network can be swamped with requests (to alleviate this problem, each request has a TTL) Gnutella: xample Two-Level Hierarchy Assume: m s neighbors are m and m; m s neighbors are m and m; m F m A m m B m m C D Current Gnutella implementation, KaZaa Leaf nodes are connected to a small number of ultrapeers (suppernodes) Query - A leaf sends query to its ultrapeers - If ultrapeers don t know the answer, they flood the query to other ultrapeers More scalable: - Flooding only among ultrapeers Oct Crawl on Gnutella Ultrapeer nodes Leaf nodes

5 Skype BitTorrent (/) Peer-to-peer Internet Telephony Two-level hierarchy like KaZaa - Ultrapeers used mainly to route traffic between NATed end-hosts (see next slide) - plus a login server to authenticate users ensure that names are unique across network B A login server Messages exchanged to login server Data traffic Allow fast downloads even when sources have low connectivity How does it work? - Split each file into pieces (~ KB each), and each piece into sub-pieces (~ KB each) - The loader loads one piece at a time - Within one piece, the loader can load up to five subpieces in parallel (Note*: probable protocol; Skype protocol is not published) BitTorrent (/) Distributed Hash Tables Download consists of three phases: Start: get a piece as soon as possible - Select a random piece Middle: spread all pieces as soon as possible - Select rarest piece next nd: avoid getting stuck with a slow source, when downloading the last sub-pieces - Request in parallel the same sub-piece - Cancel slowest downloads once a sub-piece has been received Problem: - Given an ID, map to a host Challenges - Scalability: hundreds of thousands or millions of machines - Instability Changes in routes, congestion, availability of machines - Heterogeneity Latency: ms to ms Bandwidth: Kb/s to Mb/s Nodes stay in system from s to a year - Trust Selfish users Malicious users (For details see: Content Addressable Network (CAN) Associate to each node and item a unique id in an d-dimensional space Properties - Routing table size O(d) - Guarantees that a file is found in at most d*n /d steps, where n is the total number of nodes divided between nodes All nodes cover the entire space ach node covers either a square or a rectangular area of ratios : or : xample: - Assume space size ( x ) - Node n:(, ) first node that joins cover the entire space n 9

6 Node n:(, ) joins space is divided between n and n Node n:(, ) joins space is divided between n and n n n n n n Nodes n:(, ) and n:(,) join n n n Nodes: n:(, ); n:(,); n:(, ); n:(,);n:(,) Items: f:(,); f:(,); f:(,); f:(,); n n n f n n n f f n f CAN: Query xample ach item is stored by the node who owns its mapping in the space n n n f ach node knows its neighbors in the d-space Forward query to the neighbor that is closest to the query id xample: assume n queries f n n n f n f f n n f f n f f

7 Chord Identifier to Node Mapping xample Associate to each node and item a unique id in an uni-dimensional space.. m - Key design decision - Decouple correctness from efficiency Properties - Routing table size O(log(N)), where N is the total number of nodes - Guarantees that a file is found in O(log(N)) steps Node maps [,] Node maps [9,] Node maps [, ] Node maps [9, ] ach node maintains a pointer to its successor Lookup Joining Operation ach node maintains its successor Route packet (ID, data) to the node responsible for ID using successor pointers node= lookup() ach node A periodically sends a stabilize() message to its successor B Upon receiving a stabilize() message, node B - returns its predecessor B =pred(b) to A by sending a notify(b ) message Upon receiving notify(b ) from B, - if B is between A and B, A updates its successor to B - A doesn t do anything, otherwise 9 Joining Operation Joining Operation Node with id= joins the ring Node needs to know at least one node already in the system - Assume known node is succ=nil pred=nil succ= pred= Node : send join() to node Node : returns node Node updates its successor to succ= succ=nil pred=nil succ= pred= join() succ= pred= succ= pred=

8 Joining Operation Joining Operation (cont d) Node : send stabilize() to node Node : - update predecessor to - send notify() back succ= pred=nil succ= pred= pred= stabilize() succ= Node sends a stabilize pred= message to its successor, node Node reply with a notify message Node updates its stabilize() successor to succ= pred=nil succ= pred= succ= succ= pred= Joining Operation (cont d) Joining Operation (cont d) Node sends a stabilize message to its new successor, node Node sets its predecessor to node succ= pred= This completes the joining operation! pred= succ= pred= pred=nil Stabilize() succ= pred= succ= pred= succ= Finger Table at i ft[i] Achieving fficiency: finger tables + ( + ) mod = Say m= Achieving Robustness To improve robustness each node maintains the k (> ) immediate successors instead of only one successor In the notify() message, node A can send its k- successors to its predecessor B Upon receiving notify() message, B can update its successor list by concatenating the successor list received from A with A itself i m ith entry at peer with id n is first peer with id >= n + (mod )

9 Discussion Conclusions Query can be implemented - Iteratively - Recursively Performance: routing in the overlay network can be more expensive than in the underlying network - Because usually there is no correlation between node ids and their locality; a query can repeatedly jump from urope to North America, though both the initiator and the node that store the item are in urope! - Solutions: Tapestry takes care of this implicitly; CAN and Chord maintain multiple copies for each entry in their routing tables and choose the closest in terms of network distance The key challenge of building wide area PP systems is a scalable and robust directory service Solutions covered in this lecture - Naptser: centralized location service - Gnutella: broadcast-based decentralized location service - CAN, Chord, Tapestry, Pastry: intelligent-routing decentralized solution Guarantee correctness Tapestry, Pastry provide efficient routing, but more complex 9 9

EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Overlay Networks: Motivations

EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Overlay Networks: Motivations EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks Ion Stoica Computer Science Division Department of Electrical Engineering and Computer Sciences University of California, Berkeley

More information

Goals. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Solution. Overlay Networks: Motivations.

Goals. EECS 122: Introduction to Computer Networks Overlay Networks and P2P Networks. Solution. Overlay Networks: Motivations. Goals CS : Introduction to Computer Networks Overlay Networks and PP Networks Ion Stoica Computer Science Division Department of lectrical ngineering and Computer Sciences University of California, Berkeley

More information

EE 122: Peer-to-Peer Networks

EE 122: Peer-to-Peer Networks EE 122: Peer-to-Peer Networks Ion Stoica (and Brighten Godfrey) TAs: Lucian Popa, David Zats and Ganesh Ananthanarayanan http://inst.eecs.berkeley.edu/~ee122/ (Materials with thanks to Vern Paxson, Jennifer

More information

Page 1. How Did it Start?" Model" Main Challenge" CS162 Operating Systems and Systems Programming Lecture 24. Peer-to-Peer Networks"

Page 1. How Did it Start? Model Main Challenge CS162 Operating Systems and Systems Programming Lecture 24. Peer-to-Peer Networks How Did it Start?" CS162 Operating Systems and Systems Programming Lecture 24 Peer-to-Peer Networks" A killer application: Napster (1999) Free music over the Internet Key idea: share the storage and bandwidth

More information

Main Challenge. Other Challenges. How Did it Start? Napster. Model. EE 122: Peer-to-Peer Networks. Find where a particular file is stored

Main Challenge. Other Challenges. How Did it Start? Napster. Model. EE 122: Peer-to-Peer Networks. Find where a particular file is stored Main hallenge ind where a particular file is stored : Peer-to-Peer Networks Ion Stoica (and righten Godfrey) Ts: Lucian Popa, avid Zats and Ganesh nanthanarayanan http://inst.eecs.berkeley.edu/~ee/ (Materials

More information

EE 122: Peer-to-Peer (P2P) Networks. Ion Stoica November 27, 2002

EE 122: Peer-to-Peer (P2P) Networks. Ion Stoica November 27, 2002 EE 122: Peer-to-Peer (P2P) Networks Ion Stoica November 27, 22 How Did it Start? A killer application: Naptser - Free music over the Internet Key idea: share the storage and bandwidth of individual (home)

More information

CSCI-1680 P2P Rodrigo Fonseca

CSCI-1680 P2P Rodrigo Fonseca CSCI-1680 P2P Rodrigo Fonseca Based partly on lecture notes by Ion Stoica, Sco5 Shenker, Joe Hellerstein Today Overlay networks and Peer-to-Peer Motivation Suppose you want to write a routing protocol

More information

CIS 700/005 Networking Meets Databases

CIS 700/005 Networking Meets Databases Announcements CIS / Networking Meets Databases Boon Thau Loo Spring Lecture Paper summaries due at noon today. Office hours: Wed - pm ( Levine) Project proposal: due Feb. Student presenter: rd Jan: A Scalable

More information

Lecture 6: Overlay Networks. CS 598: Advanced Internetworking Matthew Caesar February 15, 2011

Lecture 6: Overlay Networks. CS 598: Advanced Internetworking Matthew Caesar February 15, 2011 Lecture 6: Overlay Networks CS 598: Advanced Internetworking Matthew Caesar February 15, 2011 1 Overlay networks: Motivations Protocol changes in the network happen very slowly Why? Internet is shared

More information

Flooded Queries (Gnutella) Centralized Lookup (Napster) Routed Queries (Freenet, Chord, etc.) Overview N 2 N 1 N 3 N 4 N 8 N 9 N N 7 N 6 N 9

Flooded Queries (Gnutella) Centralized Lookup (Napster) Routed Queries (Freenet, Chord, etc.) Overview N 2 N 1 N 3 N 4 N 8 N 9 N N 7 N 6 N 9 Peer-to-Peer Networks -: Computer Networking L-: PP Typically each member stores/provides access to content Has quickly grown in popularity Bulk of traffic from/to CMU is Kazaa! Basically a replication

More information

CS 268: DHTs. Page 1. How Did it Start? Model. Main Challenge. Napster. Other Challenges

CS 268: DHTs. Page 1. How Did it Start? Model. Main Challenge. Napster. Other Challenges How Did it Start? CS : DHTs A killer application: Naptser - Free music over the Internet Key idea: share the content, storage and bandwidth of individual (home) users Scott Shenker and Ion Stoica April,

More information

Page 1. Key Value Storage"

Page 1. Key Value Storage Key Value Storage CS162 Operating Systems and Systems Programming Lecture 14 Key Value Storage Systems March 12, 2012 Anthony D. Joseph and Ion Stoica http://inst.eecs.berkeley.edu/~cs162 Handle huge volumes

More information

CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang

CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview Problem Evolving solutions IP multicast Proxy caching Content distribution networks

More information

0!1. Overlaying mechanism is called tunneling. Overlay Network Nodes. ATM links can be the physical layer for IP

0!1. Overlaying mechanism is called tunneling. Overlay Network Nodes. ATM links can be the physical layer for IP epartment of lectrical ngineering and omputer Sciences University of alifornia erkeley '!$$( network defined over another set of networks The overlay addresses its own nodes Links on one layer are network

More information

Announcements. EECS 122: Introduction to Computer Networks Multicast and Overlay Networks. Motivational Example: Streaming Media

Announcements. EECS 122: Introduction to Computer Networks Multicast and Overlay Networks. Motivational Example: Streaming Media Announcements EEC : Introduction to Computer Networks Multicast and Overlay Networks Ion toica (and Brighten Godfrey) TAs: Lucian Popa, David Zats and Ganesh Ananthanarayanan http://inst.eecs.berkeley.edu/~ee/

More information

CSCI-1680 Web Performance, Content Distribution P2P Rodrigo Fonseca

CSCI-1680 Web Performance, Content Distribution P2P Rodrigo Fonseca CSCI-1680 Web Performance, Content Distribution P2P Rodrigo Fonseca Based partly on lecture notes by Scott Shenker and John Jannotti Last time HTTP and the WWW Today: HTTP Performance Persistent Connections,

More information

CS 3516: Advanced Computer Networks

CS 3516: Advanced Computer Networks Welcome to CS 3516: Advanced Computer Networks Prof. Yanhua Li Time: 9:00am 9:50am M, T, R, and F Location: Fuller 320 Fall 2017 A-term 1 Some slides are originally from the course materials of the textbook

More information

CSCI-1680 Web Performance, Content Distribution P2P John Jannotti

CSCI-1680 Web Performance, Content Distribution P2P John Jannotti CSCI-1680 Web Performance, Content Distribution P2P John Jannotti Based partly on lecture notes by Sco2 Shenker and Rodrigo Fonseca Last time HTTP and the WWW Today: HTTP Performance Persistent Connections,

More information

Page 1. P2P Traffic" P2P Traffic" Today, around 18-20% (North America)! Big chunk now is video entertainment (e.g., Netflix, itunes)!

Page 1. P2P Traffic P2P Traffic Today, around 18-20% (North America)! Big chunk now is video entertainment (e.g., Netflix, itunes)! P2P Traffic" CS162 Operating Systems and Systems Programming Lecture 25 Capstone: P2P Systems, Review" 2004: some Internet Service Providers (ISPs) claimed that over 50% of their traffic was peer-to-peer

More information

CS4700/CS5700 Fundamentals of Computer Networks

CS4700/CS5700 Fundamentals of Computer Networks CS4700/CS5700 Fundamentals of Computer Networks Lecture 22: Overlay networks Slides used with permissions from Edward W. Knightly, T. S. Eugene Ng, Ion Stoica, Hui Zhang Alan Mislove amislove at ccs.neu.edu

More information

Overlay Networks. Behnam Momeni Computer Engineering Department Sharif University of Technology

Overlay Networks. Behnam Momeni Computer Engineering Department Sharif University of Technology CE443 Computer Networks Overlay Networks Behnam Momeni Computer Engineering Department Sharif University of Technology Acknowledgments: Lecture slides are from Computer networks course thought by Jennifer

More information

CS 640 Introduction to Computer Networks. Today s lecture. What is P2P? Lecture30. Peer to peer applications

CS 640 Introduction to Computer Networks. Today s lecture. What is P2P? Lecture30. Peer to peer applications Introduction to Computer Networks Lecture30 Today s lecture Peer to peer applications Napster Gnutella KaZaA Chord What is P2P? Significant autonomy from central servers Exploits resources at the edges

More information

EE122: Multicast. Kevin Lai October 7, 2002

EE122: Multicast. Kevin Lai October 7, 2002 EE122: Multicast Kevin Lai October 7, 2002 Internet Radio www.digitallyimported.com (techno station) - sends out 128Kb/s MP3 music streams - peak usage ~9000 simultaneous streams only 5 unique streams

More information

Telematics Chapter 9: Peer-to-Peer Networks

Telematics Chapter 9: Peer-to-Peer Networks Telematics Chapter 9: Peer-to-Peer Networks Beispielbild User watching video clip Server with video clips Application Layer Presentation Layer Application Layer Presentation Layer Session Layer Session

More information

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University

CS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes

More information

EE122: Multicast. Internet Radio. Multicast Service Model 1. Motivation

EE122: Multicast. Internet Radio. Multicast Service Model 1. Motivation Internet Radio EE122: Multicast Kevin Lai October 7, 2002 wwwdigitallyimportedcom (techno station) - sends out 128Kb/s MP music streams - peak usage ~9000 simultaneous streams only 5 unique streams (trance,

More information

Architectures for Distributed Systems

Architectures for Distributed Systems Distributed Systems and Middleware 2013 2: Architectures Architectures for Distributed Systems Components A distributed system consists of components Each component has well-defined interface, can be replaced

More information

Ossification of the Internet

Ossification of the Internet Ossification of the Internet The Internet evolved as an experimental packet-switched network Today, many aspects appear to be set in stone - Witness difficulty in getting IP multicast deployed - Major

More information

Peer-to-Peer Internet Applications: A Review

Peer-to-Peer Internet Applications: A Review Peer-to-Peer Internet Applications: A Review Davide Quaglia 01/14/10 Introduction Key points Lookup task Outline Centralized (Napster) Query flooding (Gnutella) Distributed Hash Table (Chord) Simulation

More information

15-744: Computer Networking P2P/DHT

15-744: Computer Networking P2P/DHT 15-744: Computer Networking P2P/DHT Overview P2P Lookup Overview Centralized/Flooded Lookups Routed Lookups Chord Comparison of DHTs 2 Peer-to-Peer Networks Typically each member stores/provides access

More information

Distributed Hash Tables Chord and Dynamo

Distributed Hash Tables Chord and Dynamo Distributed Hash Tables Chord and Dynamo (Lecture 19, cs262a) Ion Stoica, UC Berkeley October 31, 2016 Today s Papers Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications, Ion Stoica,

More information

Internet Indirection Infrastructure

Internet Indirection Infrastructure Motivations Internet Indirection Infrastructure Modified version of Ion Stoica s talk at ODU Nov 14, 05 Today s Internet is built around a unicast point-to-point communication abstraction: Send packet

More information

Distributed Systems. 17. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2016

Distributed Systems. 17. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2016 Distributed Systems 17. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2016 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can

More information

Content Overlays. Nick Feamster CS 7260 March 12, 2007

Content Overlays. Nick Feamster CS 7260 March 12, 2007 Content Overlays Nick Feamster CS 7260 March 12, 2007 Content Overlays Distributed content storage and retrieval Two primary approaches: Structured overlay Unstructured overlay Today s paper: Chord Not

More information

Lecture 6: Securing Distributed and Networked Systems. CS 598: Network Security Matthew Caesar March 12, 2013

Lecture 6: Securing Distributed and Networked Systems. CS 598: Network Security Matthew Caesar March 12, 2013 Lecture 6: Securing Distributed and Networked Systems CS 598: Network Security Matthew Caesar March 12, 2013 1 Today: Distributed Internet Services Previous cycle: how to build Internet services that run

More information

Introduction to P2P Computing

Introduction to P2P Computing Introduction to P2P Computing Nicola Dragoni Embedded Systems Engineering DTU Compute 1. Introduction A. Peer-to-Peer vs. Client/Server B. Overlay Networks 2. Common Topologies 3. Data Location 4. Gnutella

More information

DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES

DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES Dr. Jack Lange Computer Science Department University of Pittsburgh Fall 2015 Outline System Architectural Design Issues Centralized Architectures Application

More information

March 10, Distributed Hash-based Lookup. for Peer-to-Peer Systems. Sandeep Shelke Shrirang Shirodkar MTech I CSE

March 10, Distributed Hash-based Lookup. for Peer-to-Peer Systems. Sandeep Shelke Shrirang Shirodkar MTech I CSE for for March 10, 2006 Agenda for Peer-to-Peer Sytems Initial approaches to Their Limitations CAN - Applications of CAN Design Details Benefits for Distributed and a decentralized architecture No centralized

More information

Overview Computer Networking Lecture 16: Delivering Content: Peer to Peer and CDNs Peter Steenkiste

Overview Computer Networking Lecture 16: Delivering Content: Peer to Peer and CDNs Peter Steenkiste Overview 5-44 5-44 Computer Networking 5-64 Lecture 6: Delivering Content: Peer to Peer and CDNs Peter Steenkiste Web Consistent hashing Peer-to-peer Motivation Architectures Discussion CDN Video Fall

More information

Peer-to-Peer (P2P) Systems

Peer-to-Peer (P2P) Systems Peer-to-Peer (P2P) Systems What Does Peer-to-Peer Mean? A generic name for systems in which peers communicate directly and not through a server Characteristics: decentralized self-organizing distributed

More information

CS514: Intermediate Course in Computer Systems

CS514: Intermediate Course in Computer Systems Distributed Hash Tables (DHT) Overview and Issues Paul Francis CS514: Intermediate Course in Computer Systems Lecture 26: Nov 19, 2003 Distributed Hash Tables (DHT): Overview and Issues What is a Distributed

More information

Peer-to-Peer Protocols and Systems. TA: David Murray Spring /19/2006

Peer-to-Peer Protocols and Systems. TA: David Murray Spring /19/2006 Peer-to-Peer Protocols and Systems TA: David Murray 15-441 Spring 2006 4/19/2006 P2P - Outline What is P2P? P2P System Types 1) File-sharing 2) File distribution 3) Streaming Uses & Challenges 2 Problem:

More information

Introduction to Peer-to-Peer Systems

Introduction to Peer-to-Peer Systems Introduction Introduction to Peer-to-Peer Systems Peer-to-peer (PP) systems have become extremely popular and contribute to vast amounts of Internet traffic PP basic definition: A PP system is a distributed

More information

Chord: A Scalable Peer-to-peer Lookup Service For Internet Applications

Chord: A Scalable Peer-to-peer Lookup Service For Internet Applications Chord: A Scalable Peer-to-peer Lookup Service For Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented by Jibin Yuan ION STOICA Professor of CS

More information

Overlay networks. To do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. Turtles all the way down. q q q

Overlay networks. To do. Overlay networks. P2P evolution DHTs in general, Chord and Kademlia. Turtles all the way down. q q q Overlay networks To do q q q Overlay networks P2P evolution DHTs in general, Chord and Kademlia Turtles all the way down Overlay networks virtual networks Different applications with a wide range of needs

More information

Lecture 8: Application Layer P2P Applications and DHTs

Lecture 8: Application Layer P2P Applications and DHTs Lecture 8: Application Layer P2P Applications and DHTs COMP 332, Spring 2018 Victoria Manfredi Acknowledgements: materials adapted from Computer Networking: A Top Down Approach 7 th edition: 1996-2016,

More information

Peer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today

Peer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today Network Science: Peer-to-Peer Systems Ozalp Babaoglu Dipartimento di Informatica Scienza e Ingegneria Università di Bologna www.cs.unibo.it/babaoglu/ Introduction Peer-to-peer (PP) systems have become

More information

Handling Churn in a DHT

Handling Churn in a DHT Handling Churn in a DHT Sean Rhea, Dennis Geels, Timothy Roscoe, and John Kubiatowicz UC Berkeley and Intel Research Berkeley What s a DHT? Distributed Hash Table Peer-to-peer algorithm to offering put/get

More information

Peer-to-peer systems and overlay networks

Peer-to-peer systems and overlay networks Complex Adaptive Systems C.d.L. Informatica Università di Bologna Peer-to-peer systems and overlay networks Fabio Picconi Dipartimento di Scienze dell Informazione 1 Outline Introduction to P2P systems

More information

Unit 8 Peer-to-Peer Networking

Unit 8 Peer-to-Peer Networking Unit 8 Peer-to-Peer Networking P2P Systems Use the vast resources of machines at the edge of the Internet to build a network that allows resource sharing without any central authority. Client/Server System

More information

Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. What We Want. Today s Question. What We Want. What We Don t Want C 1

Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. What We Want. Today s Question. What We Want. What We Don t Want C 1 Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on

More information

Scalability In Peer-to-Peer Systems. Presented by Stavros Nikolaou

Scalability In Peer-to-Peer Systems. Presented by Stavros Nikolaou Scalability In Peer-to-Peer Systems Presented by Stavros Nikolaou Background on Peer-to-Peer Systems Definition: Distributed systems/applications featuring: No centralized control, no hierarchical organization

More information

Overlay and P2P Networks. Introduction and unstructured networks. Prof. Sasu Tarkoma

Overlay and P2P Networks. Introduction and unstructured networks. Prof. Sasu Tarkoma Overlay and P2P Networks Introduction and unstructured networks Prof. Sasu Tarkoma 14.1.2013 Contents Overlay networks and intro to networking Unstructured networks Overlay Networks An overlay network

More information

Badri Nath Rutgers University

Badri Nath Rutgers University lookup services Badri Nath Rutgers University badri@cs.rutgers.edu 1. CAN: A scalable content addressable network, Sylvia Ratnasamy et.al. SIGCOMM 2001 2. Chord: A scalable peer-to-peer lookup protocol

More information

A Survey of Peer-to-Peer Content Distribution Technologies

A Survey of Peer-to-Peer Content Distribution Technologies A Survey of Peer-to-Peer Content Distribution Technologies Stephanos Androutsellis-Theotokis and Diomidis Spinellis ACM Computing Surveys, December 2004 Presenter: Seung-hwan Baek Ja-eun Choi Outline Overview

More information

Distributed Systems. 16. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2017

Distributed Systems. 16. Distributed Lookup. Paul Krzyzanowski. Rutgers University. Fall 2017 Distributed Systems 16. Distributed Lookup Paul Krzyzanowski Rutgers University Fall 2017 1 Distributed Lookup Look up (key, value) Cooperating set of nodes Ideally: No central coordinator Some nodes can

More information

Motivation for peer-to-peer

Motivation for peer-to-peer Peer-to-peer systems INF 5040 autumn 2015 lecturer: Roman Vitenberg INF5040, Frank Eliassen & Roman Vitenberg 1 Motivation for peer-to-peer Ø Inherent restrictions of the standard client/ server model

More information

416 Distributed Systems. Mar 3, Peer-to-Peer Part 2

416 Distributed Systems. Mar 3, Peer-to-Peer Part 2 416 Distributed Systems Mar 3, Peer-to-Peer Part 2 Scaling Problem Millions of clients server and network meltdown 2 P2P System Leverage the resources of client machines (peers) Traditional: Computation,

More information

Peer-to-Peer Streaming Systems. Behzad Akbari

Peer-to-Peer Streaming Systems. Behzad Akbari Peer-to-Peer Streaming Systems Behzad Akbari 1 Outline Introduction Scaleable Streaming Approaches Application Layer Multicast Content Distribution Networks Peer-to-Peer Streaming Metrics Current Issues

More information

C 1. Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. Today s Question. What We Want. What We Want. What We Don t Want

C 1. Last Time. CSE 486/586 Distributed Systems Distributed Hash Tables. Today s Question. What We Want. What We Want. What We Don t Want Last Time Distributed Systems Distributed Hash Tables Evolution of peer-to-peer Central directory (Napster) Query flooding (Gnutella) Hierarchical overlay (Kazaa, modern Gnutella) BitTorrent Focuses on

More information

Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007

Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007 Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007 Administrivia Quiz date Remaining lectures Interim report PS 3 Out Friday, 1-2 problems 2 Structured vs. Unstructured Overlays Structured

More information

DISTRIBUTED SYSTEMS CSCI 4963/ /4/2015

DISTRIBUTED SYSTEMS CSCI 4963/ /4/2015 1 DISTRIBUTED SYSTEMS CSCI 4963/6963 12/4/2015 2 Info Quiz 7 on Tuesday. Project 2 submission URL is posted on the web site Submit your source code and project report (PDF!!!) in a single zip file. If

More information

Peer-to-Peer Systems and Distributed Hash Tables

Peer-to-Peer Systems and Distributed Hash Tables Peer-to-Peer Systems and Distributed Hash Tables CS 240: Computing Systems and Concurrency Lecture 8 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Selected

More information

Advanced Distributed Systems. Peer to peer systems. Reference. Reference. What is P2P? Unstructured P2P Systems Structured P2P Systems

Advanced Distributed Systems. Peer to peer systems. Reference. Reference. What is P2P? Unstructured P2P Systems Structured P2P Systems Advanced Distributed Systems Peer to peer systems Karl M. Göschka Karl.Goeschka@tuwien.ac.at http://www.infosys.tuwien.ac.at/teaching/courses/ AdvancedDistributedSystems/ What is P2P Unstructured P2P Systems

More information

CPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University

CPSC 426/526. P2P Lookup Service. Ennan Zhai. Computer Science Department Yale University CPSC 4/5 PP Lookup Service Ennan Zhai Computer Science Department Yale University Recall: Lec- Network basics: - OSI model and how Internet works - Socket APIs red PP network (Gnutella, KaZaA, etc.) UseNet

More information

Scalable overlay Networks

Scalable overlay Networks overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent

More information

An Expresway over Chord in Peer-to-Peer Systems

An Expresway over Chord in Peer-to-Peer Systems An Expresway over Chord in Peer-to-Peer Systems Hathai Tanta-ngai Technical Report CS-2005-19 October 18, 2005 Faculty of Computer Science 6050 University Ave., Halifax, Nova Scotia, B3H 1W5, Canada An

More information

Peer-to-Peer Signalling. Agenda

Peer-to-Peer Signalling. Agenda Peer-to-Peer Signalling Marcin Matuszewski marcin@netlab.hut.fi S-38.115 Signalling Protocols Introduction P2P architectures Skype Mobile P2P Summary Agenda 1 Introduction Peer-to-Peer (P2P) is a communications

More information

CDNs and Peer-to-Peer

CDNs and Peer-to-Peer This Lecture This will be a why lecture, not a how to one CDNs and Peer-to-Peer EECS 89 Computer Networks http://www.eecs.umich.edu/~zmao/eecs89 Z. Morley Mao Tuesday Nov 9, Emphasis is on why these developments

More information

Peer-to-Peer Applications Reading: 9.4

Peer-to-Peer Applications Reading: 9.4 Peer-to-Peer Applications Reading: 9.4 Acknowledgments: Lecture slides are from Computer networks course thought by Jennifer Rexford at Princeton University. When slides are obtained from other sources,

More information

! Naive n-way unicast does not scale. ! IP multicast to the rescue. ! Extends IP architecture for efficient multi-point delivery. !

! Naive n-way unicast does not scale. ! IP multicast to the rescue. ! Extends IP architecture for efficient multi-point delivery. ! Bayeux: An Architecture for Scalable and Fault-tolerant Wide-area Data Dissemination ACM NOSSDAV 001 Shelley Q. Zhuang, Ben Y. Zhao, Anthony D. Joseph, Randy H. Katz, John D. Kubiatowicz {shelleyz, ravenben,

More information

L3S Research Center, University of Hannover

L3S Research Center, University of Hannover , University of Hannover Structured Peer-to to-peer Networks Wolf-Tilo Balke and Wolf Siberski 3..6 *Original slides provided by K. Wehrle, S. Götz, S. Rieche (University of Tübingen) Peer-to-Peer Systems

More information

12/5/16. Peer to Peer Systems. Peer-to-peer - definitions. Client-Server vs. Peer-to-peer. P2P use case file sharing. Topics

12/5/16. Peer to Peer Systems. Peer-to-peer - definitions. Client-Server vs. Peer-to-peer. P2P use case file sharing. Topics // Topics Peer to Peer Systems Introduction Client-server vs peer to peer Peer-to-peer networks Routing Overlays Structured vs unstructured Example PP Systems Skype login server Peer-to-peer - definitions

More information

Distributed Systems. peer-to-peer Johan Montelius ID2201. Distributed Systems ID2201

Distributed Systems. peer-to-peer Johan Montelius ID2201. Distributed Systems ID2201 Distributed Systems ID2201 peer-to-peer Johan Montelius 1 Idéa use resources in edge of network computing storage communication 2 Computing 3 seti@home central server millions of clients hundred of thousands

More information

Topics in P2P Networked Systems

Topics in P2P Networked Systems 600.413 Topics in P2P Networked Systems Week 3 Applications Andreas Terzis Slides from Ion Stoica, Robert Morris 600.413 Spring 2003 1 Outline What we have covered so far First generation file-sharing

More information

Today. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables

Today. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables Peer-to-Peer Systems and Distributed Hash Tables COS 418: Distributed Systems Lecture 7 Today 1. Peer-to-Peer Systems Napster, Gnutella, BitTorrent, challenges 2. Distributed Hash Tables 3. The Chord Lookup

More information

Arvind Krishnamurthy Fall 2003

Arvind Krishnamurthy Fall 2003 Overlay Networks Arvind Krishnamurthy Fall 003 Internet Routing Internet routing is inefficient: Does not always pick the lowest latency paths Does not always pick paths with low drop rates Experimental

More information

Distributed Hash Tables: Chord

Distributed Hash Tables: Chord Distributed Hash Tables: Chord Brad Karp (with many slides contributed by Robert Morris) UCL Computer Science CS M038 / GZ06 12 th February 2016 Today: DHTs, P2P Distributed Hash Tables: a building block

More information

Peer-to-Peer Systems. Chapter General Characteristics

Peer-to-Peer Systems. Chapter General Characteristics Chapter 2 Peer-to-Peer Systems Abstract In this chapter, a basic overview is given of P2P systems, architectures, and search strategies in P2P systems. More specific concepts that are outlined include

More information

Making Gnutella-like P2P Systems Scalable

Making Gnutella-like P2P Systems Scalable Making Gnutella-like P2P Systems Scalable Y. Chawathe, S. Ratnasamy, L. Breslau, N. Lanham, S. Shenker Presented by: Herman Li Mar 2, 2005 Outline What are peer-to-peer (P2P) systems? Early P2P systems

More information

CRESCENDO GEORGE S. NOMIKOS. Advisor: Dr. George Xylomenos

CRESCENDO GEORGE S. NOMIKOS. Advisor: Dr. George Xylomenos CRESCENDO Implementation of Hierarchical Chord (Crescendo), according to Canon paradigm and evaluation, via simulation over realistic network topologies, of the Crescendo's advantages in comparison to

More information

Searching for Shared Resources: DHT in General

Searching for Shared Resources: DHT in General 1 ELT-53206 Peer-to-Peer Networks Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original

More information

Searching for Shared Resources: DHT in General

Searching for Shared Resources: DHT in General 1 ELT-53207 P2P & IoT Systems Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original

More information

Department of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing

Department of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing Department of Computer Science Institute for System Architecture, Chair for Computer Networks File Sharing What is file sharing? File sharing is the practice of making files available for other users to

More information

CSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing. March 8, 2016 Prof. George Porter

CSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing. March 8, 2016 Prof. George Porter CSE 124 Finding objects in distributed systems: Distributed hash tables and consistent hashing March 8, 2016 rof. George orter Outline Today: eer-to-peer networking Distributed hash tables Consistent hashing

More information

Peer to Peer I II 1 CS 138. Copyright 2015 Thomas W. Doeppner, Rodrigo Fonseca. All rights reserved.

Peer to Peer I II 1 CS 138. Copyright 2015 Thomas W. Doeppner, Rodrigo Fonseca. All rights reserved. Peer to Peer I II 1 Roadmap This course will feature key concepts in Distributed Systems, often illustrated by their use in example systems Start with Peer-to-Peer systems, which will be useful for your

More information

Peer-peer and Application-level Networking. CS 218 Fall 2003

Peer-peer and Application-level Networking. CS 218 Fall 2003 Peer-peer and Application-level Networking CS 218 Fall 2003 Multicast Overlays P2P applications Napster, Gnutella, Robust Overlay Networks Distributed Hash Tables (DHT) Chord CAN Much of this material

More information

P2P: Distributed Hash Tables

P2P: Distributed Hash Tables P2P: Distributed Hash Tables Chord + Routing Geometries Nirvan Tyagi CS 6410 Fall16 Peer-to-peer (P2P) Peer-to-peer (P2P) Decentralized! Hard to coordinate with peers joining and leaving Peer-to-peer (P2P)

More information

*Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen)

*Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen) Distributed Hash Tables (DHT) Jukka K. Nurminen *Adapted from slides provided by Stefan Götz and Klaus Wehrle (University of Tübingen) The Architectures of st and nd Gen. PP Client-Server Peer-to-Peer.

More information

Advanced Computer Networks

Advanced Computer Networks Advanced Computer Networks P2P Systems Jianping Pan Summer 2007 5/30/07 csc485b/586b/seng480b 1 C/S vs P2P Client-server server is well-known server may become a bottleneck Peer-to-peer everyone is a (potential)

More information

Peer-to-Peer Architectures and Signaling. Agenda

Peer-to-Peer Architectures and Signaling. Agenda Peer-to-Peer Architectures and Signaling Juuso Lehtinen Juuso@netlab.hut.fi Slides based on presentation by Marcin Matuszewski in 2005 Introduction P2P architectures Skype Mobile P2P Summary Agenda 1 Introduction

More information

Chapter 6 PEER-TO-PEER COMPUTING

Chapter 6 PEER-TO-PEER COMPUTING Chapter 6 PEER-TO-PEER COMPUTING Distributed Computing Group Computer Networks Winter 23 / 24 Overview What is Peer-to-Peer? Dictionary Distributed Hashing Search Join & Leave Other systems Case study:

More information

CS 268: Lecture 22 DHT Applications

CS 268: Lecture 22 DHT Applications CS 268: Lecture 22 DHT Applications Ion Stoica Computer Science Division Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, CA 94720-1776 (Presentation

More information

Distributed Hash Tables

Distributed Hash Tables Distributed Hash Tables CS6450: Distributed Systems Lecture 11 Ryan Stutsman Material taken/derived from Princeton COS-418 materials created by Michael Freedman and Kyle Jamieson at Princeton University.

More information

P2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli

P2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli P2P Applications Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli Server-based Network Peer-to-peer networks A type of network

More information

INF5071 Performance in distributed systems: Distribution Part III

INF5071 Performance in distributed systems: Distribution Part III INF5071 Performance in distributed systems: Distribution Part III 5 November 2010 Client-Server Traditional distributed computing Successful architecture, and will continue to be so (adding proxy servers)

More information

CSE 486/586 Distributed Systems

CSE 486/586 Distributed Systems CSE 486/586 Distributed Systems Distributed Hash Tables Slides by Steve Ko Computer Sciences and Engineering University at Buffalo CSE 486/586 Last Time Evolution of peer-to-peer Central directory (Napster)

More information

PEER-TO-PEER NETWORKS, DHTS, AND CHORD

PEER-TO-PEER NETWORKS, DHTS, AND CHORD PEER-TO-PEER NETWORKS, DHTS, AND CHORD George Porter May 25, 2018 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative Commons license

More information

P2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili

P2P Network Structured Networks: Distributed Hash Tables. Pedro García López Universitat Rovira I Virgili P2P Network Structured Networks: Distributed Hash Tables Pedro García López Universitat Rovira I Virgili Pedro.garcia@urv.net Index Introduction to DHT s Origins of structured overlays Case studies Chord

More information

Distributed lookup services

Distributed lookup services Distributed lookup services lookup services Badri Nath Rutgers University badri@cs.rutgers.edu A set of nodes cooperating Peers Run special purpose algorithms/software Doesn t have to be deployed at every

More information