Distributed Data Management
|
|
- Jeffrey Robertson
- 5 years ago
- Views:
Transcription
1 Distributed Data Management Wolf-Tilo Balke Christoph Lofi Institut für Informationssysteme Technische Universität Braunschweig
2 8.0 Content Provisioning 8.0 Content Distribution 8.1 Swarming 8.2 BitTorrent 8.3 Anonymous P2P 8.4 Tornado Codes Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 2
3 8.1 Swarming Sometimes large amounts of data have to be distributed over networks Software updates, video on demand, etc. Early approaches: Napster, Gnutella, Fasttrack, Kaazaa, Use P2P network to locate a node offering the requested content Download whole file from a single peer If download fails: repeat search, resume download from alternative source Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 3
4 8.1 Swarming Issues Poor performance due to asymmetric uplink/downlink bandwidth Most common network home network connection technology: ADSL Asynchronous Digital Subscriber Line e.g. ADSL kb/sec download, 1024 kb/sec upload No load distribution Popular files may have extremely low download speed due to congestion of the offering node Low reliability (except for small files) Connected glitches may severely hamper download Frequent re-connects and resumes necesarry Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 4
5 8.1 Swarming Idea: Chunks Split large files into small chunks Assign hash values to chunks Identification: simple and deterministic labeling of chunks Transfer Protection: download chunk and compute hash Compare computed hash with hash provided by offering peer If comparison fails, a transfer error occurred reload chunk Original File 0x9A3C 0x7C23 0x194F 0xDE6A Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 5
6 8.1 Swarming Parallelization Locate the swarm of all peers hosting the required file (and thus the required chunks) Download different chunks from different sources simultaneously Utilize upload capacity of multiple sources Overall download speed may thus exceed upload capabilities of individual sources Usually, upload capacity is the bottleneck due to asymmetrical connections Sources: Destination: Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 6
7 8.1 Swarming So called swarming strategy Download chunks in parallel from a swarm of peers Swarming Advantages Peer failures: no loss of files, only chunks Discord unfinished chunks and download them anew No complicated resume mechanism necessary Increased throughput Download chunks in parallel from different sources Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 7
8 8.1 Swarming Swarming Issues Chunk selection: in which order should chunks be requested from which peer Avoid scarcity Best overall availability? Fairness: how can the protocol ensure fair usage of bandwidth Avoid free-riding: all peers should contribute to the networks Bandwidth allocation: single pairs should not be overwhelmed by request while others are idling Systems implementing swarming BitTorrent Avalanche Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 8
9 8.2 BitTorrent BitTorrent Torrent = big stream Author: Bram Cohen, 2001 Protocol for swarming file distribution, no search features Designed for Protocol to quickly and decentrally distribute large content Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 9
10 8.2 BitTorrent Implements swarming strategy for content distribution Especially suited for flash crowds, i.e. content which is high in demand for a short period of time Central components Web server for search (torrent site) Classic web server maintains list of available content (so called torrents ) Provides search functionalities Content is represented as a torrent file containing required meta data e.g. address of the tracker Tracker for peer coordination A tracker is a centralized service maintaining the peer swarms i.e. which peers have which chunks of which torrents Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 10
11 8.2 BitTorrent Workflow for download A user uses a torrent site to obtain a torrent file Torrent file contains content meta data The user s node connects to the responsible tracker Tracker URL and content identifiers ( info hash ) are provided by torrent file Node registers itself with the tracker and the corresponding torrent i.e. joins the swarm Node obtains a list of all pairs offering the torrent Contact some swarm peers Obtain a list of available chunks Download chunks from peers Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 11
12 8.2 BitTorrent Each offered file is split in chunks between 32KB and 4MB each Each chunk is identified by an 160-Bit SHA-Hash With respect to a certain torrent, each pair cam full fill one of the following roles Seeders Have all chunks of the torrent and are actively seeding (uploading) those chunks to the swarm Leechers Do not have all chunks of a torrent Download missing chunks from other leechers or seeders Upload chunks to other leecher There is no download-only role Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 12
13 8.2 BitTorrent Torrent File metadata structure Describes the files in the torrent URL of tracker File name File length Piece length SHA-1 hashes of pieces Allow peers to verify integrity Creation date An info hash is created from some fields of the torrent file This hash uniquely identifies a torrent Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 13
14 8.2 BitTorrent Fields being hashed in InfoHash Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 14
15 8.2 BitTorrent torrent site tracker new leecher peer other swarm peers torrent file register swarm peer list request info chunk information request chunk send chunk Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 15
16 8.2 BitTorrent :-) Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 16
17 8.2 BitTorrent Which chunk next? Priority Actives Finish active chunks Rarest First Improves availability of rare chunks Delays download of common chunks Random First Chunk Get first chunk quickly (rarest chunk probably slow to get) Endgame Mode Send requests for last chunks to all known peers End of download not stalled by slow peers Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 17
18 Game Theory Basic Ideas of Game Theory Game theory offers a general theory of strategic behavior Described in mathematical form Situations in which players may choose different actions to maximize their returns Situations in which strategic interactions among rational players produce outcomes with respect to the players preferences The outcomes might not have been intended by any of them Plays an important role in Modern economics Decision theory Multi-agent systems Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 18
19 Game Theory Early game theory tries to explain the optimal strategy in two-person interactions. von Neumann and Morgenstern, 1944 Initially: zero-sum games Expected utility hypothesis Players will rationally decide for the option with the highest expected outcome Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 19
20 Game Theory John Nash Works in game theory and differential geometry Non-zero-sum games Nash equilibrium 1950 Strategic equilibrium in which no player gains any advantage when changing strategies (while knowing the opponents startegy) 1994 Nobel Prize in Economics Harsanyi and Selten Incomplete information Also 1994 Nobel Prize in Economics Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 20
21 Game Theory Games Situations are treated as games. Rules The rules of the game state which actions and decisions are possible Player's Strategies Plan for actions in each possible situation in the game Player's Payoffs Is a players expected gain or loss when winning or loosing in a particular situation Dominant Strategy If players best strategy doesn t depend on what other players do Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 21
22 Game Theory Famous example: Prisoners Dilemma A and B are arrested by the police during a robbery They are interrogated in separate cells Unable to communicate with each other Following conditions are known If they both resist interrogation and proclaim their mutual innocence, they both will get off with a three year sentence for robbery If one of them confesses to the entire string of robberies and the other does not, the confessor will be rewarded with a light, one year sentence and while the other will get a severe eight year sentence If they both confess, then the judge will sentence both to a moderate four years in prison Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 22
23 Game Theory Prisoner Dilemma Possible outcomes A - Confession A - No Confession B - Confession 4 years each 8 years for A 1 year for B B - No Confession 1 year for A 8 years for B 3 years each Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 23
24 Game Theory Decision Tree of A A B Confesses B does not confesses A: Confess A: Does Not Confess A: Confess A: Does Not Confess 4 Years in Prison 8 Years in Prison 1 Year in Prison 3 Years in Prison Best strategy The dominant strategy for A is to confess No matter what B does, confessing is better choice Nash equilibrium: both A and B will confess Also, dominant strategy of B is to confess Best strategy Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 24
25 Game Theory A repeated game Game that the same players play more than once Differ from one-shot games because people's current actions can depend on the past behavior of other players. Cooperation is encouraged Book recommendation Thinking strategically by A.Dixit and B Nalebuff German translation: Spieltheorie für Einsteiger Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 25
26 8.2 BitTorrent We can employ game theory for designing a swarming protocol Swarming Game Decisions While seeding, who should get the chunks? While leeching, from whom to download chunks? Goal Available resources should be optimally used Free-riding should be prevented Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 26
27 8.2 BitTorrent Possible strategy in repeated games: Tit for Tat An player using this strategy will initially cooperate Player will adapt to opponent If the opponent previously was cooperative, the agent is cooperative. If not, the agent is not. Depends on four conditions Unless provoked, the agent will always cooperate If provoked, the agent will retaliate The agent is quick to forgive The agent must have a good chance of competing against the opponent more than once Get-to-know each other Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 27
28 8.2 BitTorrent BitTorrent uses a so-called choking mechanism for distributing chunks Base idea Prefer uploading chunks to peers which also offered chunks for download i.e. aim for bi-directional communication channels Bi-directional communication will benefit the whole swarm most Tit-for-tat Punish peers which seem to be free-riding i.e. who only download but provide no upload Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 28
29 8.2 BitTorrent Choking as leecher Open a bi-directional transfer to another leecher Mutually exchange missing chunks If a peer does not upload any chunks for more than a minute, choke it Temporary refuse to upload Downloading continues as usual TCP Connection is kept open No Setup costs TCP congestion control Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 29
30 8.2 BitTorrent Using only this choking mechanism may endanger the health of the swarm New leechers will automatically be choked because they cannot offer upload Two nodes which would have a good connection won t use it as no node takes initiative A node may be choked by all other nodes due to unlucky circumstances / network weaknesses Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 30
31 8.2 BitTorrent Solution: Optimistic Unchoking Randomly initiate a new connection to a currently unconnected leecher in the swarm and start uploading Take initiative Hope for a good cooperation e.g. that this new node provides a high upload rate in the bi-directional transfer Allows finding better peers Allows new peers to integrate themselves in the swarm Other peers voluntarily start uploading to them Randomly unchoke some currently choked connections Quick to forgive Helps locked-out peers to return to the swarm If a leecher is currently choked by all its peers, it initiates even more unchoking connection Anti-Snubbing Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 31
32 8.2 BitTorrent Open issue: upload-only choking Once a download is complete, no bi-directional transfer connections are required anymore by that peer Peer becomes a seeder Which nodes to upload to? Seeding Policy Upload to those peers with the best upload / download ratio Probable Advantages Ensures that chunks are replicated faster within the swarm Leechers that have a good upload rates are probably not being served by others Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 32
33 8.2 BitTorrent Download chunks in parallel Look for the rarest pieces Verify each chunk by checking hash, download again if hash fails Advertise received pieces to all connected peers leecher B leecher A I have! seed leecher C Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 33
34 8.2 BitTorrent Periodically calculate data-receiving rates Upload to (unchoke) the fastest downloaders Optimistic unchoking Periodically select a peer at random and upload to it Continuously look for the fastest partners leecher B leecher A seed leecher D leecher C Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 34
35 8.2 BitTorrent Remember: Bit Torrent uses centralized trackers for managing peer lists Tracker Issues Single Point of Failure and Attack Scalability Piratebay tracker nearly overloaded (>5 Mio. Peers) Solution: Decentralized Tracker Replace with DHT (Kademlia) Does not tackle distributed search Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 35
36 8.2 BitTorrent Kademlia DHT Tracker Each torrent is identified by its infohash All BitTorrent nodes using an compatible client may join the DHT tracker Not part of the core BitTorrent protocol Authors of client usually also provide bootstrapping nodes to the DHT tracker The DHT takes over the trackers responsibility DHT Key-Value pairs Key: info hash Value: swarm peer listing Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 36
37 8.2 BitTorrent Kademlia (as a generic DHT protocol) Kademlia also uses a SHA Bit addressed ring hashing data and nodes Similar to pastry, but uses a more sophisticated routing mechanism requiring less maintenance Each key-value pair is stored redundantly on multiple nodes Usually k nodes neighboring the node which is responsible for the range of the infohash Nodes storing a certain key frequently synchronize their data with the other responsible peers Peer arrivals and departures can be tolerated without data loss Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 37
38 8.2 BitTorrent BitTorrent DHT: a new node joins Obtain the torrent infohash e.g. from torrent file or from a magnet link Contact a bootstrap node and join the DHT tracker Take over responsibility for a certain range of torrents i.e. host some (redundant) peer listings for some torrent swarms New node announces itself i.e. contacts some nodes hosting the peer lists for the required torrent The new node is added to the respective peer listing The node obtains the full peer list No central authority / tracker is required Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 38
39 8.2 BitTorrent Peer Exchange (PEX) and Multi-Tracker To further increase the performance and resilience of a torrent, multiple trackers can be used Easiest case Torrent is registered with multiple trackers which are all explicitly specified in the torrent file More complex solution: Peer Exchange Start connecting to any one tracker Ask other connected peers in the swarm for additional peers and / or trackers Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 39
40 8.2 BitTorrent PEX Example Obtain a torrent and connect to tracker i.e. official Ubuntu torrent providing two trackers Obtain peer lists Additionally, connect to DHT tracker Obtain even more peers Peers using the same torrent but are trackerless or use DHT and another tracker Start peer exchange Again, obtain more peers i.e. receive peers not using DHT from a DHT node which is using, e.g. the openbittorrent-tracker Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 40
41 Client Example μtorrent 41
42 Client Example μtorrent Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 42
43 Client Example μtorrent 43
44 Tracker Peer cache IP, port, peer id State information Completed Downloading Clients report status periodically to tracker Returns random list 50 random leechers/seeds Client first contacts of them and more if some do not respond Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 44
45 Tracker Info Hashes Info Hashes Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 45
46 Magnet Links Recently, magnet links have become quite popular Magnet links define an URI scheme for any content located in a P2P network May encode all data necessary to identify or find content, e.g. Protocol, Name, Size, Protocol-Specific metadata, etc Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 46
47 Magnet Links Example magnet link for BitTorrent magnet:?xt=urn:btih:3e16157f0879eb43e9e51f45 d485feff90a77283 &dn=ubuntu lts+x32 &tr=http%3a%2f%2ftracker.openbittorrent.com %2Fannounce Display Name exact Topic BitTorrent InfoHash InfoHash TRacker URL Other protocols might include search keywords, web sources, bootstrap nodes, etc Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 47
48 8.3 Anonymous P2P BitTorrent is provides absolutely no anonymity Peer list of a torrent contain all participating nodes IP address, port, sometimes upload/download ratio, etc. Thus, it is very easy to identify all people downloading / uploading to a torrent User behavior can be tracked quite easily by simply introducing a spy node into the swarm Privacy implication Also, no pure downloading is possible BitTorrent as download-only nodes will be choked Possibly legal implication for copyrighted content Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 48
49 8.3 Anonymous P2P Real anonymity is hard to archive Nodes need to communicate and contact each other Identities (addresses must be known) However, the number of nodes knowing a nodes identity can be limited to only trusted nodes So-called dark nets Base idea Only connect to a few trusted friend nodes Never communicate directly with a non-friend Friends forward any message anonymously to their friends If network is designed correctly, most parts should be reachable via friend-of-a-friend routing Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 49
50 8.3 Anonymous P2P Two notable systems Freenet: Pure P2P network using small worldproperties and anonymous routing OneSwarm: BitTorrent extension based on friend-to-friend-routing Friend-To-Friend Routing e.g. B passes a message from A to C C does not know that message originated from A A does not know that B passes message to C A requester B C provider Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 50
51 8.3 Anonymous P2P If the request s time-to-live expires or a node does not have neighbors to send the file to, a backtracking request failed message is sent If the request is successful, the file is sent back via the routing nodes and each node saves the file and adds the sending node s address to its local routing table i.e., frequently requested files are replicated If the routing table is full, the least recently used (LRU) entry is evicted Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 51
52 8.3 Anonymous P2P If the request s time-to-live expires or a node does not have neighbors to send the file to, a backtracking request failed message is sent If the request is successful, the file is sent back via the routing nodes and each node saves the file and adds the sending node s address to its local routing table i.e., frequently requested files are replicated If the routing table is full, the least recently used (LRU) entry is evicted Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 52
53 8.3 Anonymous P2P Example of Freenet Routing B s routing table Key Pointer D s routing table 6 C Key Pointer 15 D 9 F? key=9 A 9 F 9? 9 B 9? 9? Sorry! 9 1 E D 9? 9 F key = 9 C s routing table empty C E Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 53
54 8.4 Erasure Codes When transferring content, various failures may occur Transmission failures (e.g., packet loss) Noisy channels Storage failures (e.g. hardware breakdown, churn) Error correcting codes can help battling failures Basic idea: Encode information of length n in (n + k) symbols k-symbol redundancy! The information can be recovered from any n of the (n + k) symbols Examples Check sums detect and correct errors in noisy channels RAID-5 storage systems (Parity bits) Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 54
55 8.4 Erasure Codes Thought Experiment: Err-mail Err-mail works just like , except About half of all the mail gets lost Messages longer than 5 characters are illegal Sending a message is very expensive (similar to air-mail) Alice wants to send her telephone number (555629) to Bob Naïve approach Split into two packets (555, 629) and send separately Chances are, one of them gets lost Even repetitive sending doesn t help much, Bob will receive probably redundant packets Acknowledge messages by Bob are an option, but expensive Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 55
56 8.4 Erasure Codes Alice devises the following scheme She breaks her telephone number up into two parts a = 555, b = 629 Sends 2 messages "A = 555" and "B = 629" to Bob. She constructs a linear function, f n = a + b a n 1 in this case f(n) = (n 1) She computes the values f(3), f(4), and f(5), and then transmits three redundant messages "C = 703", "D = 777" and "E = 851". Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 56
57 8.4 Erasure Codes Bob knows that the form of f(n) is f(n) = a + (b a)(n 1), where a and b are the two parts of the telephone number Now suppose Bob receives "D = 777" and "E = 851" Bob can reconstruct Alice's phone number by computing the values of a and b from the values (f(4) and f(5)) Bob can perform this procedure using any two err-mails, so the erasure code in this example has a rate of 40% Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 57
58 8.4 Erasure Codes Tornado Codes are an important class of erasure codes for practical applications Characteristics Easy coding/decoding: linear codes with explicit construction Fast coding/decoding: each check bit depends on only a few message bits M. Luby, M. Mitzenmacher, M. A. Shokrollahi, D. A. Spielman, V. Stemann: Practical Loss-Resilient Codes. ACM Symposium on the Theory of Computing, 1997 J. W. Byers, M. Luby, M. Mitzenmacher: Accessing Multiple Mirror Sites in Parallel: Using Tornado Codes to Speed Up Downloads. INFOCOM 1999 Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 58
59 8.4 Erasure Codes Scenario Application sends a real-time data stream of symbols Network experiences unpredictable losses of at most a fraction of p symbols We know the positions of the lost bits (packet indexes) Insurance policy Let n be the block length Instead of sending n symbols, place (1 p)n symbols in each block Fill block to length n with pn redundant symbols Scheme provides optimal loss protection if message symbols can be recovered from any set of (1 p)n symbols in the block Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 59
60 8.4 Erasure Codes Forward Error Correction Interleave message bits and check bits in a stream n (1-p)n pn Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 60
61 8.4 Erasure Codes Properties of a good code There should be few check bits Linear time encoding Average degree on the left should be a small constant Easy error detection/decoding Each set of message bits should influence many check bits Existence of unshared neighbors Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 61
62 8.4 Tornado Codes Tornado code model: Bipartite Graph Each message bit is used in only a few check bits Low degree bipartite graph Check bits are computed as orthogonal combination of message bits (usually XOR) Message bits Check bits c 6 = m 3 m 7 Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 62
63 8.4 Tornado Codes Properties Expansion: every small subset (k n) on left has many ( k) neighbors on right Low degree not technically part of the definition, but typically assumed k bits (k αn) bk bits Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 63
64 8.4 Tornado Codes Important parameters: size(n), degree(d), expansion(b) Randomized constructions A random d-regular graph is an expander with a high probability Construct by choosing d random perfect matchings Perfect matching: all nodes on the left side get exactly one edge to a node on the right side Repeat d times: every node on the left side has d edges to the right side Time consuming and cannot be stored compactly Explicit constructions Cayley graphs, Ramanujan graphs etc Typical technique start with a small expander, apply operations to increase its size Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 64
65 8.4 Erasure Codes Will use d-regular bipartite graphs with (1 p)n nodes on the left and pn on the right (e.g., p = 0.5) Will need b > d/2 expansion m 1 degree = d m 2 m 3 c 1 degree = 2d c pn m (1-p)n Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 65
66 8.4 Tornado Codes Encoding Why is it linear time? m 1 Computes the sum modulo 2 of its neighbors m 2 m 3 c 1 c pn m (1-p)n Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 66
67 8.4 Tornado Codes Decoding Assume that all the check bits are intact Find a check bit such that only one of its neighbors is erased (an unshared neighbor) Fix the erased code, and repeat m 1 m 2 c 1 c pn m (1-p)n Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 67
68 8.4 Tornado Codes Decoding Need to ensure that we can always find a check bit Unshared neighbors property Consider the set of corrupted message bit and their neighbors. Suppose this set is small at least one message bit has an unshared neighbor. Can we always find unshared neighbors? Theorem: Expander graphs give us this property if b > d/2 unshared neighbor m 1 m 2 c 1 c pn m (1-p)n Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 68
69 8.4 Tornado Codes Cascading Use another bipartite graph to construct another level of check bits for the check bits Final level is encoded by some other code, e.g., Reed-Solomon k pk p 2 k Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 69
70 Content Provisioning Swarming & BitTorrent Segment a file into multiple chunks Download chunks from multiple peers in parallel Seeder and leecher peers form a swarm Increased throughput Faster dissemination of new content i.e. for countering flash crowds Main question: which chunks should be downloaded / uploaded to best benefit the whole swarm? Prevent free-riding Discourage parasitic behavior Reward cooperation Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 70
71 Content Provisioning Solution: use concepts from game theory Tit-for-tat strategy Encourages strong bi-directional links among leecher Choking If a node in a bidirectional pipe is not cooperative (provides upload bandwidth), choke it by refusing further uploads to that peer Optimistic Unchoke Randomly unchoke some choked connections Take initiative and voluntarily upload to an unconnected node May discover better and more reliable partners Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 71
72 Content Provisioning Erasure Codes Help securing message transfers and data storage Base idea Split payload into chunks Interleave payload chunks with redundant chunks which can be used to reconstruct the message in case of failures Tornado Code Popular erasure code implementation Uses linear functions to encode and decode message Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 72
73 Next Lecture Load Balancing & Data Durability Data caching, replication, etc. P2P and Databases Building database link systems on top of P2P Toward cloud storage Distributed Data Management Wolf-Tilo Balke Christoph Lofi IfIS TU Braunschweig 73
Distributed Data Management. Christoph Lofi Institut für Informationssysteme Technische Universität Braunschweig
Distributed Data Management Christoph Lofi Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 8.0 Content Provisioning 8.0 Content Distribution 8.1 Swarming
More informationDistributed Data Management
Distributed Data Management Profr. Dr. Wolf-Tilo Balke Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de Network Models and Content Provisioning Network Models
More informationP2P Content Distribution
, University of Hannover Wolf-Tilo Balke and Wolf Siberski 28.11.2007 *With slides from K. Wehrle (RWTH), A. Bharambe (CMU), P. Rodriguez, P. Chou (MSRC), Chiu (CUHK) Peer-to-Peer Systems and Applications,
More informationContent Distribution and BitTorrent [Based on slides by Cosmin Arad]
ID2210 - Distributed Computing, Peer-to-Peer and GRIDS Content Distribution and BitTorrent [Based on slides by Cosmin Arad] Today The problem of content distribution A popular solution: BitTorrent Underlying
More informationLecture 17: Peer-to-Peer System and BitTorrent
CSCI-351 Data communication and Networks Lecture 17: Peer-to-Peer System and BitTorrent (I swear I only use it for Linux ISOs) The slide is built with the help of Prof. Alan Mislove, Christo Wilson, and
More informationContent Overlays (continued) Nick Feamster CS 7260 March 26, 2007
Content Overlays (continued) Nick Feamster CS 7260 March 26, 2007 Administrivia Quiz date Remaining lectures Interim report PS 3 Out Friday, 1-2 problems 2 Structured vs. Unstructured Overlays Structured
More informationExtreme Computing. BitTorrent and incentive-based overlay networks.
Extreme Computing BitTorrent and incentive-based overlay networks BitTorrent Today we will focus on BitTorrent The technology really has three aspects A standard that BitTorrent client systems follow Some
More informationBitTorrent. Masood Khosroshahy. July Tech. Report. Copyright 2009 Masood Khosroshahy, All rights reserved.
BitTorrent Masood Khosroshahy July 2009 Tech. Report Copyright 2009 Masood Khosroshahy, All rights reserved. www.masoodkh.com Contents Contents 1 Basic Concepts 1 2 Mechanics 3 2.1 Protocols: Tracker and
More informationP2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli
P2P Applications Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Canale A-L Prof.ssa Chiara Petrioli Server-based Network Peer-to-peer networks A type of network
More informationCompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4. Xiaowei Yang
CompSci 356: Computer Network Architectures Lecture 21: Overlay Networks Chap 9.4 Xiaowei Yang xwy@cs.duke.edu Overview Problem Evolving solutions IP multicast Proxy caching Content distribution networks
More informationLecture 8: Application Layer P2P Applications and DHTs
Lecture 8: Application Layer P2P Applications and DHTs COMP 332, Spring 2018 Victoria Manfredi Acknowledgements: materials adapted from Computer Networking: A Top Down Approach 7 th edition: 1996-2016,
More informationP2P Applications. Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza
P2P Applications Reti di Elaboratori Corso di Laurea in Informatica Università degli Studi di Roma La Sapienza Versione originale delle slides fornita da Dora Spenza e Marco Barbera P2P Paradigm Late 80
More informationBitTorrent Optimization Techniques. (from various online sources)
BitTorrent Optimization Techniques (from various online sources) Announcement No recitation next week! Final review session Next Sunday (5/2) 5-7pm, GHC 4215 Let us know what you want at http://www.doodle.com/6qvsnubhmam2zkxp
More informationinternet technologies and standards
Institute of Telecommunications Warsaw University of Technology 25 internet technologies and standards Piotr Gajowniczek Andrzej Bąk Michał Jarociński Internet application layer peer-to-peer systems overview
More informationPeer-to-Peer Applications Reading: 9.4
Peer-to-Peer Applications Reading: 9.4 Acknowledgments: Lecture slides are from Computer networks course thought by Jennifer Rexford at Princeton University. When slides are obtained from other sources,
More informationCSE 486/586 Distributed Systems Peer-to-Peer Architectures
CSE 486/586 Distributed Systems eer-to-eer Architectures Steve Ko Computer Sciences and Engineering University at Buffalo CSE 486/586 Last Time Gossiping Multicast Failure detection Today s Question How
More informationDISTRIBUTED SYSTEMS CSCI 4963/ /4/2015
1 DISTRIBUTED SYSTEMS CSCI 4963/6963 12/4/2015 2 Info Quiz 7 on Tuesday. Project 2 submission URL is posted on the web site Submit your source code and project report (PDF!!!) in a single zip file. If
More informationDo incentives build robustness in BitTorrent?
Do incentives build robustness in BitTorrent? ronghui.gu@yale.edu Agenda 2 Introduction BitTorrent Overview Modeling Altruism in BitTorrent Building BitTyrant Evaluation Conclusion Introduction 3 MAIN
More informationP2P content distribution Jukka K. Nurminen
P2P content distribution Jukka K. Nurminen 1 V1-Filename.ppt / yyyy-mm-dd / Initials BitTorrent content downloading Efficient content distribution Bram Cohen, 2001 File divided into pieces Each recipient
More informationApplication Layer: P2P File Distribution
Application Layer: P2P File Distribution EECS 3214 Slides courtesy of J.F Kurose and K.W. Ross, All Rights Reserved 29-Jan-18 1-1 Chapter 2: outline 2.1 principles of network applications 2.2 Web and HTTP
More informationDepartment of Computer Science Institute for System Architecture, Chair for Computer Networks. File Sharing
Department of Computer Science Institute for System Architecture, Chair for Computer Networks File Sharing What is file sharing? File sharing is the practice of making files available for other users to
More informationBitTorrent. Internet Technologies and Applications
BitTorrent Internet Technologies and Applications Aims and Contents Aims Show benefits of P2P for file downloads Explain concepts of BitTorrent as a P2P file sharing application Show how BitTorrent protocol
More informationCS5412: TORRENTS AND TIT-FOR-TAT
1 CS5412: TORRENTS AND TIT-FOR-TAT Lecture VII Ken Birman BitTorrent 2 Widely used download technology Implementations specialized for setting Some focus on P2P downloads, e.g. patches Others focus on
More informationIntroduction to P P Networks
Introduction to P P Networks B Sc Florian Adamsky florianadamsky@iemthmde http://florianadamskyit/ cbd Internet Protocols and Applications SS B Sc Florian Adamsky IPA / Outline Introduction What is P P?
More informationLast Lecture SMTP. SUNY at Buffalo; CSE 489/589 Modern Networking Concepts; Fall 2010; Instructor: Hung Q. Ngo 1
Last Lecture SMTP SUNY at Buffalo; CSE 489/589 Modern Networking Concepts; Fall 2010; Instructor: Hung Q. Ngo 1 This Lecture Peer-to-Peer (P2P) Applications SUNY at Buffalo; CSE 489/589 Modern Networking
More informationPeer-to-Peer Networks 12 Fast Download
Peer-to-Peer Networks 12 Fast Download Arne Vater Technical Faculty Computer Networks and Telematics University of Freiburg IP Multicast Motivation - Transmission of a data stream to many receivers Unicast
More informationScalability of the BitTorrent P2P Application
Scalability of the BitTorrent P2P Application Kolja Eger, Ulrich Killat Hamburg University of Technology 5.Würzburger Workshop 8.-9. July 2005 Overview File dissemination in peer-to-peer (p2p) networks
More informationP2P content distribution
P2P content distribution T-110.7100 Applications and Services in Internet, Fall 2010 Jukka K. Nurminen 1 V1-Filename.ppt / yyyy-mm-dd / Initials Steps of content sharing Share content Find content Transfer
More informationPeer to Peer Systems and Probabilistic Protocols
Distributed Systems 600.437 Peer to Peer Systems & Probabilistic Protocols Department of Computer Science The Johns Hopkins University 1 Peer to Peer Systems and Probabilistic Protocols Lecture 11 Good
More information416 Distributed Systems. Mar 3, Peer-to-Peer Part 2
416 Distributed Systems Mar 3, Peer-to-Peer Part 2 Scaling Problem Millions of clients server and network meltdown 2 P2P System Leverage the resources of client machines (peers) Traditional: Computation,
More informationReti P2P per file distribution: BitTorrent
Reti P2P per file distribution: BitTorrent Corso di Applicazioni Telematiche A.A. 2008-09 Lezione n.22 Prof. Roberto Canonico Università degli Studi di Napoli Federico II Facoltà di Ingegneria P2p file-sharing
More informationIMPLEMENTATION AND EVALUATION OF THE MULTICAST FILE TRANSFER PROTOCOL (MCFTP)
IMPLEMENTATION AND EVALUATION OF THE MULTICAST FILE TRANSFER PROTOCOL (MCFTP) Masterarbeit der Philosophisch-naturwissenschaftlichen Fakultät der Universität Bern vorgelegt von Alican Gecyasar Dezember,
More informationCS 3516: Advanced Computer Networks
Welcome to CS 3516: Advanced Computer Networks Prof. Yanhua Li Time: 9:00am 9:50am M, T, R, and F Location: Fuller 320 Fall 2017 A-term 1 Some slides are originally from the course materials of the textbook
More informationBitTorrent from swarms to collectives
BitTorrent from swarms to collectives Some on-going research in the tribler team at TU Delft name: David Hales date: Sept. 21, 2009 event: ECCS 2009 Tribler team: 35 + collaborators tribler.org Tribler
More informationApplication-Layer Protocols Peer-to-Peer Systems, Media Streaming & Content Delivery Networks
COMP 431 Internet Services & Protocols Application-Layer Protocols Peer-to-Peer Systems, Media Streaming & Content Delivery Networks Jasleen Kaur February 14, 2019 Application-Layer Protocols Outline Example
More informationBitTorrent and CoolStreaming
BitTorrent and CoolStreaming Jukka K. Nurminen Data Communications Software (DCS) Lab, Department of Computer Science and Engineering, Aalto University Jukka K. Nurminen Aalto University P2P Networks BitTorrent
More informationGame Theory. Presented by Hakim Weatherspoon
Game Theory Presented by Hakim Weatherspoon Game Theory Main Question: Can we cheat (and get away with it)? BitTorrent P2P file distribution tool designed with incentives for contribution Users contribute
More informationA Case For OneSwarm. Tom Anderson University of Washington.
A Case For OneSwarm Tom Anderson University of Washington http://oneswarm.cs.washington.edu/ With: Jarret Falkner, Tomas Isdal, Alex Jaffe, John P. John, Arvind Krishnamurthy, Harsha Madhyastha and Mike
More informationMiddleware and Distributed Systems. Peer-to-Peer Systems. Peter Tröger
Middleware and Distributed Systems Peer-to-Peer Systems Peter Tröger Peer-to-Peer Systems (P2P) Concept of a decentralized large-scale distributed system Large number of networked computers (peers) Each
More informationChapter 2: Application layer
Chapter 2: Application layer 2. Principles of network applications app architectures app requirements 2.2 Web and HTTP 2.4 Electronic Mail SMTP, POP3, IMAP 2.5 DNS 2.6 P2P applications 2.7 Socket programming
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2018 Networking Laboratory P2P Applications Traditional P2P applications: for file sharing BitTorrent, Emule
More informationPeer to Peer Networks
Sungkyunkwan University Peer to Peer Networks Prepared by T. Le-Duc and H. Choo Copyright 2000-2017 Networking Laboratory Presentation Outline 2.1 Introduction 2.2 Client-Server Paradigm 2.3 Peer-To-Peer
More informationIntroduction to Distributed Computing Systems
Introduction to Distributed Computing Systems Alexandru Iosup Parallel and Distributed Systems Group Delft University of Technology Some material kindly provided by the PDS group. 1 What is a Distributed
More informationCooperation in Open Distributed Systems. Stefan Schmid
Cooperation in Open Distributed Systems Stefan Schmid T-Labs, Berlin, July 2, 2009 Distributed Systems 2008/9 Wireless: Many mobile phones today have WLAN (and even Skype) P2P: Olympic games 2008 live-broadcast
More informationCS 43: Computer Networks BitTorrent & Content Distribution. Kevin Webb Swarthmore College September 28, 2017
CS 43: Computer Networks BitTorrent & Content Distribution Kevin Webb Swarthmore College September 28, 2017 Agenda BitTorrent Cooperative file transfers Briefly: Distributed Hash Tables Finding things
More informationCooperative End-to-end content distribution. Márk Jelasity
Cooperative End-to-end content distribution Márk Jelasity Content distribution So far we looked at search Content distribution is about allowing clients (peers) to actually get a file or other data after
More informationLecture 21 P2P. Napster. Centralized Index. Napster. Gnutella. Peer-to-Peer Model March 16, Overview:
PP Lecture 1 Peer-to-Peer Model March 16, 005 Overview: centralized database: Napster query flooding: Gnutella intelligent query flooding: KaZaA swarming: BitTorrent unstructured overlay routing: Freenet
More informationCMSC 332 Computer Networks P2P and Sockets
CMSC 332 Computer Networks P2P and Sockets Professor Szajda Announcements Programming Assignment 1 is due Thursday Where are we? What sorts of problems are we having? 2 Recap SMTP is the language that
More informationDistributed Systems. peer-to-peer Johan Montelius ID2201. Distributed Systems ID2201
Distributed Systems ID2201 peer-to-peer Johan Montelius 1 Idéa use resources in edge of network computing storage communication 2 Computing 3 seti@home central server millions of clients hundred of thousands
More informationCS 3516: Computer Networks
Welcome to CS 3516: Computer Networks Prof. Yanhua Li Time: 9:00am 9:50am M, T, R, and F Location: AK219 Fall 2018 A-term 1 Some slides are originally from the course materials of the textbook Computer
More informationCS555: Distributed Systems [Fall 2017] Dept. Of Computer Science, Colorado State University
CS 555: DISTRIBUTED SYSTEMS [P2P SYSTEMS] Shrideep Pallickara Computer Science Colorado State University Frequently asked questions from the previous class survey Byzantine failures vs malicious nodes
More informationINTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY
INTERNATIONAL JOURNAL OF PURE AND APPLIED RESEARCH IN ENGINEERING AND TECHNOLOGY A PATH FOR HORIZING YOUR INNOVATIVE WORK PEER-TO-PEER FILE SHARING WITH THE BITTORRENT PROTOCOL APURWA D. PALIWAL 1, PROF.
More informationBitTorrent for Really Selfish Peers T4T Trading over Cycles
Semester Thesis BitTorrent for Really Selfish Peers T4T Trading over Cycles Roger Odermatt oroger@student.ethz.ch Professor: Dr. Roger Wattenhofer Advisor: Raphael Eidenbenz Department of Information Technology
More informationEfficient Content Delivery and Low Complexity Codes. Amin Shokrollahi
Efficient Content Delivery and Low Complexity Codes Amin Shokrollahi Content Goals and Problems TCP/IP, Unicast, and Multicast Solutions based on codes Applications Goal Want to transport data from a transmitter
More informationPeer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables CS 240: Computing Systems and Concurrency Lecture 8 Marco Canini Credits: Michael Freedman and Kyle Jamieson developed much of the original material. Selected
More informationPeer-to-peer systems and overlay networks
Complex Adaptive Systems C.d.L. Informatica Università di Bologna Peer-to-peer systems and overlay networks Fabio Picconi Dipartimento di Scienze dell Informazione 1 Outline Introduction to P2P systems
More informationToday. Why might P2P be a win? What is a Peer-to-Peer (P2P) system? Peer-to-Peer Systems and Distributed Hash Tables
Peer-to-Peer Systems and Distributed Hash Tables COS 418: Distributed Systems Lecture 7 Today 1. Peer-to-Peer Systems Napster, Gnutella, BitTorrent, challenges 2. Distributed Hash Tables 3. The Chord Lookup
More informationCSC 4900 Computer Networks: P2P and Sockets
CSC 4900 Computer Networks: P2P and Sockets Professor Henry Carter Fall 2017 Recap SMTP is the language that mail servers use to exchange messages. SMTP is push-based... why? You can run SMTP from a telnet
More informationPeer Assisted Content Distribution over Router Assisted Overlay Multicast
Peer Assisted Content Distribution over Router Assisted Overlay Multicast George Xylomenos, Konstantinos Katsaros and Vasileios P. Kemerlis Mobile Multimedia Laboratory & Department of Informatics Athens
More informationP2P. 1 Introduction. 2 Napster. Alex S. 2.1 Client/Server. 2.2 Problems
P2P Alex S. 1 Introduction The systems we will examine are known as Peer-To-Peer, or P2P systems, meaning that in the network, the primary mode of communication is between equally capable peers. Basically
More informationThe Design Trade-offs of BitTorrent-like File Sharing Protocols
The Design Trade-offs of BitTorrent-like File Sharing Protocols Bin Fan John C.S. Lui Dah-Ming Chiu Abstract The BitTorrent BT file sharing protocol is very popular due to its scalability property and
More informationTraffic Patterns in Peer-to-Peer-Networking. Christian Schindelhauer. joint work with Amir Alsbih Thomas Janson
Traffic Patterns in Peer-to-Peer-Networking Christian Schindelhauer joint work with Amir Alsbih Thomas Janson to be presented at ITA Albert-Ludwig University Freiburg Department of Computer Science Computer
More informationLesson 9 Applications of DHT: Bittorrent Mainline DHT, the KAD network
Lesson 9 Applications of DHT: Bittorrent Mainline DHT, the KAD network 20/03/2017 1 IMPLEMENTATION OF KADEMLIA Kademlia is the protocol used by the largest public DHTs. Vuze ( before known as Azureus),
More informationL3S Research Center, University of Hannover
, University of Hannover Dynamics of Wolf-Tilo Balke and Wolf Siberski 21.11.2007 *Original slides provided by S. Rieche, H. Niedermayer, S. Götz, K. Wehrle (University of Tübingen) and A. Datta, K. Aberer
More informationinria , version 1-6 Sep 2006
Rarest First and Choke Algorithms Are Enough Arnaud Legout I.N.R.I.A. Sophia Antipolis France arnaud.legout@sophia.inria.fr G. Urvoy-Keller and P. Michiardi Institut Eurecom Sophia Antipolis France {Guillaume.Urvoy,Pietro.Michiardi}@eurecom.fr
More information6 Structured P2P Networks
6 Structured P2P Networks Distributed Data Management Wolf-Tilo Balke Christoph Lofi Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 6.1 Hash Tables 6.3
More informationSearching for Shared Resources: DHT in General
1 ELT-53206 Peer-to-Peer Networks Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationPerformance Analysis and Improvement for BitTorrent-like File Sharing Systems
CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE Concurrency Computat.: Pract. Exper. 2007; 00:1 7 [Version: 2002/09/19 v2.02] Performance Analysis and Improvement for BitTorrent-like File Sharing
More informationLecture 4 September 17
CS 559: Algorithmic Aspects of Computer Networks Fall 2007 Lecture 4 September 17 Lecturer: John Byers BOSTON UNIVERSITY Scribe: Sowmya Manjanatha In today s lecture, we discussed and characterized the
More informationCSCI-1680 P2P Rodrigo Fonseca
CSCI-1680 P2P Rodrigo Fonseca Based partly on lecture notes by Ion Stoica, Sco5 Shenker, Joe Hellerstein Today Overlay networks and Peer-to-Peer Motivation Suppose you want to write a routing protocol
More informationNash Equilibrium Load Balancing
Nash Equilibrium Load Balancing Computer Science Department Collaborators: A. Kothari, C. Toth, Y. Zhou Load Balancing A set of m servers or machines. A set of n clients or jobs. Each job can be run only
More informationBoosting Market Liquidity of Peer-to-Peer Systems Through Cyclic Trading
Boosting Market Liquidity of Peer-to-Peer Systems Through Cyclic Trading Raphael Eidenbenz Thomas Locher Stefan Schmid Raphael+Roger: ETH Zurich DCG Thomas: ABB Research Roger Wattenhofer T-Labs Stefan:
More information12/5/16. Peer to Peer Systems. Peer-to-peer - definitions. Client-Server vs. Peer-to-peer. P2P use case file sharing. Topics
// Topics Peer to Peer Systems Introduction Client-server vs peer to peer Peer-to-peer networks Routing Overlays Structured vs unstructured Example PP Systems Skype login server Peer-to-peer - definitions
More informationTaming the Flood: How I Learned to Stop Worrying and Love the Swarm Yogesh Vedpathak Cleversafe
Taming the Flood: How I Learned to Stop Worrying and Love the Swarm Yogesh Vedpathak Cleversafe Topics Popular data Creating broadcasting storage system Bittorrent protocol Creating swarms and destroying
More informationFurther extensions to the Wigan architecture are discussed in this thesis, including possible means of handling data updates.
COMPUTING SCIENCE A BitTorrent-based Peer-to-Peer Database Server J. Colquhoun TECHNICAL REPORT SERIES No. CS-TR-1135 January, 2009 TECHNICAL REPORT SERIES No. CS-TR-1135 January, 2009 A BitTorrent-based
More informationSearching for Shared Resources: DHT in General
1 ELT-53207 P2P & IoT Systems Searching for Shared Resources: DHT in General Mathieu Devos Tampere University of Technology Department of Electronics and Communications Engineering Based on the original
More informationSome Peers Are More Equal than Others! Stefan Schmid
Some Peers Are More Equal than Others! Stefan Schmid Telefónica, March 12, 2009 Peer-to-Peer Technology Well-known p2p systems - Internet telephony: Skype - File sharing: BitTorrent, emule,... - Streaming:
More informationLIBSWIFT P2P PROTOCOL: AN ANALYSIS AND EXTENSION
Master Thesis - TRITA-ICT-EX-2012-262 LIBSWIFT P2P PROTOCOL: AN ANALYSIS AND EXTENSION Fu Tang Design and Implementation of ICT products and systems Royal Institute of Technology (KTH) fut@kth.se October
More informationLECT-01, S-1 IAD99S, Javed I.
A Course on Foundations of Peer-to-Peer Systems & Applications LECT-01, S-1 CS 6/75995 Foundation of Peer-to-Peer Applications & Systems Kent State University Dept. of Computer Science www.cs.kent.edu/~javed/class-p2p08/
More informationPeer-to-Peer Systems. Network Science: Introduction. P2P History: P2P History: 1999 today
Network Science: Peer-to-Peer Systems Ozalp Babaoglu Dipartimento di Informatica Scienza e Ingegneria Università di Bologna www.cs.unibo.it/babaoglu/ Introduction Peer-to-peer (PP) systems have become
More informationCSCI-1680 Link Layer I Rodrigo Fonseca
CSCI-1680 Link Layer I Rodrigo Fonseca Based partly on lecture notes by David Mazières, Phil Levis, John Jannotti Last time Physical layer: encoding, modulation Today Link layer framing Getting frames
More informationARTICLE IN PRESS. Journal of Network and Computer Applications
Journal of Network and Computer Applications 32 (29) 666 683 Contents lists available at ScienceDirect Journal of Network and Computer Applications journal homepage: www.elsevier.com/locate/jnca Principles
More informationPage 1. How Did it Start?" Model" Main Challenge" CS162 Operating Systems and Systems Programming Lecture 24. Peer-to-Peer Networks"
How Did it Start?" CS162 Operating Systems and Systems Programming Lecture 24 Peer-to-Peer Networks" A killer application: Napster (1999) Free music over the Internet Key idea: share the storage and bandwidth
More informationWIRELESS ACCESS IN VEHICULAR ENVIRONMENTS USING BIT TORRENT AND BARGAINING
WIRELESS ACCESS IN VEHICULAR ENVIRONMENTS USING BIT TORRENT AND BARGAINING by Barsha Shrestha A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Electrical
More informationIntroduction to Peer-to-Peer Systems
Introduction Introduction to Peer-to-Peer Systems Peer-to-peer (PP) systems have become extremely popular and contribute to vast amounts of Internet traffic PP basic definition: A PP system is a distributed
More informationScaling Problem Computer Networking. Lecture 23: Peer-Peer Systems. Fall P2P System. Why p2p?
Scaling Problem 15-441 Computer Networking Millions of clients server and network meltdown Lecture 23: Peer-Peer Systems Peter Steenkiste Fall 2010 www.cs.cmu.edu/~prs/15-441-f10 2 P2P System Why p2p?
More informationScalable overlay Networks
overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent
More informationEnforcing collaboration in a collaborative content distribution network
Enforcing collaboration in a collaborative content distribution network Namita Lal (154403) Faculty of Sciences Vrije Universiteit Amsterdam, The Netherlands August 2007 Master s thesis, Computer Science
More informationDistributed Data Management
Distributed Data Management Wolf-Tilo Balke Christoph Lofi Institut für Informationssysteme Technische Universität Braunschweig http://www.ifis.cs.tu-bs.de 2.0 Introduction 3.0 Query Processing 3.1 Basic
More informationScalable overlay Networks
overlay Networks Dr. Samu Varjonen 1 Lectures MO 15.01. C122 Introduction. Exercises. Motivation. TH 18.01. DK117 Unstructured networks I MO 22.01. C122 Unstructured networks II TH 25.01. DK117 Bittorrent
More informationDISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES
DISTRIBUTED COMPUTER SYSTEMS ARCHITECTURES Dr. Jack Lange Computer Science Department University of Pittsburgh Fall 2015 Outline System Architectural Design Issues Centralized Architectures Application
More informationAttacks and exploits targeting BitTorrent and other P2P file sharing networks
Attacks and exploits targeting BitTorrent and other P2P file sharing networks Andreas Hegenberg Betreuer: Benedikt Elser Seminar Future Internet WS 09/10 Lehrstuhl Netzarchitekturen und Netzdienste Fakultät
More informationOverview Computer Networking Lecture 16: Delivering Content: Peer to Peer and CDNs Peter Steenkiste
Overview 5-44 5-44 Computer Networking 5-64 Lecture 6: Delivering Content: Peer to Peer and CDNs Peter Steenkiste Web Consistent hashing Peer-to-peer Motivation Architectures Discussion CDN Video Fall
More informationCodes, Bloom Filters, and Overlay Networks. Michael Mitzenmacher
Codes, Bloom Filters, and Overlay Networks Michael Mitzenmacher 1 Today... Erasure codes Digital Fountain Bloom Filters Summary Cache, Compressed Bloom Filters Informed Content Delivery Combining the two
More informationLecture 6: Overlay Networks. CS 598: Advanced Internetworking Matthew Caesar February 15, 2011
Lecture 6: Overlay Networks CS 598: Advanced Internetworking Matthew Caesar February 15, 2011 1 Overlay networks: Motivations Protocol changes in the network happen very slowly Why? Internet is shared
More informationStochastic Analysis and File Availability Enhancement for BT-like File Sharing Systems
Stochastic Analysis and File Availability Enhancement for BT-like File Sharing Systems Fan Bin Dah-Ming Chiu John C.S. Lui Abstract In this paper, we present the mathematical analysis of two important
More informationUnit 8 Peer-to-Peer Networking
Unit 8 Peer-to-Peer Networking P2P Systems Use the vast resources of machines at the edge of the Internet to build a network that allows resource sharing without any central authority. Client/Server System
More informationPeer-to-Peer (P2P) Systems
Peer-to-Peer (P2P) Systems What Does Peer-to-Peer Mean? A generic name for systems in which peers communicate directly and not through a server Characteristics: decentralized self-organizing distributed
More informationFUtella Analysis and Implementation of a Content- Based Peer-to-Peer Network
8th Netties Conference Technische Universität Ilmenau September 30th to October 2nd 2002 T. Zahn / H. Ritter / J. Schiller / H. Schweppe FUtella Analysis and Implementation of a Content- Based Peer-to-Peer
More informationTelematics Chapter 9: Peer-to-Peer Networks
Telematics Chapter 9: Peer-to-Peer Networks Beispielbild User watching video clip Server with video clips Application Layer Presentation Layer Application Layer Presentation Layer Session Layer Session
More information