P2P Dissemination / ALM

Transcription

1 P2P Dissemination / ALM Teague Bick < > Su Fu < > Zixuan Wang < >

2 Definitions P2P Dissemination Want fast / efficient methods for transmitting data between peers ALM Application Layer Multicast Virtual connections because IP does not provide a method Non-virtual methods are not widespread Digital Fountain Use encoding Sources can make any combination of codes Intermediary nodes can add to encodings

3 Techniques Informed Content Delivery Across Adaptive Overlay Networks ACM Sigcomm J. Byers, J. Considine, M. Mitzenmacher, S. Rost Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh Dejan Kosti c, Adolfo Rodriguez, Jeannie Albrecht, and Amin Vahdat Enabling Content Dissemination Using Efficient andscalable Multicast Tae Won Cho, Machael Rabinovich, K.K. Ramakrishnan, Divesh Srivastava, Yin Zhang

4 Paper 1 Informed Content Delivery Across Adaptive Overlay Networks

5 Paper 1 Purpose: Nodes need to communicate Want a small # of transmissions and bits transmitted Techniques: Coarse-Grain (Estimation of differences) Set estimation allows estimation of how alike two nodes are (want to connect to diverse peers) Fine-Grained(Bloom filters / Reconciliation trees) Allows nodes to determine what data to encode and transmit

6 Paper 1 Techniques Encoding Digital Fountain Can encode any symbols to create many others, almost to an infinite degree Re-encoding Intermediary nodes can input information they have, regardless of whether or not they have the entire file Need only receive a certain # of distinct packets Typically need only receive a few % more bits than the original file size We transmit more but gain in throughput

7 Paper 1 - Background Erasure Coding Encode information (such as bitwise xor, or ReedSolomen codes) Need only receive the file size worth of unique coded packets to decode Can not decode until entire file received Digital Fountain technique Servers use Erasure Coding techniques Can create a code from any combination of data Intermediary nodes can add information to codes

8 Paper 1 - Background Unicast (Client-Server) Wastes bandwidth by transmitting multiple copies over links Sends way more data than is necessary Server is huge bottleneck Does not scale! Multicast (Peer to Peer (P2P)) Eliminates server bottleneck Accommodates asynchronous data / clients Not widely supported Use Virtual multicast (Overlays) Near optimal scaling!

9 Paper 1 - Background Overlay topologies Clients connect to each other via TCP and form their own virtual network Accommodates multicast (unlike TCP/IP) Can be dynamically altered to accommodate changes in the network Allows modification of en-route nodes to be smarter and to be incorporated in the data distribution Basis for P2P applications

10 Paper 1 - Main Issues Peers need to find out what data each other have Want to keep message transmissions down Want to keep message size down Main Question: How do we represent data on a node? System is/needs to deal with: Asynchronous - messages / processing times Heterogeneous - speed / size / data set Transience - failures / topology changes Scalable - the more nodes, the harder to keep state

11 Paper 1 - Main Issues Scalable Can not keep global state (too much info / too dynamic) Keeping track takes too much processing time / memory Want to avoid negotiating as much as possible Want to keep system diverse (so that information sent is not duplicated) Diversity of data on nodes Erasure coding

12 Paper 1 - Solutions Erasure Coding Scalable Resistant to packet loss Do not need a specific original file segment Data spread across multiple packets Can recover original file from any subset of received packets >= original file size (usually a few % more) Digital Fountain Uses erasure coding Any node may add new data to a packet without increasing its size Must still be wary of making sure all nodes receive a variety of packets

13 Paper 1 - Solutions Data Representation / Transfer We want the most cost effective method Methods: Coarse Grained Want to get a general feel for differences between data sets on nodes Several methods to estimate Speculate Transfers Make educated guess about what to send then transmit (may have duplicates) Fine Grained Compact ways to determine nearly-exact sets of data on a node More detailed than coarse-grain, but supply more information

14 Paper 1 - Coarse Grain Framework Containment Estimates fraction of elements useless to A Resemblance Estimates how similar the two data sets are to each other SA = data set on Node A, SB = data set on Node B Techniques Random sampling (look at your values at random and Tx) Sample only along modulo k (smaller set, variable size) Min-wise sketches (remedies issues of other two)

15 Paper 1 - Coarse Grain Represent data as a key (~64 bits) Send a 1KB packet from B->A with as many keys as possible Transferred data acts as a sample of keys on B A looks at transferred data and compares to local data to estimate differences Data can be chosen: At random At pre-determined locations (0 modulo k) Min-wise sketches

16 Paper 1 - Min-wise Sketches Min-wise sketches Pick some working set of contained data Permute data (predetermined functions) Represent working set as minimum from permutations Compare number of matching minimum on two nodes Elements are equal with probability of the resemblance Can be used to compare between separate sets

17 Paper 1 - Minwise Sketches

18 Paper 1 - Reconciling Differences If two nodes are different, what data should be sent? Must be quick method Must involve little communication The more accurate the better Coarse-grain does not supply a way to find differences (only know that they exist)

19 Paper 1 - Fine Grained Exact Methods: Send list of all elements Very data-heavy! Transmit a hash of data Less data-heavy Probability of double mappings Instead of going for exact representation, just go for close as possible

20 Paper 1 - Fine Grained Bloom Filters Used to find most (at a minimum, some) of the differences between nodes Very compact Operation Use m bits, initially all set to zero Hash n original elements with k = m-1 hash functions Set the hash result bits to 1 in the m-bit array To check if an element is in array, check to see if bits of hash all have value 1 May result in a false positive with probability

21 Paper 1 - Fine Grained B sends Bloom filter to A A checks to see if filter contains elements With m=4, chance of error is 14.7% With m=8, chance of error is 2.2% Use four bits for brevity Can use a new technique, approximate reconciliation trees Combines Bloom filters and merkle trees

22 Paper 1 - Reconciliation Trees Bloom filters, where each parent element represents an entire set Allows traversing tree to find out which elements are different between two sets If parents match, assume no differences in sub-tree Fast search for set differences If too many false positive, force continued search of tree

23 Paper 1 - Reconciliation Trees

24 Paper 1 - Re-encoding Have intermediate (routing) nodes add in their data to content Requires erasure coding content Increases probability of transmitting a useful element Technique: Divide file into l equal sized blocks XOR blocks together Add header indicating what symbols included Receiving a symbol (or set) allows partial decoding

25 Paper 1 - Results Parameters 128MB file 1400 byte-long symbols (95,870 symbols) 64-bit identifiers Average overhead of 2.5% Tested methods: Uninformed collaboration Speculative collaboration Reconciled collaboration

26 Paper 1- Results Results for overhead as expected Uninformed > Speculative > Reconciled Slack = % original file available to nodes Overhead decreases as more file duplicates available

27 Paper 1 - Results Speed up of 1.95 with reconciliation with overhead Best results obtained when updates transferred every symbol updates

28 Paper 1 - Conclusion Main contribution is the format for representing the data that a node has in a succinct manner Erasure Coding data is import to ensure a piece of data is valuable to receiver Paper doesn't address theoretic limits (we only know they outperform other basic methods) Coding methods other than XOR may improve performance further Paper does not address security concerns

29 Paper 2 Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh

30 Brief Intro to the paper What? A high-bandwidth system for data distribution from a single source to a large number of receivers How? Basic components Data Encoding --> improves efficiency RanSub --> distributes disjoint data Delivery Techniques TCP Friendly Rate Control --> maintains steady sending rates

31 Brief Intro to the paper How? Bullet--mesh over overlay tree Finding overlay peers Recovering data from peers Making data disjoint Evaluation Creating disjoint data Evaluating Bullet on a Lossy Network Monitoring performance under node failures

32 Brief Intro to the paper Contributions An algorithm for sending data to any node in a equal possibility A scalable and decentralized algorithm that allows nodes to locate and recover missing data items A complete implementation and evaluation of Bullet, across Internet and a large scale emulation environment

33 ABSTRACT Goal: High-bandwidth data distribution from a single source to a large number of receivers. Argument: An overlay mesh can deliver fundamentally higher bandwidth and reliability relative to typical overlay tree structures. Presented Method: Bullet, a scalable and distributed algorithm that enables nodes spread across the Internet to self-organize into a high bandwidth overlay mesh.

34 1 INTRODUCTION Structure In a tree, children receive identical data sets from parent In a mesh, children receive disjoint (divide + subdivide) data from parents and peers Must maintain properties TCP Friendly Impose low control overhead Algorithm --> decentralized Approach --> robust to individual node failure

35 1 INTRODUCTION How Bullet works? Nodes begin by self-organizing into an overlay tree. Each Bullet node transmits a disjoint set of data to its children A scalable and efficient algorithm to enable nodes to quickly locate multiple peers with missing data items to the node is performed Bullet uses TFRC (TCP Friendly Rate Control) to transfer data both down the overlay tree and among peers

36 1 INTRODUCTION Mesh VS Tree Tree limitation: high overhead associated with the tree construction protocol

37 2 SYSTEM COMPONENTS Data encoding RanSub Informed Content Delivery Techniques TCP Friendly Rate Control (TFRC)

38 2.1 Data Encoding What & Why Depending on the type of the data being distributed through the system, a number of data encoding schemes can improve system efficiency How? Redundant Tornado - Encoding scheme addressed by this paper

39 2.2 RanSub What? A scalable approach to distributing changing, uniform random subsets of global state to all nodes of an overlay tree Step Collect: Leaves--> Root Distribute: Root--> Leaves

40 2.2 RanSub

41 2.3 Informed Content Delivery Techniques What we need: Method for reconciling the differences in the data Bandwidth-efficient method with low computational overhead How? As the methods mentioned in Paper 1

42 2.3 Informed Content Delivery Techniques

43 2.4 TCP Friendly Rate Control What? TFRC targets unicast streaming multimedia applications with a need for less drastic responses to single packet losses Goal Maintaining a relatively steady sending rate while still being responsive to congestion Sending rate given by:

44 3 BULLET What? An efficient data distribution system for bandwidth intensive applications Rather than using a tree, it uses a mesh on top of an overlay tree to increase overall bandwidth to all nodes

45 3.1 Finding Overlay Peers RanSub delivers subsets of uniformly random selected nodes to each participant in the overlay Bullet receivers use the RanSub lists to locate remote peers able to transmit missing data items with good bandwidth

46 3.2 Recovering Data From Peers Sender --> Bloom filter --> Receiver Receiver Installs the received Bloom filter Bloom filter says what data sender has Periodically transmits keys not present in the filter to Bloom filter originator

47 3.2 Recovering Data From Peers

48 3.3 Making Data Disjoint A node p maintains for every child: sfi: sending factor proportion of its stream the child gets according the descendants the child has lfi: limiting factor proportion of the parent rate beyond the sending factor that each child can handle

49 3.3 Making Data Disjoint

50 3.4 Improving the Bullet Mesh Attempt to make each node have max senders and receivers How to make the current peering relationship optimal Each node evaluates the bandwidth performance it is receiving from its sending peers. If receiving too many duplicate --> drop If small portion of data is useful --> drop

51 4 EVALUATION

52 4.1 Offline Bottleneck Bandwidth Tree Currently, there is no overlay mesh constructed using Bullet (relative to the best possible tree) So a simple greedy offline algorithm determines the connectivity of the tree likeliest to deliver a high level of bandwidth Need the bottleneck to have bounds Have some assumptions Based these assumptions, concentrate on estimating the throughput available between two participants in the overlay

53 4.2 Bullet vs. Streaming

54 4.2 Bullet vs. Streaming

55 4.2 Bullet vs. Streaming

56 4.2 Bullet vs. Streaming

57 4.3 Creating Disjoint Data

58 4.4 Epidemic Approaches

59 4.5 Bullet on a Lossy Network

60 4.6 Performance Under Failure

61 4.6 Performance Under Failure

62 4.7 PlanetLab

63 CONCLUSION Argument: an overlay mesh is able to deliver fundamentally higher bandwidth than a tree Difficulty: nodes in the mesh do not repeatedly receive the same data from peers Solution: Bullet --> a scalable and efficient overlay construction algorithm that overcomes this challenge to deliver significant bandwidth improvements relative to traditional tree structure

64 Paper 3 Enabling Content Dissemination Using Efficient and Scalable Multicast

65 Intro to This Paper Multicast uses network and server resources in an efficient manner to disseminate info to groups Challenges: a. Network should focus more on information b. Dealing with increasing number of groups c. Support of persistent group membership d. Scalability

66 Proposed Solution - MAD Overview to MAD: MAD stands for Multicast with Adaptive Dual-state. MAD offers efficient multicast services at massive scale. MAD supports large numbers of multicast groups.

67 Intro to Multicast Multicast provides efficient and explicit performance benefits Multicast not been widely used: Lack of support by ISPs Lack of application demand

68 Revival of Multicast Internet is becoming more informationcentered Expectations of scalable and efficient delivery of information to large user groups For instance: IPTV Multi-player online games Video conferencing Grid computing File sharing

69 Sources of demand for multicast Increasing: Number of groups Time of group membership Range of activities across groups Flexibility of activities in a group

70 MAD Specifications Strengths of MAD: Decouples group membership from forwarding state, thus supporting persistence in group membership Has different treatments for active and inactive groups, thus realizing data forwarding efficiency and scalability of group numbers Gives transparency in the environment of dynamic changes in group's activity levels

71 MAD Specifications Related methods in terms of multicast: IP multicast-style Multicast forwarding state reduction Peer-to-Peer content distribution Points of departure (nodes leave tree)

72 Deeper exploration into MAD MAD environment: Manages user profile, authenticates users, and provides access control in a fine-grained way MAD example IP multicast tree illustrates a set of routers that forms a traditional IP multicast tree. There are 5 users to this single multicast group.

73 Membership Tree Nodes and Final Membership Tree States

74 Modular Design of MAD Protocol 5 key components*: Membership tree sub-protocol Distribution tree sub-protocol State transition Mechanisms Failure Recovery Ways of operating across domain boundaries (* Refer to the paper for details on realizing these components)

75 Scaling of MAD Trees Several figures of simulation results on network topologies are shown. The state need of MAD trees are examined. The trade-off between state reduction and forward cost are analyzed.

76 Conclusion Multicast uses network and server resources in an efficient way to distribute data to groups Challenges include: information-centric network, increasing number of groups, supporting persistent group membership, and scalability. MAD is devised to provide efficient multicast service and offers good scalability through decoupling group membership and forwarding state, and targets active and inactive groups with different methods

77 Conclusion Cont'd A high-level analysis of multimedia trees and membership trees is illustrated, together with mathematical derivations. Moreover, MAD protocols are analyzed, simulated and implemented. The results indicate that MAD achieves about 10% reduction in control overhead and state requirements. Increasing demands from networks are met in an more efficient and scalable way.

78 Main Paper SplitStream: High-Bandwidth Multicast in Cooperative Environments

79 Introduction Another alternative for IP multicast: endsystem or application-level multicast Advantages: applies only unicast network services to route and disseminate messages Main theme of this paper: application-level multicast in p2p / cooperative environments Feature of this environment: peers are awarded for contributing resources and are allowed to use the service

80 Shortcomings Conventional Multicast: Primarily relies on tree-based multicast, not a good match for the desired cooperative environment Copying and forwarding traffic is done through a small portion of peers inside nodes in the tree topology A large portion of nodes are at the leaf and make no contributions Conflict 1: not all nodes are contributing in this cooperative environment Conflict 2: in high BW applications, a peer's limited capacity may be prohibitive in applications like video

81 Briefly, what does SplitStream do? Splits the content into k stripes and multicasts each stripe using a separate tree Peers may be asked to join many trees, in terms of desired stripes, and can set an upper limit on the number of stripes they wish to forward Challenges: building a forest of various trees so that inside nodes are leaf nodes in all the other trees within the forest, under the constraints of per-node BandWidth

82 Advantages of SplitStream Improves system flexibility in the case of node failures by striping across multiple trees The role of SplitStream: the majority of nodes are interior nodes to only one tree Result 1: on average, a node's failure leads to a loss of <=1 stripes Result 2: the tree topology is more fault-tolerant for the benefits of multicast

83 Challenges in Designing SplitStream Build the multicast forest in a scalable, decentralized, independent and efficient way for the joined nodes Depends on a p2p overlay to build and maintain the trees Using the PlanetLab Internet testbed and a large-scale network simulator to evaluate SplitStream's performance

84 2. SplitStream Approach Overview Tree-based multicast SplitStream Applications Properties

85 Tree-based multicast The interior nodes are in charge of sending all the forwarding multicast messages. In a balanced tree with fanout f and height h, we obtain that the number of leafs is f, and the number of interior nodes is (fh-1)/(f-1)

86 SplitStream Why SplitStream? To solve the issue of unbalanced load forwarding in the previous tree-based multicast system To disseminate the forwarding load in a balanced way to the corresponding nodes To split the multicast stream into several stripes To use separate multicast trees to disseminate each stripe

87 SplitStream Example: The original content is split into two stripes and multicast separate trees. The content is separated into Besides the source, every peer receives both stripes inducing an inbound bandwidth. k stripes. peers receive a portion of the stripes, and have the inbound bandwidth requirement in increments of B/K. B = inbound BW

88 Applications SplitStream: Offers an infrastructure for content distribution with high-bandwidth Manages how content is encoded and split into stripes Constructs the multicast trees for the stripes, and is constrained by the per-node bandwidth limits Applications: Need mechanisms to obtain content from other peers in the system Control when to create and tear down a SplitStream forest

89 Properties Necessary and sufficient conditions for the feasibility for building the SplitStream tree Condition 1: Sum of desired in-degrees cannot exceed the sum of forwarding capacities Condition 2: Any node whose forwarding capacity exceeds their indegree must receive or forward all stripes

90 Background SplitStream is implemented using tree-based application-level multicast. Many proposals exist for building and maintaining overlay multicast trees Two methods of implementation: Pastry Scribe

91 Pastry Scalable and self-organizing structured p2p overlay network Similar to CAN, Chord, and Tapestry Given a message and a key, Pastry routes the message to the node with the node ID which is numerically closest to the key. This is called the key's root. Each node maintains a routing table and a leaf set. Each Pastry node also maintains the set of neighboring nodes

92 Pastry

93 Scribe An application-level group communication system constructed on Pastry A pseudo-random Pastry key is chosen from each multicast group. It is known as groupid. A multicast tree of a certain group is formed by unioning Pastry routes from each group member to the groupid's root Messages are multicast from the root to the members using reverse path forwarding.

94 Scribe Pastry s local route convergence ensures that the load imposed on the physical network is small because most message replication occurs at intermediate nodes that are close in the network to the leaf nodes in the tree. Scribe can support lots of groups, arbitrary numbers of group members, and groups with highly dynamic membership well.

95 Pastry

96 4. SplitStream Design

97 4.1 Building interior-node-disjoint trees Tree --> interior-node-disjoint if Each interior node is in at most one tree Leaf node in the other tree

98 4.1 Building interior-node-disjoint trees

99 4.2 Limiting node degree Inbound BW is proportional to desired indegree Desired indegree --> number of stripes node chooses to receive Assumption: each node receives and forwards at least the stripe whose stripeid shares a prefix with its nodeid --> interior node Outbound BW May exceed a node's forwarding capacity without limiting its outdegree

100 4.2 Limiting Node Degree Scribe has a built-in mechanism to limit outdegree in the following procedure When node reaches max outdegree, receives request from perspective children Reply with a list of current children Perspective child seek the one with lowest delay Root --> Leaves to terminate BUT NOT FOR SPILITSTREAM

101 4.3 Locating parents Node adopts the prospective child regardless of limit Evaluate its new set of children to select one to reject Look for stripeid that does not share a prefix with local nodeid If prospective node's ID matches --> select If not, randomly select from set Look for nodeid with shortest prefix match If prospective node's ID matches --> select If not, randomly select from set Then the chosen child is orphaned

102 4.3 Locating parents

103 4.3 Locating Parents For an orphaned child to locate new parent Examines former siblings and attempts to attach to a random former sibling that shares a prefix match with the stripeid for which it seeks a parent Siblings use same algorithm Execute recursively If orphan hasn't found a parent, use the spare capacity group

104 4.4 Spare Capacity Group (SCG) Spare capacity group includes all nodes that have less children in their stripe trees than their forwarding capacity limit Scribe delivers anycast message to SCG near the orphan DFS For nodes: if no children, check whether it receives the stripe the orphan tried to receive

105 4.4 Spare Capacity Group

106 4.5 Correctness and Complexity 2 assumptions: The communication is reliable to ensure all spare capacity in the system is available in spare capacity group Nodes do not leave the system either voluntarily or due to failures

107 4.5 Correctness and complexity Estimate the probability that the algorithm leaves an orphan that cannot find a desired stripe

108 4.5 Correctness and complexity 2 ways for a node i to acquire a parent for each selected stripe s (1): joining the stripe tree directly without using the spare capacity group (2): anycasting to the spare capacity group if s.stripeid starts with the same digit as i. nodeid (1) will be used, may orphan other node (2) will never be used

109 4.5 Correctness and complexity If s.stripeid does not start with the same digit as i.nodeid i may fail after being pushed down at most h times (h = height of the tree) using (1) Does not make other node orphan Balanced tree h = log N

110 4.5 Correctness and complexity If i and s share digits, and i finds a parent in s, and j becomes orphan, j attempts use (1), 3 cases: (a): j looses a stripe r (r!= s) (b): j looses stripe s and the identifiers of j and s do not share the first digit (c): j looses stripe s and the identifiers of j and s share the first digit In (a) and (b), j either finds a parent or fails to find a parent after being pushed down for h times In (c), j = i

111 4.5 Correctness and complexity If orphan j attempts to locate a parent for stripe r by anycasting, it may fail to find a node in the spare capacity group that receives stripe r. This probability --> Pf

112 4.5 Correctness and complexity Pf An upper bound of Pf Assuming set 1...L nodes in spare group, with indegree I1...IL If all nodes join at least Imin stripes, no indegrees needed

113 4.5 Correctness and complexity This bound holds even with concurrent anycasts, because we use the minimum spare capacity in the system to compute the bound.

114 4.5 Correctness and complexity At most N nodes join at most k stripes for each node The number of anycasts issued during forest construction that may fail is bounded by N * k The P that algorithm fails to build a feasible forest The probability of failure is very low even with a modest amount of spare capacity in the system. Guarantee: build a forest if desired indegrees do not exceed forwarding capacities even with no spare capacity

115 4.5 Correctness and complexity Complexity ( of forest construction) The expected amount of the state maintained by each mode O(log N ) The expected number of messages to build the forest is O( N log N ) --> well balanced O( N 2) --> worst case

116 5. Experimental Evaluation Experimental Setup Added in own propagation delay (1ms) Ignores (impedes large-scale simulation): Queuing Delay Packet loss Cross traffic Physical SplitStream setup: LAN for setting up

117 5. Experimental Evaluation Three Models used: GATech Georgia Tech random graph generator Average of 10 topologies used Mercator Topology generated from measurements of the Internet Includes AS generation CorpNet Measurements of Microsoft's corporate network

118 5. Experimental Evaluation Gnutella Trace for actual data SplitStream Configuration Split files into l=16 stripes 320Kbps / stream 20Kbps / stripe SplitStream Implementation Optimizations to: Reduce stored path complexity Batch together transmissions Reduce total transmissions

119 5. Experimental Evaluation Metrics Node Stress - Average number of messages received per node Link Stress - Total number of messages sent over a link

120 5.2 Forest Construction Overhead Setup 90% complete by stress level 100. Stress level varies little as more nodes added 6.8 times lower setup than Centralized system with equivalent number of nodes

121 5.3 Forest Multicast Performance Over 40,000 nodes - performed well Results: RAD = Average (delay per stripe / IP multicast) RMD=(max delay per stripe)/(max delay IP multicast)

122 5.4 Resilience to Node Failures 25% failures instantaneous(of 10,000) One failure affects entire system Recovers after ~1 minute

123 5.4 Resilience to Node Failures PlanetLab 72 total systems, 8 brought down at t=35 Huge drop, recovery takes some time

124 5.4 Resilience to Node Failures Large churn on Gnutella trace Results over 7 minute interval Figure shows control messages per time

125 Conclusions Did not address non-lan setup Much of design ignored node failures Paper runs with delay of 1ms (unrealistic)

126 Questions