Scalable Application Layer Multicast

Transcription

1 Scalable Application Layer Multicast Suman Banerjee Bobby Bhattacharjee Christopher Kommareddy

2 Group Communication A C A C B D B D Network-layer Multicast Replication at routers Sequence of Direct Unicasts Replication only at source

3 Application-layer Multicast A C 1 2 B D Replication at end-hosts Examples: Narada, Yoid, Gossamer, HMTP, Scribe, Bayeux, CAN-multicast, DT, NICE

4 Application-layer Multicast A C 1 2 B D Replication at end-hosts Metrics Tree Quality State / Control Overheads Robustness

5 Talk Outline Introduction NICE Application-layer Multicast Protocol Results Conclusions

6 NICE Application-layer Multicast Scales to large group sizes Low average and worst case control overheads Does not compromise tree quality or robustness Even low-bandwidth applications are efficient Web tickers Uses a hierarchy

7 NICE Topologies Control topology Detects host failures and re-structure the overlay Data delivery topology Basic path: Implicitly defined by the hierarchy Can be independent of the control path

8 NICE Hierarchy A Set of Members

9 NICE Hierarchy Clusters Non-overlapping Proximity-based Size: k to 3k-1

10 NICE Hierarchy Clusters Non-overlapping Proximity-based Size: k to 3k-1 B1 Layer 0 C0 B0 B2 Graph-theoretic center is the cluster leader

11 NICE Hierarchy B1 Layer 1 A1 C0 B0 B2 Leaders form the higher layer and repeats

12 NICE Hierarchy B1 Layer 2 C0 B0 B2 log N layers

13 Control Topology Soft state about all cluster peers HeartBeats B1 A1 B0 A0 A2 B2

14 Control Topology Soft state about all cluster peers HeartBeats B1 A1 C0 B0 B2 A0 A2 State and Control message overheads: Average: Constant Worst case: O(k log N)

15 Basic Data Path A1 A0 B0 A2

16 Basic Data Path B1 C0 A1 A0 B0 A2 B2

17 Basic Data Path B1 C0 A1 A0 B0 A2 B2

18 Basic Data Path C0 A1 A0 B0 A2 B2

19 NICE Invariants Cluster sizes between k and 3k-1 Cluster leader is the central member Leaders for next higher layer NICE protocol maintains these invariants

20 NICE Protocol Operations Member Join Member Depart Cluster Split Cluster Merge Cluster Refine

21 Join Procedure B1 C0 B0 Assume a Rendezvous Point RP B2 A3

22 Join Procedure B1 C0 B0 RP Join L0 B2 A3

23 Join Procedure B1 C0 B0 RP B2 L2: {C0} A3

24 Join Procedure B1 C0 B0 RP Join L0 B2 A3

25 Join Procedure B1 A1 C0 B0 RP L1: {B0,B1,B2} B2 A3

26 Join Procedure B1 C0 B0 RP Join L0 B2 A3

27 Join Procedure RP B1 C0 L0: { } B0 L0: { } L0: { } A3 B2 L0: { }

28 Join Procedure RP B1 C0 B0 Attach A3 B2 Overhead: O(log N) RTTs and O(k log N) messages Optimizations possible

29 Cluster Split Cluster size: 4 to 11 B1 C0 B0 B2

30 Cluster Split B1 C0 B0 B2

31 Cluster Split B1 C0 B0 B2

32 Cluster Split Split into two new clusters Each new cluster has at least 3k/2 members B3 LeaderTransfer B0 B4 B1 C0 B2

33 Cluster Split B1 C0 B3 Join L1 B4 B2

34 Cluster Split B1 C0 B3 Leave L1 B0 B4 B2

35 Cluster Split B1 C0 B3 B4 B2

36 Results Simulations 10,000 node Transit-Stub graphs Group sizes upto 2048 Comparisons with Narada [CMU] Wide-area Experiments Members at 8 sites Group sizes upto 96 Dynamic joins and (ungraceful) leaves Constant rate data source

37 Evaluation Metrics Tree Quality: Stress Number of copies of the same data packet on a link/router Example: Stress on link [A-1] = 2 Tree Quality: Stretch Ratio of the overlay latency to the direct unicast latency Example: Stretch for receiver D = 5/3 State at end-hosts Control overheads Robustness Host failures Application-layer Multicast A C 1 2 B D

38 Example Scenario 128 members join 16 members leave within 10 seconds Time (in seconds)

39 Tree Quality: Stress Resource usage at links First 200 seconds

40 Tree Quality: Stretch End-to-end latency to receivers First 200 seconds

41 Failure Recovery After 1000 secs

42 Control Overheads

43 Control Overheads Group Size Narada-30 NICE Bandwidth overheads averaged over all network routers

44 35.5 Wide-area Testbed Source A: cs.ucsb.edu B: asu.edu C: cs.umd.edu D: glue.umd.edu E: wam.umd.edu F: umbc.edu G: poly.edu H: ecs.umass.edu

45 Failure Recovery Includes the effects of network losses

46 Failure Recovery Includes the effects of network losses

47 Related Work Mesh-first Narada, Gossamer Tree-first Yoid, HMTP Implicit Scribe, Bayeux, CAN-multicast, Delaunay-Triangulation A Comparative Study of Application Layer Multicast Protocols, S. Banerjee and B. Bhattacharjee - Available at:

48 Current Work Detailed analysis of tree quality Stress and stretch Implementing applications Video delivery

49 Conclusions NICE scales to large member groups Low control overhead Does not sacrifice tree quality or robustness Scalability using hierarchy