Heterogeneous Gossip. Davide Frey Rachid Guerraoui Anne-Marie Kermarrec Boris Koldehofe Maxime Monod Martin Mogensen Vivien Quéma

Transcription

1 Heterogeneous Gossip Davide Frey Rachid Guerraoui Anne-Marie Kermarrec Boris Koldehofe Maxime Monod Martin Mogensen Vivien Quéma

2 Outline Context Live Streaming Gossip Limitations Heterogeneous Gossip Protocol Evaluation Conclusion

3 Context Large-scale (no IP-multicast) Churn/Failures Constrained, asymmetric and heterogeneous bandwidth Target application: Live streaming

4 Live Streaming Sent at t s Received at t r t 0 t 1 t 2 t 3? Maximize stream quality Play the stream at t r +δ Minimize δ (buffering delay) Play the stream at t 0 +ε Minimize ε (stream lag)

5 Existing approaches Structured overlay Static overlay / Reactive repair Unstructured overlay Dynamic overlay / Proactive repair DHT-based systems Trees Multiple trees Mesh-based systems Gossip Trees over mesh

6 Existing approaches Structured overlay Unstructured overlay Static overlay / Reactive repair SplitStream (2003) Chunkyspread (2006) Coolstreaming (2005) Chainsaw (2005) PULSE (2007) PRIME (2007) Dynamic overlay / Proactive repair BAR Gossip/FlightPath (2006/2008) Gossip Coolstreaming (2007) GridMedia (2007) Heterogeneity awareness?

7 Gossip in the real world George meets Bob:

8 Gossip in computer science Infect-and-die model fanout partners p 0 p 1 p 2 p 3 p 4 p f gossip period

9 Gossip - Theory 1. fanout = ln(n) + c P[connected graph] goes to exp(-exp(-c)) c=2 87% c=3 95% c=1 69% c=0 37% 0.2 c=-1 7% 0 ln(n)-10 ln(n)-5 ln(n) ln(n)+5 ln(n) Holds as long as the fanout is ln(n) + c on average

10 Gossip Practice (600kbps) 100 Percentage of nodes receiving at least 99% of the stream Percentage of nodes (CDF) PlanetLab nodes have: Large bandwidths Small delays Stream lag (s)

11 Gossip is load-balancing S p 3 Proposals arrive randomly Nodes pull from the first proposal p 1 p 2 q Highly-dynamic S q S q Node q will serve f nodes whp Node q will serve f nodes wlp

12 but the world is heterogeneous! Load-balancing Capability 3 classes (691kbps avg): 3Mbps 5% 512kbps 85% 1Mbps 10% Percentage of nodes (CDF) Percentage of nodes receiving at least 99% of the stream

13 How to cope with heterogeneity? Goal: contribute according to capability Advertize more = sell more: Propose more = serve more Increase fanout and decrease it too! Challenges: Preserve reliability of gossip vs average fanout (f avg ) initial fanout = ln(n) + c Cope with dynamic capabilities

14 Heterogeneous Gossip - HEAP Capability q and r with bandwidths b q > b r q should upload b q / b r times as much as r Who should increase/decrease its contribution? and by how much? How to ensure reliability? How to keep f avg constant?

15 HEAP b avg Capability Total/average contribution is equal in both homogeneous and heterogeneous settings f q = f init b q /b avg ensuring the average fanout is constant and equal to f init = ln(n) + c

16 HEAP Get b avg with (gossip) aggregation Advertize own and freshest received capabilities Aggregation follows change in the capabilities Get n with (gossip) size estimation Estimation follows change in the system Join/leave Crashes

17 Stream Lag reduction 100 Percentage of nodes receiving at least 99% of the stream 90 Percentage of nodes (CDF) No cap HEAP 691kbps Standard gossip 691kbps Stream lag (s)

18 Experimental Setup 270 PlanetLab nodes Network capabilities Bandwidth cap by throttling Communication with UDP Stream rate of 600kbps Windows of 110 events, including 9 FEC events Gossip period of 200 ms f avg = 7 (ln(270) = 5.60)

19 Evaluation Metrics Stream Lag Time difference between creation at the source and delivery to the player Stream Quality A window is considered jittered if < 101 events Stream with maximum of 1% jitter means at least 99% of the windows are complete Incomplete does not mean blank!

20 Quality improvement Stream lag of 10s Jitter-free percentage of the stream 512kbps 1Mbps 3Mbps Standard Gossip HEAP

21 Stream Lag For those who can have a jitter-free stream Stream lag (s) Average stream lag to obtain a jitter-free stream kbps 35 1Mbps 30 3Mbps Standard Gossip HEAP

22 Proportional contribution Average bandwidth usage by bandwidth class kbps Mbps 3Mbps % 91.56% 48.44% 94.38% 90.58% 87.56% 0 Standard Gossip HEAP

23 20% nodes crashing Percentage of nodes receiving each window Failure of 20% of the nodes at time t=60s HEAP - 12s lag Standard Gossip - 20s lag Standard Gossip - 30s lag Stream time (s)

24 Conclusion Limitations UDP usage TCP-unfriendliness Incoming traffic Probability of acceptance also depends on latency Future work Compare with mesh systems Freeriders Biasing partner selection