CoopNet: Cooperative Networking

Transcription

1 CoopNet: Cooperative Networking Venkat Padmanabhan Microsoft Research September

2 Collaborators MSR Researchers Phil Chou Helen Wang MSR Intern Kay Sripanidkulchai (CMU) 2

3 Outline CoopNet motivation and overview web content distribution streaming media content distribution multiple description coding multiple distribution trees related work summary and ongoing work Other networking projects at MSR 3

4 Motivation A flash crowd can easily overwhelm a server often due to news event of widespread interest but not always (e.g., Webcast of birthday party) can affect relatively obscure sites (e.g., election.dos.state.fl.us, firestone.com, nbaa.org) site becomes unreachable precisely when popular! affects Web content as well as streaming content infrastructure-based CDNs aren t for everyone too expensive even for big sites (e.g., CNN) uninteresting for CDN to support small sites Goal: solve the flash crowd problem without requiring new infrastructure! 4

5 Cooperative Networking Client-server Pure peer-to-peer CoopNet CoopNet complements client-server system client-server operation in normal times P2P content distribution invoked on demand to alleviate server overload clients participate only while interested in the content server still plays a critical role 5

6 CoopNet Tradeoffs Avoids dependence on expensive CDN infrastructure but no performance guarantees P2P network size scales with load Availability of resourceful server simplifies many P2P tasks but is the server a potential bottleneck? 6

7 Flash Crowd Characteristics Traffic > 10x normal Lasted many hours An MSNBC server on Sep 11,

8 Where is the bottleneck? Disk? no, most requests are for popular content MSNBC: 90% of requests were for 141 files CPU? perhaps for dynamic content a single server node can pump out > 1 Gbps Network? yes, most likely close to the server 65% of servers have bottleneck bandwidths of less than 1.5 Mbps (Stefan Saroiu, U.W.) 8

9 Outline CoopNet motivation and overview web content distribution streaming media content distribution multiple description coding multiple distribution trees related work summary and ongoing work Other networking projects at MSR 9

10 CoopNet for Web Content Server maintains a cache of recent client IP addresses When overloaded, it redirects new clients to old ones that have the content Huge bandwidth savings (100X) 200 B redirect instead of 20 KB page 10

11 Operation of CoopNet A B C E D 11

12 Operation of CoopNet A B C E D 12

13 Operation of CoopNet A B A C E D 13

14 Operation of CoopNet A B A C E D 14

15 Operation of CoopNet A B A, B A C E D 15

16 Operation of CoopNet A A B A, B C E D 16

17 Operation of CoopNet A A B A, B C E D 17

18 Operation of CoopNet A A B A, B, C A, B C E D 18

19 Operation of CoopNet A A B A, B, C C A, B E D 19

20 Issues Peer selection network proximity: BGP prefix, delay-based coordinates matching peer bandwidth Server bottleneck large # of CoopNet peers large volume of redirects small # of CoopNet peers server remains overloaded CoopNet still beneficial, but initial redirect can take long solution: initial search in peer group high locality small group size suffices greatly simplifies distributed search fall back to server-based redirect upon miss Privacy 20

21 Finding content (optimistic) 100% Minus Compulsory Miss Repeated Request Rate New Hit Rate 5 Peers New Hit Rate 10 Peers New Hit Rate 30 Peers New Hit Rate 50 Peers Rate Over All Requests (%) High locality during flash crowd Most content can be found amongst peer groups of size :00 06:15 06:30 06:45 07:00 Time of Day 21

22 100 Rate Over All Requests (%) Finding content (pessimistic) % Minus Compulsory Miss Repeated Request Rate New Hit Rate 5 Peers Fresh Hit Rate 5 Peers New Hit Rate 10 Peers Fresh Hit Rate 10 Peers New Hit Rate 30 Peers Fresh Hit Rate 30 Peers New Hit Rate 50 Peers Fresh Hit Rate 50 Peers Clients fall back on server-based redirection only 15% of the time 40 06:00 06:15 06:30 06:45 07:00 Time of Day 22

23 Alternative approaches Proxy caching deployment barriers not effective when clients are scattered across the Internet Commercial CDNs (e.g., Akamai) not cost-effective for small sites P2P system of servers (e.g., Backslash) feasible in practice? 23

24 Outline CoopNet motivation and overview web content distribution streaming media content distribution multiple description coding multiple distribution trees related work summary and ongoing work Other networking projects at MSR 24

25 CoopNet for Live Streaming More likely that server will be overwhelmed Key issue: robustness peers are not dedicated servers potential disruption due to: node departures and failures higher priority traffic traditional application-level multicast (ALM) falls short 25

26 Traditional Application-level Multicast 26

27 CoopNet Approach to Robustness Add redundancy in data multiple description coding (MDC) and in network paths multiple, diverse distribution trees 27

28 Multiple Description Coding MDC Layered coding Unlike layered coding, there isn t an ordering of the descriptions Every subset of descriptions must be decodable Modest penalty relative to layered coding 28

29 Multiple Description Coding Simple MDC: every M th frame forms a description More sophisticated MDC combines: layered coding Reed-Solomon coding priority encoded transmission optimized bit allocation 29

30 Multiple Distribution Trees Tree diversity provides robustness to node failures 30

31 MDC Analysis Key parameters: number of nodes (N) number of descriptions (M) out-degree of each node repair time node departure rate Two scenarios of interest large N, high churn multiple node failures in repair interval small N, stable occasional, single node failures 31

32 Quality During Multiple Failures 32

33 Quality During Single Failure 33

34 Tree Management Goals: short and wide trees efficiency diversity quick join and leave processing scalability CoopNet approach: centralized protocol anchored at the server single point of failure but server is source of data anyway 34

35 Basic Tree Management Protocol Nodes inform server of their arrival and departure Server tracks node capacity and tells new nodes where to join high up in the tree but randomized fan out of server is typically much larger Each node monitors its packet loss rate and takes action when loss rate becomes too high Simple, scales to joins/leaves per sec. 35

36 Optimizations Achieving efficiency and diversity cluster nodes into super nodes using delaybased coordinates (akin to GeoPing) logical topology matches physical topology at the macroscopic level Migrate stable nodes to higher levels in the tree 36

37 Achieving Efficiency and Diversity SEA S NY SF Supernode 37

38 Performance Evaluation MSNBC access logs from Sep 11, 2001 Live streaming ~18,000 simultaneous clients ~180 joins/leaves per second on average; peak rate of ~1000 per second ~70% of clients tuned in for less than a minute On-demand streaming 300,000 requests in a 2-hour period 38

39 Live Streaming Key questions how beneficial is MDC? does well is diversity preserved as trees evolve? how does repair time impact performance? 39

40 MDC versus SDC Based on MSNBC traces from Sep 11 40

41 Random Trees vs. Evolved Trees Random Trees Evolved Trees 41

42 Impact of Repair Time 1 sec 3 sec 6 sec 10 sec Average fraction of descriptions received Time (seconds) 42

43 CoopNet for On-demand Streaming Distributed streaming of multiple descriptions Improves robustness and load distribution 43

44 On-demand Streaming Key results: server bandwidth requirement drops from 20 Mbps to 300 Kbps peer bandwidth requirement: average over all peers is 45 Kbps average over active peers is 465 Kbps storage requirement at a peer is less than 100 MB probability of finding peer in the same BGP prefix cluster is under 20% 44

45 CoopNet Transport Architecture ZSF GOF Parse RD Curve Embedded Stream Optimize (M, p(m)) Server Break Points Packetize RS Encoder M descriptions Internet m M descriptions Depacketize Embedded Stream (truncated) RS Decoder Reformat Client GOF (quality depends on # descriptions received) Decode Render 45

46 Outline CoopNet motivation and overview web content distribution streaming media content distribution multiple description coding multiple distribution trees related work summary and ongoing work Other networking projects at MSR 46

47 Related Work Infrastructure-based CDNs Akamai, Digital Island P2P CDNs Pseudo-serving, PROOFS, Backslash SpreadIt, Allcast, vtrails Application-level multicast ALMI, Narada, Scattercast Bayeux, Scribe Multi-path content delivery Byers et al. 1999, Nguyen & Zakhor 2002, Apostolopoulos et al

48 Summary Client-server applications can benefit from selective use of peer-to-peer communications Availability of server simplifies system design Web content high degree of locality server-based redirection plus small peer group Streaming content robustness to dynamic membership is the key challenge MDC with multiple, diverse distribution trees improves robustness in peer-to-peer media streaming centralized tree management is efficient and can scale 48

49 Ongoing Work Prototype implementation Dealing with client heterogeneity for live streaming combine MDC with layering More info: research.microsoft.com/~padmanab/projects/coopnet Papers at IPTPS 02 and NOSSDAV 02 49

50 Outline CoopNet motivation and overview web content distribution streaming media content distribution multiple description coding multiple distribution trees related work summary and ongoing work Other networking projects at MSR 50

51 Networking Research at MSR Internet measurement and performance Passive Network Tomography IP2Geo: Internet Geography PeerMetric: broadband network performance Peer-to-Peer networking Herald: scalable event notification system CoopNet: P2P content distribution Wireless networking UCoM: energy-efficient networking Mesh Networks: multi-hop wireless access network 51

52 PeerMetric Goal: characterize broadband network performance DSL, cable modem, satellite, etc. P2P as well as client-server performance Deployment on ~25 distributed nodes underway none in Atlanta volunteers welcome! Joint work with Karthik Lakshminarayanan (MSR intern from Berkeley) 52

53 Mesh Networks: Capacity is the Key Challenge RTS RTS CTS CTS 4 nodes are active, 2 packets in flight (example courtesy of Victor Bahl) 53