Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem

Transcription

1 Internet Video: The Next Wave of Massive Disruption to the U.S. Peering Ecosystem William B. Norton Co-Founder & Chief Technical Liaison Equinix, Inc. 0.3 Clean Slate Networking esearch Workshop Wednesday, March 21, :30-14:00 Frances C. Arrillaga Alumni Center

2 NSF funded $ UMich NSFNET Project Logistics Host egional Techs Meetings Agenda NSFNET Open sharing of Ops Data Then port 80 grew, NSF stopped funding

3 NSF $ Self-Sustaining funded $ UMich NSFNET Project Logistics Host egional Techs Meeting egional North American Techs Network Meetings Operators Group (NANOG) Agenda Commercial Internet NSP NSFNET NSP NSFNET NSP NSP N A P Slow transition Little sharing Weak SLAs&NDAs 1998->EQIX

4 Equinix Carrier Neutral Internet Exchange Bay Area, LA, CHI, DAL, DC, NYMetro Singapore, Hong Kong, Tokyo, Sydney IPO in 2001, Market Cap $2.5B today 1200 Customers, 200 Networks Tier 1 ISPs, Cable Cos, Content, etc. Efficient interconnection of services Catbird seat 90% documenting Internet Ops Practices

5 Internet Operations White Papers 1) Interconnection Strategies for ISPs 2) Internet Service Providers and Peering 3) A Business Case for Peering 4) The Art of Peering: The Peering Playbook 5) The Peering Simulation Game On the Internet, 6) Do ATM-based Internet Exchanges Make Sense Anymore? Everyone if a Publisher 7) Evolution of the U.S. Peering Ecosystem 8) The Asia Pacific Internet Peering Guidebook 9) The Great (Public vs. Private) Debate 10) The Folly of Peering Traffic atios? 11) Video Internet: The Next Wave. Internet makes anyone a publisher, similar effect now emerging for video

6 Massive Disruption in U.S. Peering Ecosystem Short Videos YouTube founded 2005 Short video clips 50 million view per day! 20Gbps of peering traffic Feb 2006 Now, On the Internet $1M/month in Sept 2006! Entering Peering Ecosystem 30 Other competitors 600Gbps peerable? Everyone is a Broadcaster DoveTail Video may dwarf current peered traffic % Internet is Video Inculcate video guys into peering ecosystem Short video clips Full TV shows Source: Source:

7

8 Massive Disruption in U.S. Peering Ecosystem Full Episodes Desperate Housewives 210MB/hour For 320x240 H.264 Video itunes image 10,000,000 households 2,100,000,000 MB = 2.1 peta-bytes How long will that take to download? 3 64Gbps non-stop! Just one show Try 250M*180 Channels*HDTV The Point: Massive Wave of Incremental Traffic to document Source:

9 The esearch Questions How to distribute video across the Internet? How much does it cost per video?

10 Transit =Metered pipe to the Internet CDN =Content Distribution Network Peering =free & reciprocal access to each others Customers P2P =PeerToPeer Modeling Varying Sized Loads Small =Distribute 10 videos every 5 minutes on avg. Medium =Distribute 100 videos every 5 minutes on avg. Large =Distribute 1000 videos every 5 minutes on avg. Models A:10 videos B: 100 C: 1000 $ per video? 1: Transit Model 1A Model 1B Model 1C 2: CDN Model 2A Model 2B Model 2C 3: Hybrid Model 3A Model 3B Model 3C 4: P2P Model 4A Model 4B Model 4C Shift from Avg to more typical demand curve...

11 Demand Curve Modeling 95 th % ω~ 4 x α ρ=6.6xα α = average load ρ = peak load (6.6*α) ω =95th percentile (4*α) α = 10 videos/5 min=400mbps ω =1.6Gbps ρ = 2.64Gbps α = 100 videos/5 min=4,000mbps ω =16Gbps ρ = 26.4Gbps α = 1000 videos/5 min=40,000mbps ω =160Gbps ρ = 264Gbps

12 Model 1C Large Load Commodity Transit Upstream ISPs Server1 : Server24 : Server262 : : Server263 Server264 : GigE Switch1 : GigE Switch14 10G 10G outer4 outer2 outer2 outer1 8 * 10GE to upstreams each Average Load 40,000 Mbps 95th Percentile Load 4 160,000 Mbps Peak Load ,000 Mbps Model 1C - Simple Commodity Transit for Video Distribution monthly Transit Fee 160,000 mbps@ $10 permbps $1,600,000 Colo 14 rack@ $2,000 perack $28,000 Network Equip $150,000 3yrAmort $16,667 AggregationSw 14 $10,000 3yrAmort $3,889 Servers 264 $4,000 3yrAmort $29,333 Maintenance 15% $7,483 Staff 3 $180,000 $45,000 Total $1,730,372 # videos downloaded per month 8,640,000 Cost per video downloaded $0.20

13 Model 2C: CDN Large Load Average Load 40,000 Mbps 95th Percentile Load 4 160,000 Mbps Peak Load ,000 Mbps Model 2C - Content Delivery Network for Video Distribution monthly Transit Fee 160,000 mbps@ $13 permbps $2,080,000 Colo 1 rack@ $1,500 perack $1,500 Network Equip $30,000 3yrAmort $833 Servers 1 $4,000 3yrAmort $111 Maintenance 15% $367 Staff 0.5 $180,000 $7,500 Total $2,090,311 # videos downloaded per month 8,640,000 Cost per video downloaded $0.24

14 Model 3C: Transit/Peering Large Load Average Load 40,000 Mbps 95th Percentile Load 4 160,000 Mbps Peak Load ,000 Mbps Model 3C - Blended Transit and Peering for Video distribution 3 site 25% peering monthly Transit Fee 120,000 mbps@ $10 permbps $1,200,000 Colo 42 rack@ $2,000 rack+port $84,000 Network Equip $150,000 3yrAmort $50,000 AggregationSw 42 $10,000 3yrAmort $3,889 Servers 792 $4,000 3yrAmort $88,000 Maintenance 15% $21,283 Staff 3 $180,000 $45,000 Total $1,492,172 # videos downloaded per month 8,640,000 Cost per video downloaded $0.17

15 Model 4C: P2P Large Load Average Load Mbps Mbps 95th Percentile Load 4 160,000 Mbps Peak Load ,000 Mbps Model 4C - Peer-to-Peer Network for Video Distribution single-site stormcasting monthly Transit Fee 100 mbps@ $50 permbps $5,000 Colo 1 rack@ $1,500 perack $1,500 Network Equip $30,000 3yrAmort $833 Servers 1 $4,000 3yrAmort $111 Maintenance 15% $367 Staff 0.5 $180,000 $7,500 Total $15,311 # videos downloaded per month 8,640,000 Cost per video downloaded $0.0018

16 Summary Internet Video Distribution Methods $/Video Download $0.90 $0.80 $0.70 $0.60 $0.50 $0.40 $0.30 $0.20 $0.10 $0.00 A B C Model 1:Transit Model2: CDN Model3: Transit/Peering Model 4: P2P Load Models Models A:10 videos B: 100 C: : Transit 1A: $0.60 1B: $0.36 1C: $0.20 2: CDN 2A: $0.77 2B: $0.44 2C: $0.24 3: Hybrid 3A: $0.69 3B: $0.31 3C: $0.17 Per Video Cost Of delivery 4: P2P 4A:$0.18 4B: $ C: $0.0018

17 Observations Internet Transit Supply Price Internet Transit Model src/dst specific Bottlenecks IX Power, outer Capacity, Peer s Capacity, Backbone Capacity, Last Mile bottleneck, 100G NIC? Do I need to upgrade $$$$ gear to support my competitor (peer)? Identify Players, Positions, Motivations, Behavior Geoff Huston: P2P has won. Telco/Cable co trying to keep its 1998 biz plan relevant. Predictions from the Field

18 Content Providers: "We are sitting around this table in 2010 and we are commenting how remarkable the last few years have been, specifically that: 1. Video streaming volume has grown 100 fold 2. Last mile wireless replaced local loop 3. Botnets (DDOS attacks) are still an issue 4. Non-mechanical (i.e. Flash) Drives replaced internal hard drives on laptops 5. 10% of all cell phones are now video phones 6. We have cell phones that we actually like 7. The U.S. is insignificant traffic wise relative to the rest of the world 8. Most popular question discussed around the table: 'How do we operate business in China? 9. No online privacy. And the gov't watches everything demographic is best reached w/ads on the Internet 11. Next Gen 3D on-line Social Networks are so successful 12. No physical network interfaces are needed 13. We will big brother ourselves (video cams 'who scraped my car?') 14.So many special purpose Internet apps - in car google maps, livetraffic updates, etc. 15.So much of our personal information is on the net 16.Video IM emerged as a dominant app 17.P2P will emerge for non-pirated videos - DM in place and embraced 18.Voice calls are free, bundled with other things Source: May 2006 Web Content Private Forum

19 ISPs: We are sitting around this table in 2010 at NANOG and we are commenting how remarkable the last few years have been, specifically that: 1. We have 10G network interface(s) on laptops (I assumed wired, but someone else might have been thinking wireless) 2. $5/mbps is the common/standard price of transit (other prediction was $30/mbps) 3. Internet traffic is now so heavily localized (as in 75% of telephone calls are across town type of thing but for the Internet) 4. Ad revenue will cover the cost/or subsidize significantly of DSL 5. 90% of Internet bits will be video traffic 6. VoIP traffic exceeds the PSTN traffic 7. Private networks predominantly migrate to overlays over the Internet 8. Wireless Internet Service Providers (WISPs) are serious competitive threat to DSL and Cable Internet 9. Sprint is bought by Time Warner 10. Cable companies form cabal & hookup with Sprint or Level Government passes Net Neutrality Law of some flavor 12. Earthlink successfully reinvents themselves as Wireless Metro player in esponse to ATT and Verizon % paid or subscription as opposed to Content Click Ads. Like Cable Company channel packages, folks will flock to subscriptions for Internet Content packages. 14. IAA proposes surcharge on network access (like Canada tax on blank CDs) 15. NetFlix conversion to Internet delivery of movies to Tivo or PC,or open source set top box 16. ISPs will be in pain 17. Last mile (fiber, wireless,.) in metro will be funded by municipal bonds 18. Death of TV ads, Death of broadcast TV, Tivo & Tivo like appliances all use the Internet with emergence of targeted ads based ondemographic profiles of viewer 19. Google in charge of 20% of ALL ads (TV, adio, Billboards,.) 20. Ubiquitous wifi in every metro with wifi roaming agreements 21. Congestion issues drive selective customer acceptance of partial transit offerings 22. IPTV fully embraced by cable cos - VOD - no need for VD and ala carte video services replace analog frequency 23. Near simultaneous release of movies to the theaters, DVDs for the home, PPV, and Internet download to meet needs of different demographics. Source: Discussions at NANOG 37 San Jose