A Cloud Gateway - A Large Scale Company s First Line of Defense. Mikey Cohen Manager - Edge Gateway Netflix

Transcription

1 A Cloud - A Large Scale Company s First Line of Defense Mikey Cohen Manager - Edge Netflix

2 Today, more than 36% of North America s internet traffic is controlled by systems in the Amazon Cloud

3

4 Global Streaming of TV Shows and Movies

5 Nearly 70 Million Subscribers In over 80 Countries

6 Netflix accounts for over 36% of Downstream Traffic in North America

7 From the Internet to Services in the Cloud????? API Website

8 Our Netflix Handles most netflix.com hosts Over 20 production Zuul clusters ~ 50 elbs handles ~10 origin services

9 Netflix Scale Tens of billions of requests per day 3 AWS regions Over 1000 device types Hundreds of permutations of protocols and device versions

10 Our Journey Success Evolution Scale Failure

11 So What!? - Change your perspective!!

12 Traditional Cloud Proxy Mission Simple static rule-based routing API portal Request authentication Throttling - request caps Monitoring Caching

13 The - a grown-up proxy! Dynamic routing Deep Insights Load balancing Availability focused Service protection Quality assurance tool

14 Evolving to a

15 Netflix s Public API Late 2008 Mashery Datacenter

16 Streaming Devices using public API Early Streaming Devices Windows Media Center XBox PS3

17 Migration to AWS 2010 Sonoa / Apigee proxy Device traffic, not public Controlling DC -> cloud migration Running in AWS Under Netflix control

18 Streaming Success 2011 Chaos Complexity Failure Success Leveraging Cloud benefits

19 Anti-patterns of most cloud proxies Static configurations Service push needed to change behavior Limited range of functionality Limited to HTTP

20 Zuul Created 2012 Dynamically injected and compiled filters Manipulate requests and responses Headers / Body / etc Change routing Add metrics and other functions Built on Netflix s OSS stack Open Sourced

21 Zuul - A Victim of Success Easy and convenient Instant results High adoption Happy customers Business logic in proxy Affects system resiliency Zuul team in critical path

22 Creating a Strategy

23 Principles of Netflix s Strategy Creative Routing Dynamic Routing Delivery Focused Traffic Shaping React Fast Insights

24 Creative Routing - Subclusters with Purpose v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

25 Red / Green Deployments v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

26 Developer Test Branches v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

27 Instrumented Clusters v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

28 Squeeze Testing v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

29 Targeted Routing v1 v1 test test Instrumented Instrumented v2 v2 debug debu g canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

30 Service Canarying v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

31 Sticky Canary v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

32 Failure Injection Testing v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

33 Degraded Experience Testing v1 v1 test test Instrumented Instrumented v2 v2 debug debug canary squeeze squeeze sticky canary sticky canary FIT sticky sticky

34 Traffic Shaping

35 A Global Cloud Deployment Network Tier Proxy Network Tier Proxy Presentation Tier Websites API Presentation Tier Websites API Business services Tier Business services Tier Persistence Tier DB Persistence Tier DB Network Tier Proxy Presentation Tier Websites API Business services Tier Persistence Tier DB

36 Global Cloud Routing Network Tier Proxy Network Tier Proxy Presentation Tier Websites API Presentation Tier Websites API Business services Tier Business services Tier Persistence Tier DB Persistence Tier DB Network Tier Proxy Presentation Tier Websites API Business services Tier Persistence Tier DB

37 A Failing region Network Tier Proxy Network Tier Proxy Presentation Tier Websites API Presentation Tier Websites API Business services Tier Business services Tier Persistence Tier DB Persistence Tier DB Network Tier Proxy Presentation Tier Websites API Business services Tier Persistence Tier DB

38 routing to other regions Network Tier Proxy Network Tier Proxy Presentation Tier Websites API Presentation Tier Websites API Business services Tier Business services Tier Persistence Tier DB Persistence Tier DB Network Tier Proxy Presentation Tier Websites API Business services Tier Persistence Tier DB

39 Attack prevention API Website

40 Smart Load Balancing

41 Smart Load Balancing - Bad Nodes

42 Backoff and Blacklists Bad Nodes

43 Zone Failure - Blacklist the Zone automatically

44 React Quickly - Runtime Filter changes Runtime Policy Injection API Website

45 A Room with a View - Insights Insights API Website

46 What s Next for Netflix s? as a service Self-service dynamic routing / route validation Control APIs for special routing functions Netty Based Zuul (using RxNetty) Handling persistent connections non-blocking, async Transport protocol agnostic routing Reactive Socket

47 Top Ten Lessons Learned

48 Build for handling Failures

49 Expect the Unexpected

50 Using Routing Creatively

51 Shard to Reduce Blast Radius

52 Devices are Weird Protocols are Weird

53 Devices are Forever Protocols are Forever

54 It will be built wrong

55 Keep Business Logic out of your

56 For More Info... Zuul OSS Netflix Tech Blog RxNetty Jobs