Cloud 3.0 and Software Defined Networking October 28, Amin Vahdat on behalf of Google Technical Infratructure Google Fellow

Size: px

Start display at page:

Download "Cloud 3.0 and Software Defined Networking October 28, Amin Vahdat on behalf of Google Technical Infratructure Google Fellow"

Ilene Scott
6 years ago
Views:

1 Cloud 3.0 and Software Defined Networking October 28, 2016 Amin Vahdat on behalf of Google Technical Infratructure Google Fellow

doing so can be huge Our research question: how do we build a network that can allow a

2 Overview This talk: example of the Google research model Driven by novel application requirements, we must solve problems at the frontier of computer systems The impact of doing so can be huge Our research question: how do we build a network that can allow a building to be the unit of storage access and a shared medium for compute?

3 Subset of Google Networking Publications 1. Congestion Based Congestion Control, ACM Queue An Internet-Wide Analysis of Traffic Policing, SIGCOMM Evolve or Die: High-Availability Design Principles Drawn from Failures in a Global-Scale Content Provider, SIGCOMM Maglev: A Fast and Reliable Software Network Load Balancer, NSDI TIMELY: RTT-based Congestion Control for the Datacenter, SIGCOMM Condor: Better Topologies through Declarative Design, SIGCOMM Bandwidth Enforcer: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing, SIGCOMM Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google s Datacenter Network, SIGCOMM Libra: Divide and Conquer to Verify Forwarding Tables in Huge Networks, NSDI B4: Experience With a Globally-Deployed Software Defined WAN, SIGCOMM Google Cloud Platform 3

4 Computing at a Crossroads Distributed programming has faced similar challenges since sockets The free lunch in performance improvements are over Storage capacity has increased through disaggregation I/O latency gap remains Next-gen storage remains largely untapped at scale Networking will drive future improvements in compute performance by blurring the line between individual servers and then making them disappear. Google Cloud Platform 4

5 Last Decade Cloud 1.0 Virtualization delivers capex savings to enterprise DCs Google Cloud Platform 5

6 Now Cloud 1.0 HW on Demand Cloud 2.0 Public cloud frees enterprise from private HW infrastructure Scheduling, load balancing primitives, big data query processing Google Cloud Platform 6

7 The Third Wave of Cloud Computing Cloud 1.0 Cloud 2.0 Compute, not servers Cloud 3.0 Serverless compute, actionable intelligence, and machine learning Not data placement, load balancing, OS configuration and patching Google Cloud Platform 7

8 The Third Wave of Cloud Computing Cloud 1.0 Cloud 2.0 Cloud 3.0 Networking should be aiming for Cloud 3.0 Google Cloud Platform 8

9 Networking and Cloud 3.0 Storage disaggregation: the datacenter is the storage appliance Seamless telemetry and scale up/down Transparent live migration Open Marketplace of services, securely placed and accessed Google Cloud Platform 9

10 Networking and Cloud 3.0 Applications not VMs Policy not middleboxes Actionable Intelligence not data processing SLOs not placement/load balancing/scheduling Google Cloud Platform 10

11 Making the Network Disappear with Software Defined Networking Making the Network Disappear with Software Defined Networking

13 Google Software Innovations Driven by Unprecedented Demand for Scale, Bandwidth, Reliability Borg FlumeJava Spanner Colossus Dremel Bigtable Pregel MapReduce GFS Google Cloud Platform 13

14 Google Networking Innovations Our distributed computing infrastructure required networks that did not exist BBR QUIC Jupiter grpc Google Global Cache Watchtower Freedome BwE Onix B Andromeda Google Cloud Platform 14

15 DCN Bandwidth Growth 50x Traffic generated by servers in our datacenters Aggregate traffic 1x Jul 08 Jun 09 May 10 Apr 11 Mar 12 Feb 13 Dec 13 Nov 14 Time Google Cloud Platform 15

SDN Motivation Traditional network architectures could not keep up with bandwidth demands in the data center Operational complexity of box-centric deployment

16 SDN Motivation Traditional network architectures could not keep up with bandwidth demands in the data center Operational complexity of box-centric deployment Google s DCN redesign, inspired by server & storage scale out Clos Topologies Merchant Silicon Centralized Control Software Defined Networking Google Cloud Platform 16

Why Balance Matters @ Building Scale An unbalanced data center means: Some resource is scarce...limiting your value Other resources are idle.

17 Why Balance Building Scale An unbalanced data center means: Some resource is scarce...limiting your value Other resources are idle...increasing your cost Substantial resource stranding [Eurosys 2015] if we cannot schedule at scale Amdahl s Amdahl s lesser lesser known known law: law: 1Mbit/sec of IO for every 1 Mhz of computation in parallel computing 1Mbit/sec of IO for every 1 Mhz of computation in parallel computing Google Cloud Platform 17

18 Building Scale 64*2.5 Ghz server Compute Slice 100 Gb/s Flash 100k+ IOPS 100 us access PB s storage Compute Slice Datacenter Network 50k servers 5 Pb/s Network?? NVM 1M+ IOPS 10 us access TB s storage Based on Amdahl s observation, we might need a 5 Pb/s network Even with 10:1 oversub 500Tb/s datacenter network Every building needs more bisection than the Internet Google Cloud Platform 18

19 Building Scale Compute Slice Flash 100k+ IOPS 100 us access PB s storage Compute Slice Datacenter Network 10 us latency NVM 1M+ IOPS 10 us access TB s storage To exploit future NVM, we need ~10 usec latency Even for Flash, we need 100 usec latency Or, expensive servers sit idle while they wait for IO Google Cloud Platform 19

Availability @ Building Scale Compute Slice Flash Compute Slice Datacenter Network 50k servers NVM Cannot take down a XX MW building for

20 Building Scale Compute Slice Flash Compute Slice Datacenter Network 50k servers NVM Cannot take down a XX MW building for maintenance New servers always added; older ones decommissioned with zero service impact Network evolves from 1G 10G 40G 100G??? Google Cloud Platform 20

21 Datacenter Network infrastructure to support Google scale, performance, and availability underpinnings of Google Cloud Platform Google Cloud Platform 21

22 Five Generations of Networks for Google scale Spine Block 1 Spine Block 2 Spine Block 3 Spine Block 4 Spine Block M Edge Aggregation Block 1 Edge Aggregation Block 2 Edge Aggregation Block N Server racks with ToR switches Google Cloud Platform 22

23 Bisection B/w 1000T Jupiter Watchtower Firehose T Saturn 10T Firehose Post 1T Google Cloud Platform 12 Year 23

Bisection B/w 1000T Jupiter Watchtower Firehose 1.0 100T Saturn 10T Firehose 1.

24 Bisection B/w 1000T Jupiter Watchtower Firehose T Saturn 10T Firehose Post 1T Scales out building wide Google Cloud Platform Pbps Year 24

25 Bisection B/w 1000T Jupiter Watchtower Firehose T Saturn + Enables 40G to hosts 10T + External control servers Firehose OpenFlow 4 Post 1T Google Cloud Platform 12 Year 25

26 B4: Google's Software Defined WAN B4: [Jain et al, SIGCOMM 13] BwE: [Jain et al, SIGCOMM 15] Google Cloud Platform 26

27 Andromeda Network Virtualization VNET: /24 VNET: /24 VNET: 5.4/16 Load Balancing DoS ACLs VPN ToR ToR ToR ToR NFV Internal Network / / / /24 Google Infrastructure Services Google Cloud Platform 27

28 Making the Network Disappear Software Defined Networking enables the network to disappear, driving the next wave of computing Sufficiently high bandwidth, low latency, and low cost Fabrics, not boxes, programmable for performance and isolation Highest level of availability, zero downtime for new features/performance Google Cloud Platform 28

29 Thank You!

Google SDN Peering: An Early Engagement Case Study

Google SDN Peering: An Early Engagement Case Study Murali Suriar, msuriar@google.com On behalf of Google Technical Infrastructure and Network Infrastructure SRE August 30, 2017 Who am I? Murali Suriar