Building warehouse-scale computers or what s it like to supply exponential growth. john wilkes

Transcription

1 Building warehouse-scale computers or what s it like to supply exponential growth john wilkes

2 Video:

3 You re all Some of you are thinking too small

4 Scale has been the single most important force driving changes in system software over the last decade Technical perspective: Is scale your enemy, or is scale your friend? John Ousterhout, CACM 54(7):110, July 2011.

5 The kind of things we like to be able to do

6 Here s the system! Image The Dalles, Oregon

7 What if the system included not just the computers, but the network fabric and the WAN endpoints Council Bluffs, Iowa

8 and the cooling system

9 and the building-management system St Ghislain, Belgium

10 and the power system and the technicians Backup generator at St. Ghislain, Belgium

11 In fact the entire warehouse scale computer

12 By the way it costs O($200M). No pressure.

13 Key challenges Add capacity optimally The Fleet Keep capacity optimized

14 Planning for compute resources All have multiple generations/options CPU RAM disk, SSD capacity, performance new NVRAM thingies... accelerators (complication: a wide variety) inter- and intra-datacenter networking power datacenters land, water, sewage

15 Response times are weeks/months/years, not milliseconds

16 Planning for network Google network: 12 years to build and still growing Edge points of presence (>100)

17 How long does it take to build capacity? Note: variance is as bad as delay

18 Planning for compute resources What kind of machine to buy? different groups want different things (e.g., search, Cloud) MotD + customizations Idea: reuse machines when they get handed down when is it worthwhile? (price of power? room for expansion?) Also: do we have space? power? networking? budget? models; what-if analyses; uncertainty (demand, supply) objective function: total cost-of-ownership

19 Planning for compute resources: Total Cost of Ownership (TCO) what do you mean by total? average? over what? continent? time? depreciation schedule? what do you mean by cost? initial purchase price, or the average over time? is delivery or installation included? what if they are being reused? what do you mean by ownership? machines can be transferred/given/sold/break who owns a machine running a shared service for a customer?

20

21 A simplified overview demand forecast time sites, data centers, power network, machines, storage usage $$ prices compute/storage capacity

22 In a world of exponential demand growth Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google s Datacenter Network. SIGCOMM 2015.

23 In a world of exponential demand growth Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google s Datacenter Network. SIGCOMM 2015.

24 In a world of exponential demand growth Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google s Datacenter Network. SIGCOMM Prediction is very difficult, especially about the future Mark Twain(?) via Neils Bohr

25 A few factors affecting forecasts user growth new features better software better hardware (e.g., accelerators) time

26 How much capacity do we need? and when do you need to know? PA forecasts more forecasting magic firm demand aggregate quantities detailed orders time

27 Putting it all together 1 second 1 minute 1 hour 1 day 1 week 1 month 1 year 2 years 5 years Forecasts resource demand Borg resource reclamation online workload prediction short-term PA demand forecasts mid-tiers long-term fleet forecasts

28 It takes a few moving parts...

29 Cloud Datastore transactions/s

30 Planning for power PUE (Power Usage Effectiveness) = total Watts / compute Watts smaller is better industry average ~1.7

31 Planning for power High PUE ML Control On ML Control Off Low PUE

32 Planning for power Some of the challenges Narrow range of experience Exploration may discover unsafe states Inputs out of our control (e.g., weather) Control system reliability/availability Agility (new hardware) Reinforcement learning with long delays between action and change in system state counts Safety. Safety. Safety. setpoint 1 Graphics by Steve Webster setpoint 2

33 We re set to reach 100% renewable energy in 2017

34

35 Meanwhile what s up with Moore s law? Single-core performance plateauing after decades of exponential growth Graph from 40 Years of Microprocessor Trend Data, Karl Rupp, CC-BY 4.0.

36 Meanwhile what s up with Moore s law? Tensor Processing Units

37 Meanwhile what s up with Moore s law? Tensor Processing Unit pods...

38 Putting it all together Operations Delivery + fulfillment Supply Usage Monitor + report Supply requests (orders) Demand forecasting Demand Planning

39 Putting it all together a few more details Operations Fulfillment Supply (current + future) Monitoring Usage data Customer demand system Monitor + report Supply planning Ordering Historic-demand system Demand forecast system High-level services + lower-level stuff Borg, Colossus + lower-level stuff Machines, racks,... Datacenters, power, parts

40 Putting it all together a few more details Operations Fulfillment Supply (current + future) Monitoring Usage data Customer demand system Monitor + report Supply planning Ordering Historic-demand system Demand forecast system Policies High-level services + lower-level stuff Borg, Colossus + lower-level stuff Machines, racks,... Datacenters, power, parts

41 Putting it all together Operations Delivery + fulfillment Supply Usage Monitor + report Supply requests (orders) Demand forecasting Demand Planning

42 Putting it all together one Google-wide supply 0-6 months machines Product PAs PAs Areas each PA does these Policies 43

43 Putting it all together multiple timelines 0-6 months machines Product PAs PAs Areas Policies supply forecast months parts Policies supply forecast 24+ months power, data centers Policies

44 Image by Connie Zhou

45 2018 Q1 CapEx = $5.3B (+$2.4B for an office building in New York) source: Alphabet SEC filing $29.4B 3-year trailing CapEx, as of March 2017

46 Final thoughts: There s a lot of technology behind the cloud At scale, efficiency really matters [and: we re hiring!] johnwilkes@google.com Images by Connie Zhou