CSE 124: THE DATACENTER AS A COMPUTER. George Porter November 20 and 22, 2017

Transcription

1 CSE 124: THE DATACENTER AS A COMPUTER George Porter November 20 and 22, 2017

2 ATTRIBUTION These slides are released under an Attribution-NonCommercial-ShareAlike 3.0 Unported (CC BY-NC-SA 3.0) Creative Commons license These slides incorporate material from: The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines, 2nd ed., by Barroso, Clidaras, and Hölzle.

3 ANNOUNCEMENTS

4 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

5 DATA CENTERS WITH 100,000+ SERVERS Microsoft Google Facebook

6 THESE THINGS ARE REALLY BIG 100 billion searches per month 1.15 billion users 120+ million users

7 CLOUD COMPUTING Elastic resources Expand and contract resources Pay-per-use Infrastructure on demand Multi-tenancy Multiple independent users Security and resource isolation Amortize the cost of the (shared) infrastructure Flexible service management Resiliency: isolate failure of servers and storage Workload movement: move work to other locations

8 CLOUD SERVICE MODELS Software as a Service (Saas) Provider licenses applications to users as a service e.g., customer relationship management, , Avoid costs of installation, maintenance, patches, Platform as a Service (Paas) Provider offers software platform for building applications e.g., Google s App-Engine Avoid worrying about scalability of platform Infrastructure as a Service (Iaas) Provider offers raw computing, storage, and network e.g., Amazon s Elastic Computing Cloud (EC2) Avoid buying servers and estimating resource needs

9 NOT JUST A COLLECTION OF SERVERS A data center isn t just a small internet Why? Administered as a single domain Trusted administrators No need to be compatible with the outside world Except for traffic to/from users No need for international standards bodies Though why do standards help?

10 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

11 FRONT-END DATACENTER TRAFFIC Wide-area Internet Data center Internet Users

12 FRONT-END DATACENTER TRAFFIC Data center Wide-area Internet Data sizes driven by the content that users actually consume Growth largely due to higher bitrate content (IP TV/movies, iphone Facetime) Mobile Internet source of new users Often constrained by the last mile Web Video Voice Music Photos

13 MULTI-TIER APPLICATIONS Applications consist of tasks Many separate components Running on different machines Front end Server Commodity computers Many general-purpose computers Not one big mainframe Easier scaling Aggregator Aggregator Aggregator Aggregator Worker 13 Worker Worker Worker Worker

14 BACK-END DATACENTER TRAFFIC Back-end analytics: Connections between information Users who bought X also bought Y Key differentiator determining success Facebook vs Friendster Amazon vs Buy.com Large-scale join computations spanning thousands of nodes Need bandwidth as well as all-to-all connectivity Sorting / Searching Collaborative Filtering Map/Reduce Distributed Key/Value stores Video storage, postproduction, and transmission

15 DATA-INTENSIVE APPLICATION REQUIREMENTS All-to-all Gather/Scatter Performance gated on latency of shuffle phases Need high bisection bandwidth Performance gated on speed of slowest RPC/parallel operation Need low variance

16 INCREASINGLY STRINGENT NETWORK REQUIREMENTS Low one-way latency (10s of microseconds) 10/40 Gbps at TOR (and soon endhosts) Congestion-free operation/low queuing Dynamic traffic and an increasingly dynamic topology

17 FROM NETWORKS TO BACKPLANES Before: Network connects servers to users (FTP, telnet, ) Massive computing = tightly coupled supercomputer Proprietary interconnects Working sets of data Today: Servers connected to each other Data-intensive, web-scale Massive computing = Datacenters Commodity Datacenter network becomes the computing backplane

18 DATA CENTER CHALLENGES Traffic load balancing Support for VM migration Achieving bisection bandwidth Power savings / Cooling Network management (provisioning) Security (dealing with multiple tenants)

19 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

20 BUILDING BLOCKS OF MODERN DATA CENTERS Network switch Rack

21 TOP-OF-RACK ARCHITECTURE Rack of servers Commodity servers And top-of-rack switch Modular design Preconfigured racks Power, network, and storage cabling Aggregate to the next level

22 RACKS OF SERVERS (GOOGLE)

23 FACEBOOK

24 GOOGLE

25 EXTREME MODULARITY Containers Many containers

26 HOST VIRTUALIZATION Multiple virtual machines on one physical machine Applications run unmodified as on real machine VM can migrate from one computer to another

27 VMM VIRTUAL SWITCHES

28 THE STORAGE HIERARCHY

29 LATENCY, BANDWIDTH, AND CAPACITY

30 PERFORMANCE OF FLASH

31 HARDWARE COMPARISONS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

32 BIG COMPUTER VS. LOTS-OF-SMALL-COMPUTERS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

33 BIG COMPUTER VS. LOTS-OF-SMALL-COMPUTERS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

34 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

35 400V Automatic Transfer Switch Xiaobo Fan, Wolf-Dietrich Weber, and Luiz Andre Barroso Power provisioning for a warehouse-sized computer. In ISCA '07.

36 HEAT MANAGEMENT CRAC unit rack rack rack rack CRAC unit floor tiles Liquid supply Regents of the University of Michigan

37 QUANTIFYING ENERGY-EFFICIENCY: PUE PUE = Power Usage Effectiveness Simply compares Power used for computing vs Total power used Historically cooling was a huge source of power E.g., 1 watt of computing meant 1 Watt of cooling! PUE = (Facility Power) / (Computing Equipment power)

38 LBNL PUE SURVEY (2007) Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

39 LBNL PUE Survey (2013)

40 BREAKDOWN OF DATA CENTER OVERHEADS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

41 NUMBERS FROM JAMES HAMILTON (MSFT, AMAZON)

42 LIMITS OF LOWERING PUE? Google s Chillerless data center in Belgium Most of the year it is cool enough to not need cooling What about on hot days? Shed load to other data centers!

43 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

44 POWER-PROPORTIONAL HUMANS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

45 WEB-SERVICE LOAD FLUCTUATIONS Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

46 DO DIFFERENT COMPONENTS SCALE SIMILARLY? Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

47 CPU UTILIZATION Barroso and Holzle, The Data Center as a Computer: An introduction to the Design of Warehouse-Scale Machines

48 WHAT ABOUT POWER SAVING FEATURES ON MODERN COMPUTERS?

49 POWER USAGE VS. UTILIZATION

50 POWER SUPPLY EFFICIENCY AND SPUE

51 OUTLINE 1. Datacenter overview 2. Traffic patterns 3. Components and building blocks 4. Power and energy 5. Power-proportional computing 6. Networking and power

52 TRADITIONAL DC TOPOLOGY Core Internet Layer-3 router Data Center Aggregation Layer-2/3 switch Access Layer-2 switch Servers

53 LAYER 2 PODS W/L3 BACKBONE Internet DC-Layer 3 DC-Layer 2 CR CR AR AR... AR AR S S S S S S... ~ 1,000 servers/pod == IP subnet Key CR = Core Router (L3) AR = Access Router (L3) S = Ethernet Switch (L2) A = Rack of app. servers

54 CAPACITY BOTTLENECKS CR CR ~ 200:1 AR AR AR AR S S S S S ~ 40:1 ~ S5:1 S S... S S S S Discussion: Implications for energy efficiency? Recall:

55