Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism

Transcription

1 Warehouse-Scale Computers to Exploit Request-Level and Data-Level Parallelism

2 The datacenter is the computer Luiz Andre Barroso, Google (2007)

3 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

4 Introduction Warehouse-scale computer (WSC) Provides Internet services Search, social networking, online maps, video sharing, online shopping, , cloud computing, etc. Differences with HPC clusters : Clusters have higher performance processors and network Clusters emphasize thread-level parallelism, WSCs emphasize request-level parallelism Differences with datacenters: Datacenters consolidate different machines and software into one location Datacenters emphasize virtual machines and hardware heterogeneity in order to serve varied customers

5 Introduction Important design factors for WSC: Cost-performance Small savings add up Energy efficiency Affects power distribution and cooling Work per joule Dependability via redundancy Network I/O Interactive and batch processing workloads Ample computational parallelism is not important Most jobs are totally independent Request-level parallelism Operational costs count Power consumption is a primary, not secondary, constraint when designing system Scale and its opportunities and problems Can afford to build customized systems since WSC require volume purchase

6 Introduction - Failures Outages and anomalies with the approximate frequencies of occurrences of a cluster with 2400 servers

7 Introduction The WSC goal is to make the hardware/software in the warehouse act like a single computer that typically runs a variety of applications. The largest cost in a conventional datacenter is the people to maintain it, whereas, in a well-designed WSC the server hardware is the greatest cost, and people costs shift from the topmost to nearly irrelevant.

8 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

9 Programming Models and Workloads Batch processing framework: MapReduce Map: applies a programmer-supplied function to each logical input record Runs on thousands of computers Provides new set of key-value pairs as intermediate values Reduce: collapses values using another programmersupplied function A generalization of the single-instruction, multiple-data (SIMD) operation

10 Programming Models and Workloads Example: map (String key, String value): // key: document name // value: document contents for each word w in value EmitIntermediate(w, 1 ); // Produce list of all words reduce (String key, Iterator values): // key: a word // value: a list of counts int result = 0; for each v in values: result += ParseInt(v); // get integer from key-value pair Emit(AsString(result));

11 Programming Models and Workloads MapReduce runtime environment schedules map and reduce task to WSC nodes Availability: Use replicas of data across different servers Replicas can help with read performance as well as with availability; with proper placement Use relaxed consistency: No need for all replicas to always agree Workload demands Often vary considerably

12 Programming Models and Workloads Annual MapReduce Usage at Google over time

13 Features of WSC To reduce operational costs of availability, all WSCs use automated monitoring software so that one operator can be responsible for more than 1000 servers. Programming frameworks rely upon internal software services for their success Google FileSystem, BigTable, Dynamo

14 Features of WSC The workload demands of public interactive services all vary considerably even a popular global service such as Google search varies by a factor of two depending on the time of day It is more important for servers in a WSC to perform well while doing little than to just to perform efficiently at their peak, as they rarely operate at their peak. Average CPU utilization of more than 5000 servers during a 6-month period at Google. WSC hardware and software must cope with variability in load based on user demand and in performance and dependability due to the vagaries of hardware at this scale

15 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

16 Computer Architecture of WSC WSCs often use a hierarchy of networks for interconnection

17 Computer Architecture of WSC The 19-inch (48.26-cm) rack is the standard framework to hold servers Each 19-inch rack holds 48 1U servers connected to a rack switch Bandwidth within the rack is the same for each server It does not matter where the software places the sender and the receiver Rack switches are uplinked to switch higher in hierarchy Uplink has 48 / n times lower bandwidth, where n = # of uplink ports Oversubscription Goal is to maximize locality of communication relative to the rack

18 Storage Storage options: Use disks inside the servers, or Network attached storage (NAS) through Infiniband NAS solution is generally more expensive per terabyte of storage, but it provides many features, including reliability WSCs generally rely on local disks Google File System (GFS) uses local disks and maintains at least three replicas

19 Array Switch Switch that connects an array of racks Array switch should have 10 X the bisection bandwidth of rack switch Cost of n-port switch grows as n 2 Often utilize content addressable memory chips and FPGAs

20 WSC Memory Hierarchy Servers can access DRAM and disks on other servers using a NUMA-style interface Each server has 16GB of memory Every pair of racks includes one rack switch and holds 80 2U servers The array switch can handle 30 racks

21 WSC Memory Hierarchy (2) Network overhead increases latency from local DRAM to rack DRAM and array DRAM, but still have more than 10X better latency than local disk WSC needs 20 arrays to reach servers, one more level of the networking hierarchy is added.

22 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

23 Infrastructure and Costs of WSC Location of WSC Proximity to Internet backbones Low electricity cost Low property tax rates Near Internet users Low risk from earthquakes, floods, and hurricanes

24 Infrastructure and Costs of WSC Power distribution UPS: uninterruptible power supply. Located in a separate room from the IT equipment Power conditioning, holding the electrical load when switching

25 Infrastructure and Costs of WSC Efficiency of conversion The substation switches from 115,000 volts to medium-voltage lines of 13,200 volts, with an efficiency of 99.7%. The efficiency of a large UPS is 94%, so the facility loses 6% of the power by having a UPS Power distribution unit (PDU) converts to low-voltage, internal, threephase power at 480 volts. conversion efficiency is 98%. Another down step to two-phase power at 208 volts that servers can use, once again at 98% efficiency. The connectors, breakers, and electrical wiring to the server have a collective efficiency of 99% Overall efficiency 99.7% * 94% * 98% * 98% * 99%=89%

26 Infrastructure and Costs of WSC Cooling Air conditioning used to cool server room using chilled water 18 C 22 C Keep temperature higher (closer to 22 C) fans push warm air past a set of coils filled with cold water and a pump moves the warmed water to the external chillers to be cooled down.

27 Infrastructure and Costs of WSC Cooling (fans and water pumps that move air and water)

28 Infrastructure and Costs of WSC Cooling towers can also be used Leverage the colder outside air to cool the water before it is sent to the chillers Minimum temperature is wet bulb temperature Warm water flows over a large surface in the tower, transferring heat to the outside air via evaporation and thereby cooling the water. This technique is called airside economization. An alternative is use cold water instead of cold air. Google s WSC in Belgium uses a water-to-water intercooler that takes cold water from an industrial canal to chill

29 Infrastructure and Costs of WSC Airflow is carefully planned. Efficient designs preserve the temperature of the cool air by reducing the chances of it mixing with hot air. Cooling system also uses water (evaporation and spills) E.g. 70,000 to 200,000 gallons per day for an 8 MW facility Power cost breakdown: Chillers: 30-50% of the power used by the IT equipment Air conditioning: 10-20% of the IT power, mostly due to fans How man servers can a WSC support? Each server: Nameplate power rating gives maximum power consumption To get actual, measure power under actual workloads Oversubscribe cumulative server power by 40%, but monitor power closely

30 Infrastructure and Costs of WSC Breaking down power usage inside the IT equipment itself 33% of the power for processors 30% for DRAM 10% for disks 5% for networking 22% for other reasons (inside the server)

31 Measuring Efficiency of a WSC Power Utilization Effectiveness (PEU) = Total facility power / IT equipment power Median PUE on 2006 study was 1.69 The bigger the PUE, the less efficient the WSC Performance Latency is important metric because it is seen by users Cutting system response time 30% shaved the time of an inter- action by 70%.

32 Measuring Efficiency of a WSC Bing study: users will use search less as response time increases A high percentage of requests be below a latency threshold rather just offer a target for the average latency. Service Level Objectives (SLOs)/Service Level Agreements (SLAs) E.g. 99% of requests be below 100 ms

33 Measuring Efficiency of a WSC

34 Cost of a WSC Capital expenditures (CAPEX) Cost to build a WSC Operational expenditures (OPEX) Cost to operate a WSC

35 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

36 Cloud Computing Datacenters tend to be utilized only 10% to 20% of the time. By making WSCs available to the public, uncorrelated peaks between different customers can raise average utilization above 50%. WSCs offer economies of scale that cannot be achieved with a datacenter: 5.7 times reduction in storage costs 7.1 times reduction in administrative costs 7.3 times reduction in networking costs This has given rise to cloud services such as Amazon Web Services Utility Computing

37 Cloud Computing - AWS Virtual Machines Protect users from each other Simplify software distribution within a WSC Easy to control resource usage Enable multiple price points Hide the identity of older hardware Introduce new and faster hardware Very low cost Reliance on open source software No initial guarantee of service No contract required Pay-for-use

38 Cloud Computing Cloud computing providers take on the risks of over-provisioning and under-provisioning Cloud computing has made the benefits of WSC available to everyone 1000 servers for a hour cost no more than 1 server for 1000 hours

39 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

40 WSC network as a Bottleneck Large switches are manually configured and fragile at a large scale It is difficult to afford more than dual redundancy in a WSC using these large switches, which limits the options for fault tolerance WSC network bottlenecks constrain data placement and complicate WSC software

41 Energy Efficiency Another source of electrical inefficiency is the power supply inside the server Power supplies were 60%~80% efficient, thus there were greater losses inside the server Supply a range of voltages to chips and disks

42 Energy Efficiency Energy Proportionality servers should consume energy in proportion to the amount of work performed

43 Outline Introduction to WSCs Programming Models and Workloads Computer Architecture of WSCs Infrastructure and Costs of WSCs Cloud Computing Crosscutting Issues A Google WSC

44 Containers Google built WSCs using shipping containers. The idea of building a WSC from containers is to make WSC design modular

45 Cooling By controlling airflow to prevent hot spots, the container can run at a much higher temperature. The cooling is below a raised floor that blows into the aisle between the racks Hot air is returned from behind the racks. The restricted space of the container prevents the mixing of hot and cold air, which improves cooling efficiency Two racks on each side of the container

46 Power Rather than have a separate battery room, each server has its own lead acid battery that is 99.99% efficient Deployed incrementally with each machine, thus no money or power spent on overcapacity Use standard off-the-shelf UPS units to protect network switches DVFS was not deployed to avoid quality-of-service violation for online workloads

47 Server Two sockets Eight DIMMs Single network interface card (NIC) Two sata disk drives

48 Networking servers are divided into three arrays of more than servers each The 48-port rack switches uses 40 ports to connect to servers, leaving 8 for uplinks to the array switches The number of uplink ports used per rack switch varies from a minimum of 2 to a maximum of 8 Applications with significant traffic demands beyond a rack tended to suffer from poor network performance

49 Monitoring and Repair Use monitoring software to track the health of all servers and networking gear Diagnostics are running all the time. When a system fails, many of the possible problems have simple automated solutions Failed machines are addressed in batches to amortize the cost of repair

50 Next Lecture: Prof. Wenli Zheng

51 谢谢! Thanks! Q&A