Evolution of Facebook s data center design. Veerendra Mulay, PhD Strategic Engineering and Development

Transcription

1

2 Evolution of Facebook s data center design Veerendra Mulay, PhD Strategic Engineering and Development

3 Agenda 1 Introduction 2 Electrical design 3 Mechanical design 4 Design evolution 5 RDDC

4 Introduction

5 Facebook Greenfield Datacenter Goal Design and build the most efficient datacenter eco-system possible Control Application Server configuration Datacenter design Sites Prineville, OR Forest City, NC Luleå, Sweden Altoona, IA

6 Prineville, OR

7 Forest City, NC

8 Luleå, Sweden

9 Altoona, IA

10 Electrical design

11 Electrical Overview Eliminate 480V to 208V transformation Used 480/277VAC distribution to IT equipment Remove centralized UPS Implemented 48VDC UPS System Result a highly efficient electrical system and small failure domain

12 Typical Power Prineville Power Utility Transformer 480/277 VAC 2% loss Standby Generator Utility Transformer 480/277 VAC 2% loss Standby Generator AC/DC DC/AC UPS 480VAC 6% - 12% loss ASTS/PDU 208/120VAC 3% loss % Availability 480/277VAC % Availability SERVER PS 10% loss (assuming 90% plus PS) FB SERVER PS 5.5% loss Total loss up to server: Total loss up to server: 21% to 27% 7.5% 48VDC DC UPS (Stand-by)

13 DC UPS Backup Scheme 48VDC Battery Cabinet 277VAC 48VDC Battery Charger Server PS Normal Power Backup Power 4x12V Batt 4x12V Batt 4x12V Batt 48VDC Standby Back-up Converter 48VDC- 12VDC AC to DC 277VAC- 12VDC 12VDC 4x12V Batt 4x12V Batt MOBO

14 Battery Cabinet Custom DC UPS 56kW or 85kW 480VAC, 3-phase input 45 second back-up 20 sealed VRLA batteries Battery Validation System Six 48VDC Output Two 50A 48VDC aux outputs

15 Reactor Power Panel Custom Fabricated RPP Delivers 165kW, 480/277V, 3-phase to CAB level Contains Cam-Lock connector for maintenance wrap around Line Reactor Reduces short circuit current < 10kA Corrects leading power factor towards unity (3% improvement) Reduces THD for improved electrical system performance (ithd 2% improvement) Power consumption = 360 Watt

16 Mechanical design

17 Overview Removed Centralized chiller plant HVAC ductwork System Basis of Design ASHRAE Weather Data: N=50 years TC : Recommended Envelopes Built-up penthouse air handling system Server waste heat is used for office space heating

18 Typical Datacenter Cooling Return Ductwork CT CHILLER AHU SUPPLY DUCT DATA CENTER Facebook Data Center Cooling Return Air Ductless Return Air Relief Air Ductless Relief Air 100% Outside Air Intake DUCTLESS SUPPLY DATA CENTER Return Air Plenum Filter Wall DX System Misting System Fan Wall

19 PRN1 Datacenter Cooling Filter Wall Misting System Fan Wall 100% Outside Air Intake Mixed Air Corridor Supply Air Corridor Relief Air Corridor Relief Air Fan Hot Aisle Return Air Common Return Air Plenum Hot Aisle Relief Air Hot Aisle Common Cold Aisle Hot Aisle Data Center Server Cabinets Server Cabinets

20 Vertical drainable louvers Outside air intake corridor

21 Air mixing section Evaporative cooling / humidification corridor

22 DX Cooling Coil Forest City Only 50 year extreme weather conditions

23

24

25 Fan wall array inlet corridor Fan array outlet / supply air corridor

26 Cold aisle pressurization ductless supply

27

28

29 Design evolution

30 Basis of Design Comparison 80ºF inlet 85ºF inlet 85ºF inlet 85ºF inlet 65% humidity 80% humidity 90% humidity 80% humidity 20ºF ΔT 22ºF ΔT 22ºF ΔT 22ºF ΔT PRN1A1B PRN1C1D FRC1A1B LLA1A1B

31 Wetted Media

32 Typical Generator Topology

33 LLA Reduced Generator Topology

34 Rapid Deployment Data Center

35 PRE MANUFACTURED UNIT ASSEMBLIES RDDC FLAT PACKED PANELIZED ASSEMBLIES

36 RDDC Pre-fab Assemblies

37 RDDC building plans MECH UNITS MECH UNITS A ELEC SKIDS B MECH UNITS MECH UNITS

38 RDDC Unitized IT Modules

39 RDDC Section Unit IT Unit IT a b c

40 RDDC Power Modules

41 Luleå, Sweden

42 RDDC Flat Pack Solution

43 Flat pack solution RACK RACK 14-0

44 Flat pack solution

45 Flat pack solution RACK RACK

46 Leaner simplification

47

48 (c) 2007 Facebook, Inc. or its licensors. "Facebook" is a registered trademark of Facebook, Inc.. All rights reserved. 1.0