Prof Ian F Bitterlin

Transcription

1 Prof Ian F Bitterlin CEng PhD BSc(Hons) BA DipDesInn ATD FIET MCIBSE MBCS Visiting Professor, School of Mechanical Engineering, University of Leeds Data Centre Infrastructure Consulting Engineer

2 Is growth in capacity assured? Which Tier rating? Cost comparisons Multi-Tier, multi-tenant, facilities Sustainability drivers and how Norway fits in what is coming in the standards world? Size is bigger better? Is energy effectiveness the only target? Electrons versus photons, exporting hydro?

3 2001: 690 TB/month 2015: 690,000 TB/month 1000x 54% CAGR* 4% CGR per month Peak 3.34 TB/s Capacity TB/s *CAGR = Compound Annual Growth Rate

4 Gordon Moore, founder of Intel, wrote his Law in 1965 when he predicted the doubling of the number of transistors on a microprocessor every two years It was a physical photo-etching process improvement cycle that has held true ever since Directly applies to Doubling compute capacity Halving the Watts/FLOP Halving kwh per unit of compute load Law updated by Intel to 18 months for clock-rate increase Encourages ever-shorter hardware refresh rates Facebook <12 months slows down their data centre growth Google was <36 months refresh rate, now much shorter Keeping ICT hardware >2 years is now energy profligate

5

6 1947, Bell Labs, the first transistor was hand built 2012, Intel 22nm tri-gate transistor 100 million into a 1.8mm pin-head 6 million onto a typed full-stop, 0.1mm 2 3 rd Generation quad-core has 1.48 Billion transistors Each can switch on/off 100 Billion times per second Pundits predict that 22nm is the end of Moore s Law Intel make 5 Billion transistors every second 20 million per person on the planet, per year ,000x faster, 1/5,000 th energy, 1/50,000 th cost

7 100% in % in 2015 Moore's Law Clock-rate PUE, 3 to 1.5 Virtualization, 0-40% Utilization, 5-40% 1,000x

8 What s the difference between a mega data centre and a micro data centre? The answer is not size... The answer is time

9 Best-in-class, today 1000x 2001 Best practice, today 700x 2001 Best practice, 4 year old base 100x IDC 5MW site 2MW IT load 2000 cabinets ~3000m Best Practice/4y 45kW site 30kW IT load 5 cabinets ~16m Best-in-class, today 2.2kW site 2kW IT load <1 cabinet ~3m 2 If business is fixed ICT power can decline quickly if server hardware is regularly refreshed

10 2001: LD photos and pornography 2015: HD photos, YouTube, Gaming, Gambling and Films Limits to growth? 7Bn people (not growing exponentially) each with a maximum viewing time of 24h/day Exponential growth is coming from The-Internetof-Things with multi-quadrillion sensors, each transmitting a stream of short data packets Underpinned by growth from the third world

11 Tier/Availability Class/Type/F(x) Not 4 in bulk very few users will pay for it and energy effectiveness cannot be maximised 3 (often with dual-bus enhanced power, e.g. 3+) is the majority of today s demand including for the finance/banking sector 2 describes most cloud service with multiple sites allowing the maintainability normally offered by Tier III 1 hardly exists in reality Security in line with the client expectations

12 Multi-tenant Consider scalable modular facilities (not containerised ) Choose a module size carefully, kW? Many tenants don t like sharing infrastructure Avoid DR office space if possible Multi-Tier Much easier to do in small sections Multi-density 3kW/cabinet covers most 2015 collocation 5-6kW/cabinet is the average real load High density market of c10kw is small 20-30kW is very rare indeed Partial load is endemic in data centres so ensure the design can transfer under-utilised power to other areas

13 Constrained by the available market and the available utility capacity There is little economy of scale above 5MVA utility connection Small facilities in city-centres provide the best opportunity for reuse of waste heat, e.g. 250kW data centre embedded in an office building

14 The best design minimises the initial investment and offers true invest-as-you-sell without disruption to existing tenants Higher density saves little cost as the realestate/build cost is a small proportion of the electrical and mechanical services and the danger of partial load rises so a scalable UPS design is recommended Land cost is usually low in relation to technical services

15 Not including the cost of land, utility provision, fibre provision, cabinets and ICT hardware Including centralised office/support building and site works 3MVA transformer, 2MW UPS, chilled water with freecooling, containerised gensets, PUE 1.2 1,000m 2 net technical 2kW/m2 for 330 cabinet locations of 6kW cabinets Cost per kw of IT power Tier 1 2,700 M&E plus 2,200 Construction Tier 2 3,200 M&E plus 2,400 Construction Tier 3 4,000 M&E plus 2,600 Construction Tier 4 5,400 M&E plus 3,000 Construction Energy cost per year = 12m at 60% load = 50% 10y TCO

16 Reduce consumption What is the social & economic value of the data processed? Virtualization, high utilisation, frequent ICT refresh Improve: 2014 average server runs at 10% utilisation on 40% power Restrict access for certain applications? De-duplicate data Improve efficiency (effectiveness) Moore s Law plus a paradigm shift in network photonics? PUE, WUE, REF, CUE, RWH, ITEE & ITEU +... Power from renewable energy sources Building a legacy data-centre next to a hydro-electric scheme is NOT a sustainable design, it could be a waste of renewable energy on an ineffective data centre if the first two steps are not taken

17 EU (DG Connect) is starting down the road of regulating data centre power Fixated by renewables for ICT EU is discussing (but will reject as political suicide) the throttling-back of broadband for certain applications but regulation or taxation? Remove HD-cameras from mobile-phones? Natural limits on network capacity will place Jevons type barriers upon unlimited data bundles Users will not reverse the demand trend unless tax or cost intervenes

18 ISO/IEC KPIs for Resource Effective Data Centres Best practice data centres lower PUE Better ICT hardware high efficiency SMPS ITEE Higher utilisation (more virtualisation) - ITEU Low WUE Low CUE, high REF = utility constrained? Reuse of waste heat, RWH

19 The EU is fixated in using ICT as an energy exemplar There is no logical reason why ICT should be powered by renewable power more than any other application like research, medicine, education or social services On-site solar-pv power can only contribute <1.5% to a data centres power demand, everything else has to be imported Norway has 99.5% renewable utility power

20 Reduce consumption of ICT? No chance! Increase ICT hardware effectiveness and dramatically improve utilisation Minimise PUE to meet business case Cooling ppue is still the low hanging fruit Eco-mode UPS Re-use waste heat Hard to do in bulk All of these, and more, will be in ISO/IEC 30134

21 The industry talks a lot about energy saving but enduser don t do much, why? The cost of energy is very often a minor issue with respect to the cost of ICT failure and the business enabled by ICT 1kWh = NOK of UK business product At NOK 1/kWh = 0.1% of costs Cutting data centre power by 25% = 0.025% impact So, energy effectiveness is important, but not (yet) paramount and avoidance of business failure dominates behaviour... Norwegian data centres should not expect imported business based entirely on energy cost... Or CUE/REF

22 Norwegian gas powers Europe Norwegian hydro-power could power European data centres through the deregulated markets Example 2500MW (150% of all the UK s data centres) 1000km 800kV HVDC Losses 2.6%

23 Cost drives attitude and CSR behaviour ASHRAE TC9.9 has been widening the Recommended and, much faster, the Allowable thermal envelopes Data centres will be hot, not cold The Green Grid Data Center Maturity Model Eco-mode UPS and no refrigeration, even in back-up EU CoC Best Practice Guide Eco-mode UPS and 100% free-cooling ISO/IEC X pushing resource effectiveness Strong preference for renewable power ICT by the EU

24 Standards EN Infrastructure standard ISO/IEC Metrics are coming along ASHREA TC9.9 Thermal Guidelines are not a standard but nothing else is published globally and is, by default, a standard TIA-942-A is an ANSI Standard now a schism with TUI BICSI Design Guide is an ANSI Standard Plenty of Best Practice & Design guides The Uptime Institute, now withdrawn TIA 942-A, can be applied outside of North America BICSI, quite North American centric, cabling based EU CoC The Green Grid DCMM Certifications incl UTI, CEEDA, DCA + bespoke to standards

25 Power PUE overhead, annualised energy, kwh/kwh CUE carbon content of utility, kg/kwh REF renewable energy content of utility and local, factor Water WUE potable and grey, Litres/kWh Waste RWH reuse of waste heat, both capacity and actual use Recycling similar to the next version of the GG DCMM Others...

26 The RWH metric is coming from ISO/IEC Importance will be on the efficiency of the transfer AND the amount actually reused Difficult to find constant (high) heat loads 8760h Low energy cooling produces low grade heat Data centres are very power dense 50x a standard office environment Mega-centres are distant from potential loads Maybe small water cooled rooms are part of the future? Higher PUE to gain high RWH?

27 If you want to grasp it yes N+1 Indirect evaporative cooling ppue = N advanced eco-mode modular UPS ppue = 1.02 Lights, NOC, security etc ppue = 1.02 PUE = ASHRAE Recommended envelope, no fresh-air in the room, no contamination and humidity controlled to client requirements Typical WUE = 0.12 L/kWh External wet-bulb 23⁰C = max server inlet temperature of 27⁰C <5ms zero-volts power fidelity, better than the 10ms limit of the 2012 Typical server Could be even lower if risks are acceptable Single-bus, direct fresh-air cooling, ASHRAE Allowable

28 Maybe, except for... City-centre locations? Reuse of waste heat? Smart Cities? But the industry is slow to change PUE versus RWH?

29 ppue Evaporative Cooling Units - Indirect Evaporative Air Freecooling - Range: 170 to 350 kw - Perimeter or Roof installation - Reduced installation costs, located outside the white space Rack Arrangements - Hot or Cold containment - Control of air at server level Optimizing the System Control of air flow and temperature in front of the servers Always provides the right quantity of air at the right temperature Downside: Needs a specific building Downside: Low grade waste heat

30 Abundant power (unlike the UK!) 99.5% renewable Cost not related to oil-price 100% free-cooling without risks to load No water shortages Educated workforce Politically and ethically stable, moral Lots of land Reuse of waste heat in city-centres is established

31 Low population and high internet penetration so major business must come from outside the region The cool climate advantage can be wholly or partially negated by evaporative systems in warmer climates Norway s renewable power can be exported as easily as Norwegian gas The perception of connectivity capacity and speed High cost base Reuse of waste heat precludes mega-centres

32