How to Manage Petabytes of Data in an Environmentally Friendly way using Less Power, Less Cooling, Less Cost. David Honey Storage Solutions Architect

Size: px

Start display at page:

Download "How to Manage Petabytes of Data in an Environmentally Friendly way using Less Power, Less Cooling, Less Cost. David Honey Storage Solutions Architect"

Noah Lawson
6 years ago
Views:

1 How to Manage Petabytes of Data in an Environmentally Friendly way using Less Power, Less Cooling, Less Cost David Honey Storage Solutions Architect

2 Agenda Where does energy come from and go to? Mass storage device power use; disk, tape, optical Multi tier vs disk only HSM is green What SGI is doing to help customers reach sustainability Q & A

3 Where does electricity come from?! 43 billion kwh of electricity generated in NZ in 2006 Source: World Nuclear Association

Australian electricity sources are very different! In 2006 power stations produced 255 billion kilowatt hours (TWh) of electricity, 65% more than the 1990 level and growing at 3.3% pa.

4 Australian electricity sources are very different! In 2006 power stations produced 255 billion kilowatt hours (TWh) of electricity, 65% more than the 1990 level and growing at 3.3% pa.! 18 TWh is used by power stations, leaving 237 TWh net production. 17 TWh is lost or used in transmission and 9-10 more in energy sector consumption, leaving 210 TWh for final consumption (or 187 TWh apart from aluminium exports)! Much of the energy exported is used for generating electricity overseas; three times as much black coal is exported as is used 25% of electricity produced is lost in generation and transmission 90% energy sources are fossil fuels Source: World Nuclear Association

Power: Energy used per unit of time Power Energy Energy (Watts) (Wh p.a.) (J p.a.) Hubble telescope 2.8K 24.5M 88.3G Space shuttle launch 33T 1920P Altix 330K 2.9G 10.

5 Power: Energy used per unit of time Power Energy Energy (Watts) (Wh p.a.) (J p.a.) Hubble telescope 2.8K 24.5M 88.3G Space shuttle launch 33T 1920P Altix 330K 2.9G 10.4T Synchrotron 12M 105G 380T CERN LHC 120M 1T 3.8P Because the Data Centre is always on it consumes a lot of energy Conventional research uses energy too!

6 Where does energy go? Average of 12 Data Centres in NY&CA: Co-Location, Hosting, Government, Financial Institution, Telecom, Scientific Computing, Data Storage HVAC air movers 8% UPS 8% HVAC 23% Lighting 4% Other 11% Servers & Storage 46% Source: Lawrence Berkeley National Laboratory,

7 There s a Real Problem Energy consumption for US data centers and servers will nearly double by 2011 requiring an additional 10 power plants (EPA Study) Power consumption in data centers doubled from 2001 to 2005 (Lawrence Berkeley / Stanford study) Half the data centers will not have sufficient power for expansion by 2008 (Gartner survey 06) In the highest level of redundancy and reliability data centers, for every kilowatt used for processing, $22,000 is spent on power and cooling infrastructure (Uptime Institute) Through 2009, energy costs will emerge as the second highest operating cost in 70% of data centers worldwide (Gartner survey 06)

Concerns No Longer Focused on Servers Servers currently account for about 60% of data center power storage will soon take that place (Glass House 07) eweek.

8 Concerns No Longer Focused on Servers Servers currently account for about 60% of data center power storage will soon take that place (Glass House 07) eweek.com Average number of storage terabytes maintained by Fortune 1000 organizations in 2004: 138 Average number of storage terabytes maintained by Fortune 1000 organizations in October 2006: 600

9 Disk drive power dissipation Hitachi Deskstar 7K1000 Seagate Barracuda ES 750 WD Raptor WD1500ADFD idle seek Seagate Barracuda ES.2 WD Caviar SE16 WD7500AAKS WD Caviar GP WD10EACS Maxtor Atlas 15K II Seagate Cheetah 15K.4 Fujitsu MAU3147 Hitachi Ultrastar 15K147 Seagate Cheetah 15K Watts SATA drives use 8 10W SCSI drive use 12 20W SAS drives use ~1W more than SCSI FC drives use ~2W more than SCSI Source: StorageReview.com

10 Disk drive startup power consumption WD Caviar SE16 WD7500AAKS Seagate Barracuda ES750 WD Raptor WD 1500ADFD WD Caviar GP WD10EACS Seagate Barracuda ES.2 Hitachi Deskstar 7K1000 Maxtor Atlas 15K II 12V rail 5V rail Seagate Cheetah 15K.4 Seagate Cheetah 15K.5 Hitachi Ultrastar 15K147 Fujitsu MAU Watts SATA similar to SCSI Drives use ~1.5x (scsi) 2x (SATA) peak power to spin up Source: StorageReview.com

11 Energy required for 20TB of disk Capacity (GB) Speed (rpm) Type Power (Watts) Drives Energy p.a. (KWh) Joules (GJ) Seagate Savvio 15K SAS , Seagate Cheetah 15K SCSI , Hitachi 10K SCSI , Seagate Cheetah NS SAS , Seagate Barracuda ES SATA , Hitachi 7K SATA ,269 8 WD GP WD10EACS SATA , Seagate Barracuda PR SATA , Power based on 80% seeks 20% idle 1TB drives use ~10x less power per TB than 146GB SCSI drives 1TB drives use ~2x less power per TB than 400GB SCSI drives SCSI/SAS/FC has 2x IOPs and less latency New drives from Seagate and Western Digital use 50% less power

12 Tape drives SAIT / LTO4! 800GB! 45 / 120MB/s native! 19sec load time! 57sec file access time! 30W (active)! 5W (idle) TS1130! 1000GB! 160MB/s native! 13sec load time! 49sec data access! 46W (active)! 17W (idle) T10000B Capacity cart Sport cart! 1000GB 75GB! 120MB/s native! 16sec load time! 48sec data access 12sec! 63W (N/A)! No electricity to store data! Very high data transfer rates per device sequential access! Tape Libraries 10TB - 8PB+! 30 year media life Sources: Manufacturers data sheets; Clipper Group

Optical Storage Holographic (3D)! 300GB! 20MB/s data transfer! 5 sec load time! 50 year archive life! Blue laser! 10,000,000 reads! WORM! 80W Ultra Dense Optical (UDO)! 60GB (UDO2)!

9MB/s data transfer! 4.5sec media access time! Blue laser! Re-write 10,000 times! 140W (jukebox)! 20W (drive)! Like tape, optical media requires no electricity to store data!

13 Optical Storage Holographic (3D)! 300GB! 20MB/s data transfer! 5 sec load time! 50 year archive life! Blue laser! 10,000,000 reads! WORM! 80W Ultra Dense Optical (UDO)! 60GB (UDO2)! 12MB/s data transfer! 6.5sec media access time! 50 year archive life! Blue laser! WORM! 310W (19TB jbox)! 20W (drive) Blu-Ray / DVD BD-R, BD-RE, DVD-R, DVD-RW! 50GB / 5GB per disc! 9MB/s data transfer! 4.5sec media access time! Blue laser! Re-write 10,000 times! 140W (jukebox)! 20W (drive)! Like tape, optical media requires no electricity to store data! Data transfer rates are low compared to tape and disk! Largest jukebox capacity 30TB Sources:

14 Case Studies, tiered storage Small Medium Large 50TB 250TB 1PB

15 50TB Storage Solution Power Comparison Model Watts Type Drives Capacity (TB) SGI IS SAS SL T LTO Disk only 2.3x SGI IS4000 1,793 SATA 64 51! 8+2 RAID6! 400GB FC drives for HSM! 1TB SATA drives disk only vs

16 250TB Storage Solution Power Comparison Model Watts Type Drives Capacity (TB) SGI IS4000 1,655 FC STK SL LTO , Disk only 4.6x SGI IS6700 9,600 SATA ! 8+2 RAID6! 400GB FC drives for HSM! 1TB SATA drives disk only vs Source:

17 1PB Storage Solution Power Comparison Model Watts Type Drives Capacity (TB) SGI IS4600 4,092 FC SL T LTO4 12 1,154 5,058 1,195 Disk only 7x SGI IS ,800 SATA 1, ! 8+2 RAID6! 400GB FC drives for HSM! 1TB SATA drives disk only vs Source: Manufacturers data sheets

18 Results Summary Multi tier Disk only Greener Small 17W/TB 35W/TB 2.3x Medium 7.6W/TB 37.5W/TB 4.6x Large 4.4W/TB 37W/TB 7x! In each case multi tier RAIDs used SAS or FC disks to handle tape and network IO! A fundamental difference in service terms is the higher latency involved in retrieving data from disk

HSM should be first choice for long term storage Disk and tape have a place in the data center With HSM, a relatively small volume of high performance disk satisfy user requests with good bandwidth

19 HSM should be first choice for long term storage Disk and tape have a place in the data center With HSM, a relatively small volume of high performance disk satisfy user requests with good bandwidth and IOPS characteristics The bulk of data resides on tape when it s not active HSM provides the smarts to move data to the appropriate storage medium in response to usage It is easy being green WRT storage and capital investment is lower than a disk only solution

20 Customer Feedback SGI s Data Migration Facility (DMF) has been a key tool for storage management for CSIRO High Performance Computing since It gives users a productivity boost, by allowing them to access vast quantities of data seamlessly, without having to manage off-line media (CDs, DVDs, etc.) themselves. It allows users to have almost instant access to large amounts of scratch disc space. It provides the foundation for a data-intensive computing platform, by allowing users to easily work with large quantities of data. Lastly, it future-proofs their storage holdings, by removing the work of moving data through generations of storage media away from the users and onto the systems management, where it can be done in bulk automatically. CSIRO has situated its Data Store based on the DMF platform on four different sites and four different hosts since 1991, and has in each case completed the move between platforms and sites in a weekend. Users can still access their data holdings in the same way as they did in 1991, while additional services such as WWW and SRB access have been added. Dr. Robert C. Bell, Chief Technical Officer CSIRO High Performance Scientific Computing

21 Traditional Multi Tier Storage no HSM 2 nd disk tier user to reduce backup window Users have access to 10TB Full time operators reqd to administer backups and manually archive 10TB = 5 years of fulls + 12 months of fulls + 13 days of incrementals + special backups + archives 190TB online offline

22 Multi tier with HSM Users now have access to 180TB 1 part time operator to load new tapes, monitor backups and HSM exceptions Reduced TCO 180TB 10TB nearline offline

23 The backup problem Accesses to file 123.dat a 100GB file Acquire & Convert Analyse & Interpret Week5 Week dat is backed up every week even after it s no longer interesting because to do otherwise would require resources. 10.4TB of IO is performed to write 123.dat backups per year 1.3TB of media is used to store 123.dat and its copies most of which will never be used Retire Week3 full backup Week15 Week2 full backup Week1 full backup Day1 incremental backup Time Week20 Week25 Week30 Week35

24 The backup solution HSM Accesses to file 123.dat a 100GB file Acquire & Format Week5 Week10 Analyse & Interpret Backups are very much shorter because only the inode (512B) is required for recovery. 0.6TB of IO is performed to write the HSM copies of for 123.dat 0.3TB of media is used to store 123.dat and its copies 53KB of IO per year is performed to ensure 123.dat is recoverable a 94% reduction Retire Week1 full backup Week3 full backup Week2 full backup Day1; copy to tier2 Week15 Day 180; tier1 copy automatically removed Day 203; tier1 copy automatically reinstated from tier2 Day 233; tier1 copy automatically removed Day1 incremental backup Week20 Week25 Week30 Week35 Time

25 Without HSM, Backups Consume More Energy 30W per CPU core 2W per LAN switch port 20W per TB 120W per socket 10W per DIMM 2W per SAN switch port 2W per TB

SGI Contribution to Energy Initiatives tc99.ashraetcs.org/ www.thegreengrid.org/ www.climatesaverscomputing.

26 SGI Contribution to Energy Initiatives tc99.ashraetcs.org/

27 Green500 list Green500 Rank Site Computer Mflops per Watt Power (kw) TOP500 Rank 21 Total Exploration Production SGI Altix ICE 8200EX, Xeon quad core 3.0 GHz NASA/Ames Research Center/NAS SGI Altix ICE 8200EX, Xeon QC 3.0/2.66 GHz Grand Equipement National de Calcul Intensif - Centre Informatique SGI Altix ICE 8200EX, Xeon quad core 3.0 GHz HLRN at Universitaet Hannover / RRZN SGI Altix ICE 8200EX, Xeon quad core 3.0 GHz HLRN at ZIB/Konrad Zuse-Zentrum fuer Informationstechnik SGI Altix ICE 8200EX, Xeon quad core 3.0 GHz Irish Centre for High-End Computing SGI Altix ICE 8200EX, Xeon quad core 2.80 GHz University of Arizona SGI Altix ICE 8200, Xeon quad core 2.83 GHz SGI SGI Altix ICE 8200, 3.0 GHz New Mexico Computing Applications Center (NMCAC) SGI Altix ICE 8200, Xeon quad core 3.0 GHz Naval Research Laboratory (NRL) SGI Altix ICE 8200, 3.0 GHz Idaho National Laboratory SGI Altix ICE 8200, 2.66 GHz University of Minnesota SGI Altix XE 1300 Cluster Solutions, 2.66GHZ, Inf Roshydromet SGI Altix ICE 8200, Xeon quad core 2.83 GHz Japan Marine Science and Technology Altix GHz Wright-Patterson Air Force Base/DoD ASC Altix GHz Leibniz Rechenzentrum Altix GHz NASA/Ames Research Center/NAS SGI Altix 1.5/1.6/1.66 GHz, Voltaire Infiniband November 2008 List Source: green500_ xls 12 of greenest 100 are SGI systems SGI systems rank 55.4% better than list average (17 SGI systems average Mf/W vs Mf/W overall) The SGI ICE at Total is the 3rd most energy efficient x86- based machine among the world's 30 fastest supercomputers measured running Linpack

28 Building Water-Cooled Racks (4) Individual Coils Condensate Drain Pan Target Heat Rejection 95% water / 05% air Branch Feed to Individual Coil Chilled-Water Supply 45 F to 60 F (7.2 C to 15.6 C) 14.4 gpm (3.3 m 3 /hr) Max. 3/4 (1.91 cm) Coupling Swivel Coupling to Supply Hose

29 Building Award Winning Power Supplies SGI awarded 80plus silver and bronze certification for Altix systems shipping since January W Power Supply exceeds ClimateSaver specifications for 1 and 2U servers. Planned

Commodity components (mobile device processor) 20x memory bw of a single

30 Building More Efficient Future HPC Systems Kelvin liquid cooling system SGI Molecule concept computer 10x core density, ½ the power usage Commodity components (mobile device processor) 20x memory bw of a single rack x86 cluster Intel Atom N330 based; 8W TDP compare 120 W for XEON Harpertown

31 Delivering Green Consulting Services! Data Center Assessment! Dense Systems Assessment! Energy Efficiency Assessment! Power Distribution Assessment! Storage Assessment

32 Projected Electricity Use, All Scenarios 2007 to 2011 Source: U.S. EPA Report to Congress on Server and Data Center Energy Efficiency, Aug 07

The problem with disk 90 80 70 Enterprise storage capacity growth Enterprise HDD capacity growth (% Growth) 60 50 40 30 20 10 0 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

33 The problem with disk Enterprise storage capacity growth Enterprise HDD capacity growth (% Growth) ! Drive capacity growing at diminishing rate ~30%! Enterprise capacity increasing ~ constant 50%! Drive capacity growth exceeds IOPs growth! Forces tending to increase drive counts (and power consumption) Source: IDC 2008

34 Power dissipation per unit area Source: Uptime Institute Source: Uptime Institute

$DMF Saving the planet Automated tape libraries use a fraction of the power of RAID storage SATA disk is greener than SCIS/SAS/FC but isn t appropriate for all workloads It is better for the planet$

35 DMF Saving the planet Automated tape libraries use a fraction of the power of RAID storage SATA disk is greener than SCIS/SAS/FC but isn t appropriate for all workloads It is better for the planet for users to manage metadata better (w. MediaFlux) and wait a little longer for data It s cheaper to push everything off to tape than have someone look at it Centralised storage services improve data protection and data access to wider audiences The delete key is the greenest on the keyboard! Every Watt saved by ITS reduces green house gas emissions

36 Q & A Source: NZ Herald

37 Thank You! 2008 Silicon Graphics, Inc. All rights reserved. Silicon Graphics, SGI, SGI Altix, the SGI logo and the SGI cube are registered trademarks and Innovation for Results is a trademark of Silicon Graphics, Inc., in the United States and/or other countries worldwide. All other trademarks mentioned herein are the property of their respective owners. (06/08)

How green is my HSM? David Honey SGI ANZ

How green is my HSM? David Honey SGI ANZ Agenda Where does energy come from and go to? Mass storage device power use; disk, SSD, tape, optical Multi tier vs disk only HSM is green What SGI is doing to