Using SPEC SFS with the SNIA Emerald Program for EPA Energy Star Data Center Storage Program Vernon Miller IBM Nick Principe Dell EMC

Size: px

Start display at page:

Download "Using SPEC SFS with the SNIA Emerald Program for EPA Energy Star Data Center Storage Program Vernon Miller IBM Nick Principe Dell EMC"

Donna Pope
6 years ago
Views:

1 Using SPEC SFS with the SNIA Emeald Pogam fo EPA Enegy Sta Data Cente Stoage Pogam Venon Mille IBM Nick Pincipe Dell EMC v6

2 Agenda Backgound on SNIA Emeald/Enegy Sta fo block Intoduce NAS/File test addition; intoduce SFS 2014 Testbed configuation and measuement points Test pocedue A look at some eal data and the deivation of the metics 2

3 Geen Peamble Inceased egulatoy and societal pessues to lowe enegy footpints Gowing awaeness of envionmental impact of IT equipment Rising enegy cost fo powe and cooling is a lage pat of the cost of owneship Data centes cannot eadily add additional powe o cooling capacity Oveview: Geen Stoage, Enegy Sta and SNIA Emeald Pogam 3

4 EPA ENERGY STAR fo Data Cente Stoage Enegy Sta cetification pogam fo stoage systems. EPA patneed with SNIA s Geen Stoage Initiative (GSI) to develop technical specification and equiements. EPA Data Cente Stoage website 4

5 SNIA Geen Activities Geen Stoage Initiative (GSI) Maket geen stoage and manage the Emeald Pogam Reseach, educate, leveage SNIA esouces, povide diection Geen Technical Woking Goup (GTWG) Technical body of stoage expets developing geen stoage specifications, white papes, tutoials, technical guidance Develop the SNIA Emeald Powe Efficiency Measuement Specification (cuently 2.1.1) and how to Use Guide fo it Emeald Pogam Pomote use of the SNIA Emeald Specification methodology and test esults Help dive geen stoage decisions fo both vendos and custome Oveview: Geen Stoage, Enegy Sta and SNIA Emeald Pogam 5

6 SNIA Geen Stoage Initiative (GSI) Establish and maintain the SNIA Emeald Pogam fo SNIA Emeald Enegy Efficiency Measuement and conduct taining of SNIA Emeald testes and industy stakeholdes Educate the IT industy, vendo community and egulatoy bodies on techniques to conseve enegy fo entepise stoage envionments Povide extenal advocacy and suppot of the technical wok of the SNIA Geen Stoage Technical Woking Goup (TWG) Povide input to the SNIA Geen Stoage TWG on equiements fo geen stoage measuement specifications, metics and standads Establish and maintain coss-industy elationships and alliances to coodinate and advance data cente enegy efficiency elated pogams, test and measuement methods, and standads 6

7 SNIA Geen Technical Woking Goup Technical body woking on geen stoage metics and standads Gets diection fom GSI Wites the SNIA Emeald Powe Efficiency Measuement Specification and elated documents Suppots the Emeald Pogam White papes Tutoials Taining Woks with egulatoy agencies (i.e. EPA) on geen stoage specifications 7

8 SNIA Emeald Pogam Oveview The pupose is to povide public access to stoage system powe usage and efficiency though use of a well-defined testing pocedue, and additional infomation elated to system powe. Povides a standadized way of epoting vendopefomed test esults that chaacteize the seveal aspects of stoage system enegy usage and efficiency. Powe Efficiency Measuement Specification Taxonomy Measuement Metics 8

9 Emeald Powe Efficiency Measuement Specification Maket Taxonomy Simplifies compaisons and egulatoy effots Measuement SUT configuation equiements Block level I/O (Vdbench, COMgen) Active state, idle state, hot bands Powe/envionmental measuements Metics Pimay metics atios of pefomance pe watt Random access (Tansactional) of the data pe unit of powe Sequential access (Steaming) of the data pe unit of powe Stoage Capacity pe unit of powe Seconday metics Capacity Optimization veification, i.e. existence test 9

10 Emeald Powe Efficiency Measuement Specification Maket Taxonomy Classifies stoage systems in tems of opeational pofile and suppoted featues Simplifies compaisons and egulatoy effots 10

Emeald Powe Efficiency Measuement Specification I/O and Measuement Standad input voltages and datacente conditions equied Pefill Test SUT

11 Emeald Powe Efficiency Measuement Specification I/O and Measuement Standad input voltages and datacente conditions equied Pefill Test SUT Conditioning Test Active Test (Vdbench) Hot Band Random Wite Random Read Sequential Wite Sequential Read Ready Idle Test Capacity Optimization Test (comgen) 11

12 Emeald Powe Efficiency Measuement Specification Powe Metics Pimay metics atios of pefomance / watt Random access (Tansactional) of the data pe unit of powe Input Output pe Second pe Watt (IOPS/W) Sequential access (Steaming) of the data pe unit of powe Mebibyte pe Second pe Watt (MiBPS/W) Stoage Capacity pe unit of powe Gigabyte pe Watt (GB/W) Seconday metics Capacity Optimization veification, i.e. existence test Six techniques that educe the numbe of stoage devices to stoe the same amount of data thus educing the powe equied to stoe the data Oveview SNIA Emeald Measuement 12

13 SNIA Emeald Test Data Submission SNIA Emeald Pogam Recod esults in SNIA Emeald Test Data Repot (TDR) Complete Test Submission Ode Fom EPA ENERGY STAR EPA ecognized lab must pefom tests Obtain cetification fom an EPA ecognized Cetification Body 13

14 Disclaime The SNIA Emeald specification with file-access suppot, as epesented in this pesentation, is pe-elease; the benchmak famewok, wokloads, and esults and epoting stuctue ae still unde intenal SPEC and SNIA eview and may change befoe final elease of SNIA Emeald Specification vesion

15 NAS/File Addition to Specification Vesion 3 of SNIA Emeald Powe Efficiency Measuement Specification Addesses both block and file access New wokloads and toolkit fo file access testing SPEC SFS 2014 New methodology fo detemining powe metics fo file access Expected ollout stating 1H17 15

16 SPEC Standad Pefomance Evaluation Copoation The Standad Pefomance Evaluation Copoation (SPEC) is a non-pofit copoation fomed to establish, maintain and endose a standadized set of elevant benchmaks that can be applied to the newest geneation of high-pefomance computes. SPEC develops benchmak suites and also eviews and publishes submitted esults fom membe oganizations and othe benchmak licensees SPEC and SPEC SFS ae egisteed tademaks of the Standad Pefomance Evaluation Copoation. Additional poduct and sevice names mentioned heein may be the tademaks of thei espective ownes. 16

17 Why SPEC SFS 2014? SPEC SFS 2014 is a Stoage Solution Benchmak Realistic, Solution-based, Industy-standad wokloads DATABASE, SWBUILD, VDA, VDI Wokloads based on taces, like pevious SFS 2008 Moden scenaios based on standad solutions Advanced measuement quality of sevice Ops and latency don t tell the whole stoy business metics Ability to measue boad ange of poducts and configuations Taditional (HDD), Hybid, All-Flash Key easons SNIA Emeald is using SFS 2014 Vendos likely aleady unning SFS 2014 in-house Wokloads aleady ageed upon by multiple vendos Robust wokload geneato fo file access 17

The SPEC SFS 2014 Wokloads Fo moe details, see: - SDC 2014 pesentation: SPEC SFS 2014: An Unde-the-Hood Review - The SPEC SFS 2014 website http://www.spec.

18 The SPEC SFS 2014 Wokloads Fo moe details, see: - SDC 2014 pesentation: SPEC SFS 2014: An Unde-the-Hood Review - The SPEC SFS 2014 website DATABASE Simulates OLTP database consolidation Measued in # of concuent DATABASES SWBUILD Simulates lage softwae poject compilation Measued in # of concuent BUILDS VDA Simulates acquisition of steaming data Measued in # of concuent STREAMS VDI Simulates heavy steady-state VDI wokload Measued in # of concuent DESKTOPS 18

File vs Block Configuations Block Access ESXi Seve 1

Win VM Win VM IO Load Dive Systems Block I/O geneato:

caching 8 Gb FC Switch NAS Seve SUT Minimal impact to

19 File vs Block Configuations Block Access ESXi Seve 1 ESXi Seve 2 ESXi Seve 10 Win VM Win VM Win VM Win VM Win VM Win VM IO Load Dive Systems Block I/O geneato: vdbench 10 GbE Switch Custom wokloads No client side caching 8 Gb FC Switch NAS Seve SUT Minimal impact to powe metics due to load geneatos Powe Mete Stoage Contolle 19

20 File vs Block Configuations File Access ESXi Seve 1 ESXi Seve 2 ESXi Seve 10 Win VM Win VM Win VM Win VM Win VM Win VM IO Load Dive Systems File I/O geneato: SPEC SFS official wokloads sflow Collection 8 Gb FC Switch 10 GbE Switch NAS Seve SUT Client side caching Load geneatos have geate effect on oveall pefomance. sflow Powe Mete Stoage Contolle Industy standad fo collecting laye 2 netwok countes Collect data ate 20

21 File vs Block Test Sequences Block Access Pe-fill test, puts data in SUT SUT conditioning Active test Ready idle test Capacity optimization test (if defined) File Access Calibation of SFS benchmaks Execution of the 4 SPEC SFS 2014 wokloads, in sequence Ready idle test Capacity optimization test (if defined) 21

22 File vs Block Pimay Metics Block Access Powe efficiency fo active phase Hot Band Random Read Random Wite Sequential Read Sequential Wite Powe efficiency fo eady idle test phase File Access Powe efficiency fo each wokload (MiB/sec/Watt) VDA DATABASE VDI SWBUILD Powe efficiency composite metic Powe efficiency fo Ready Idle test phase 22

23 File Access Measuement Points The SPEC SFS 2014 metics ae only used to calibate the appopiate load points fo each wokload The Emeald efficiency metic (MiB/s/W) is deived fom the data collected by the sflow collecto in font of the SUT SPEC SFS 2014 measues at the applicationlevel SNIA Emeald measues at the system-level 23

24 Additional Hadwae/Softwae Requiements fo File Access In addition to the envionmental and powe metes common fo both Block and File Access Emeald testing, File Access testing equies: A netwok switch that suppots sflow An sflow collecto that can log fo extended peiods of time 24

25 File Access Test Pocedue The fou basic phases of file access testing Calibation Measuement Data Reduction SNIA Emeald Metics 25

26 File Access Test Pocedue Calibation Fo each SPEC SFS 2014 wokload, find maximum SUT pefomance Known fom existing pefomance testing Many vendos un SFS 2014 fo egession analysis Test to detemine as pat of Emeald test pocess Run seveal SFS 2014 uns, adjusting load points to pobe fo the maximum valid load point 26

File Access Test Pocedue Calibation Example On a new test system, setup SWBUILD to un fom 1 to 30 load points, incementing the load by 1 each step Afte 28 load points, achieved ops/sec stopped

27 File Access Test Pocedue Calibation Example On a new test system, setup SWBUILD to un fom 1 to 30 load points, incementing the load by 1 each step Afte 28 load points, achieved ops/sec stopped scaling and SFS 2014 was epoting INVALID_RUN At this point, the benchmak was manually teminated 12,000 10,000 Maximum valid load point INVALID_RUN = LOAD too high Achieved Ops/sec 8,000 6,000 4,000 2, Concuent BUILDS (SWBUILD) 27

28 File Access Test Pocedue Measuement Using maximum valid load fom calibation data Run each wokload Ten evenly-spaced load points up to the maximum valid load point Collect envionmental, powe, and sflow data fo each un Ready-idle test Collect envionmental and powe data while aay is idle 28

File Access Test Pocedue Measuement Example Fom the calibation phase, we know the system s maximum valid load point is 20 BUILDS Theefoe, fo measuement, we an fom 2 though 20 BUILDS, incementing by 2

29 File Access Test Pocedue Measuement Example Fom the calibation phase, we know the system s maximum valid load point is 20 BUILDS Theefoe, fo measuement, we an fom 2 though 20 BUILDS, incementing by 2 BUILDS 12,000 10,000 Maximum valid load point INVALID_RUN = LOAD too high Achieved Ops/sec 8,000 6,000 4,000 Calib Meas 2, Concuent BUILDS (SWBUILD) 29

30 File Access Test Pocedue Data Reduction Using collected sflow and Powe data Fo each wokload, fo the measuement phase only at each load point, calculate Aveage powe Aveage netwok thoughput Using the aveage powe and netwok thoughput Calculate efficiency metic at each load point fo each wokload MiB/s/W This pocess is known as data eduction Time-based data fom multiple souces is educed to calculated metics pe-load point, pe-wokload An open-souce tool is expected to be available to assist with this pocess 30

31 File Access Test Pocedue Data Reduction Example The aw powe data log will include data fo the whole duation of benchmak execution Only want data fom the measuement phase 31

32 File Access Test Pocedue Data Reduction Example Using data eduction techniques, compute aveage powe usage duing measuement phase Fo each load point Aveage Watts SWBUILD Iteation Numbe The same pocess is used fo the sflow data Aveage netwok thoughput duing measuement phase 32

33 File Access Test Pocedue SNIA Emeald Metics Find the sweet-spot fo all fou wokloads Fo each wokload, find the highest efficiency metic, using data fom the data eduction step Usually, but not necessaily, the highest valid load point Compute combined metic based on sweetspot metics Expected to be computed as the aveage of the highest efficiency metic fo each wokload 33

34 File Access Test Pocedue SNIA Emeald Metics Example Example data fom anothe system In a eal Emeald un, thee would be ten load points fo each wokload and eady-idle data EFFICIENCY METRICS Load Point DATABASE SWBUILD VDA VDI This system demonstates highest efficiency at the highest valid load point Maximum Efficiency (MiB/s/W) COMBINED METRIC (PROPOSED) 0.12 MiB/s/W 34

35 Key Takeaways The SNIA Emeald pogam is adding suppot fo file access stoage systems SPEC SFS 2014 and its wokloads ae used to evaluate file access stoage systems The SUT fo SNIA Emeald file access testing only includes the stoage aay Efficiency metics ae deived fom: Netwok taffic to and fom the stoage aay Not SPEC SFS 2014 metics Powe consumption by the stoage aay 35

36 Q&A Thank you fo attending! Please emembe to submit feedback on ou session! 36

Multidimensional Testing

Multidimensional Testing QA appoach fo Stoage netwoking Yohay Lasi Visuality Systems 1 Intoduction Who I am Yohay Lasi, QA Manage at Visuality Systems Visuality Systems the leading commecial povide of