File System and Storage Benchmarking Workshop SPECsfs Benchmark The first 10 years and beyond

Transcription

1 1 File System and Storage Benchmarking Workshop SPECsfs Benchmark The first 10 years and beyond Sorin Faibish, EMC

2 2 NFS Chronology March 1984: SUN released NFS protocol version 1 used only for inhouse experimental purposes. March 1989: SUN and IBM partner on version 2 of the protocol operated entirely over UDP (RFC 1094) Stateless protocol Locking implemented outside of the core protocol June 1995: NFSv3 is introduced with TCP support (RFC 1813) and: 64-bit file sizes and offsets asynchronous writes on the server additional file attributes in many replies READDIRPLUS operation December 2000: NFS v4.0 became the first version developed with the Internet Engineering Task Force (IETF) (RFC 3010) April 2003: NFS v4.0 revised (RFC 3530) September 2006: NFS version 4.1 with pnfs support

3 3 SPECsfs Chronology October 1992: LADDIS the first NFS servers benchmark is introduced by the constituent companies: Legato; Auspex; Data General; Digital Equipment; Interphase; SUN Microsystems April 1993: SPEC introduces SPECsfs93 one test benchmark 097.LADDIS based on nhfsstone workload December 1997: SPEC introduced its new SFS (System File Server) benchmark: sfs97 June 2001: SPEC replaces SFS97 with SFS97 V2.0 fixing bugs September 2001: SPEC releases SFS97_R1 V3.0 with additional support for Linux and FreeBSD March 2008: SPEC releases SPECsfs2008 benchmark for CIFS and NFSv3

4 4 LADDIS Benchmark LADDIS is a synthetic benchmark used to measure the NFS request response time and throughput capacity of an NFS file server LADDIS NFS file server benchmark has been developed to resolve the popular nhfsstone's shortcomings LADDIS is executed concurrently on one or more NFS clients NFS client systems, call LADDIS load generators and send a controlled stream of NFS requests to the server, according to a specific operation mix and file access distribution LADDIS benchmark measures the server as if it were a black box LADDIS becomes an industry-standard NFS file server benchmark Paper: or wittle.txt

5 5 What Makes SPECsfs A Good Benchmark? Measure both response time and bandwidth Measure guaranteed performance; request response Benchmark both disk and server subsystems performance Workload characteristics relevant for multiple types of applications: DB, , SQL, ClearCase, Code build, File Servers NFS operation mix consists of both MD and data operations Can saturate all server resources: CPU, disk, Pipes (FC, IP), BUS and Memory Benchmark both backend (disk) and front end (network) Has flexibility to allow modifications to operation mix as well as different I/O sizes and read/write mix

6 6 Reading SPECsfs97_R1 Publications SPECsfs97_R1.v3 = Ops/Sec (Overall Response Time = 2.23 msec) IOPS ORT NFSv3 - TCP Throughput Response (ops/sec) (msec) ORT = Av(RT) Equally Spaced Requests: 3000 Requested Achieved (ops/sec) (ops/sec) IOPS

7 SPECsfs97_R1 Benchmark Characteristics NFSv3 Distribution Tables Selection based on Real Applications Characteristic (not much real customer data as NFS was too young) NFS Operation Mix File Size I/O Sizes: Read Write ---- SFS 3.0 NFS OP mix size distribution table ---- /* name mix function */ /* pcnt */ { "getattr", 11, op_getattr }, { "setattr", 1, op_setattr }, { "lookup", 27, op_lookup }, { "readlink", 7, op_readlink }, { "read", 18, op_read }, { "write", 9, op_write }, { "create", 1, op_create }, { "remove", 1, op_remove }, { "readdir", 2, op_readdir }, { "fsstat", 1, op_fsstat }, { "access", 7, op_access }, { "commit", 5, op_commit }, { "fsinfo", 1, op_fsinfo }, { "readdirplus", 9, op_readdirplus }, { "TOTAL", 100, 0 }} ---- SFS 3.0 file size distribution table ---- sfs_io_file_size_dist Default_file_size_dist[] = { /* % KB size */ { 33, 1}, /* 33% */ { 54, 2}, /* 21% */ { 67, 4}, /* 13% */ { 77, 8}, /* 10% */ { 85, 16}, /* 8% */ { 90, 32}, /* 5% */ { 94, 64}, /* 4% */ { 97, 128}, /* 3% */ { 99, 256}, /* 2% */ { 100, 1024}, /* 1% */ { 0, 0} ---- SFS 3.0 read size distribution table ---- static sfs_io_op_dist_type Default_read_size_dist[] = { /* percentile 8KB xfers fragments */ { 0, 0, 0 }, /* 0% 1-7 KB */ { 85, 1, 0 }, /* 85% 8 KB */ { 93, 2, 1 }, /* 8% KB */ { 97, 4, 1 }, /* 4% KB */ { 99, 8, 1 }, /* 2% KB */ {100, 16, 1 }, /* 1% KB */ } ---- SFS 3.0 write size distribution table ---- static sfs_io_op_dist_type Default_write_size_dist[] = { /* percentile 8KB xfers fragments */ { 49, 0, 1 }, /* 49% 1-7 KB */ { 85, 1, 1 }, /* 36% 9-15 KB */ { 93, 2, 1 }, /* 8% KB */ { 97, 4, 1 }, /* 4% KB */ { 99, 8, 1 }, /* 2% KB */ { 100, 16, 1 }, /* 1% KB */ 7

8 8 Examples from Published Data SPECsfs97_R1.v2 =20113 Ops/Sec (Overall Response Time = 3.04 msec) NFSv2 - UDP Throughput Response (ops/sec) (msec) SPECsfs97_R1.v2 =16063 Ops/Sec (Overall Response Time = 3.62 msec) NFSv2 - TCP Throughput Response (ops/sec) (msec) SPECsfs97_R1.v3 =16791 Ops/Sec (Overall Response Time = 2.07 msec) NFSv3 - UDP Throughput Response (ops/sec) (msec) SPECsfs97_R1.v3 =14848 Ops/Sec (Overall Response Time = 2.22 msec) NFSv2 - TCP Throughput Response (ops/sec) (msec)

9 Additional Examples 9

10 SPECsfs97_R1 Menagerie 10

11 11 SPECsfs97_R1 Extremes Lowest ORT = 0.64 msec Highest ORT = 4.46 msec Lowest IOPS = 1957 Ops/Sec Highest IOPS = Ops/Sec

12 10 Years After Introducing New SPECsfs2008 Benchmark Characteristics Based on Data Collected from Real Customers (NFS is widely used in industry) Includes both NFS and CIFS components Common components between CIFS and NFS SFS manager and reporting Directory layouts (unchanged from SPECsfs97_R1) Changed from sfs97: Total data set size, Working set size, File size distribution NFS New I/O size distribution New operation mix Concept of logical read / write operations No UDP or NSFv2 support Usage of NFSv2 and UDP has waned substantially in the customer base CIFS Totally new benchmark Based on Hidden Markov Model (HMM) HMM model driven by traces Op mix derived from data from many real customers. Network traces Call home data 12

13 13