Managing Array of SSDs When the Storage Device is No Longer the Performance Bottleneck

Transcription

1 Managing Array of Ds When the torage Device is No Longer the Performance Bottleneck Byung. Kim, Jaeho Kim, am H. Noh UNIT (Ulsan National Institute of cience & Technology)

2 Outline Motivation & Observation Our Idea Provide full network performance Eliminate inconsistent performance Evaluation Full network bandwidth Consistent performance ummary & Future work

3 Outline Motivation & Observation Our Idea Provide full network performance Eliminate inconsistent performance Evaluation Full network bandwidth Consistent performance ummary & Future work

4 D Garbage Collection: Degraded Performance Lies, Damn Lies And D Benchmark Test Result, sights/lies-damn-lies-and-ssd-benchmark-master-ti/.

5 IOP D Garbage Collection: Inconsistent Performance udden drop in IOP Time Intel olid-tate Drive DC 3700 eries Quality of ervice,

6 Rise of All-Flash Array Google Datacenter Improving performance for application request

7 Architecture of All-Flash Array

8 Architecture of All-Flash Array

9 Interface Bandwidth Growth Trend

10 Interface Bandwidth Growth Trend Network becomes bottleneck

11 Commercial D Trend D product Read performance Write performance Product A 6.8GB/s 4.8GB/s Product B 3.5GB/s 2GB/s Product C 2.7GB/s 1.5GB/s 10Gbps Ethernet: 1.25GB/s Fibre channel: 1GB/s

12 Commercial All-Flash Array Trend All-Flash Array products # of Ds # of network ports Product A 10 Up to 4 Product B 150 Up to 48 Product C 68 Up to 2 Product D 96 Up to 4 Up to 34 Ds per network port

13 Commercial All-Flash Array Trend All-Flash Array products # of Ds # of network ports Product A 10 Up to 4 Product B 150 Up to 48 Product C 68 Up to 2 Product D 96 Up to 4 Up to 34 Ds per network port Does these many Ds really help?

14 Performance of RAID 0 with 4 Ds RAID 0 config. by 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) Ideal performance equential write with 128KB I/O size

15 RAID 0: Inconsistent Performance RAID 0 config. by 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) Ideal performance (Measured read: 2000MB/s, write 1000MB/s) Inconsistent performance equential write with 128KB I/O size

16 Performance of RAID 0 with 4 Ds RAID 0 config. by 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) Ideal performance (Measured read: 2000MB/s, write 1000MB/s) 10GbE (1.25GB/s) Does not saturate network bandwidth equential write with 128KB I/O size

17 Limited by Network Bandwidth Performance is limited by network bandwidth Another motivation: provide full network performance under network connection

18 Limited by Network Bandwidth Performance is limited by network bandwidth Another motivation: provide full network performance under network connection ustained full network performance

19 Problem and Our Goal All-Flash Array suffers from inconsistent, limited performance We want consistent, full network performance!

20 Outline Motivation & Observation Our Idea Provide full network performance Eliminate inconsistent performance Evaluation Full network bandwidth Consistent performance ummary & Future work

21 Our olution erial configuration Front-end D Back-end Ds

22 erial Configuration Design Provide full network performance Absorb all writes with Front-end D Log-structured manner Provide consistent performance Eliminate GC within Front-end D Propose xgc

23 Full Network Performance Absorb all writes with Front-end D Network bandwidth < Front-end D bandwidth Log-structured manner equential, append only writes

24 Issues in Providing High Performance Front-end D will eventually fill up Garbage Collection? Managing Front-end Ds electing next Front-end Making space available

25 Handling Garbage Collection Eventually, the Front-end D becomes full and garbage collection is needed External GC (xgc) Garbage collection never occurs at Front-end D Front-End (Write ptr) D 1 D 2 D 3

26 Handling Garbage Collection When the Front-end D fills up New Front-end is selected Old Front-end D becomes a Back-end D External GC (xgc) is performed between Back-end Ds Front-End (Write ptr) D 1 D 2 D 3 xgc

27 Handling Garbage Collection When all valid data is moved Old front-end is cleaned by issuing TRIM command Trim Front-End (Write ptr) D 1 D 2 D 3

28 Effect of xgc Front-end performance is not affected by GC Front-end always provides consistent performance

29 Outline Motivation & Observation Our Idea Provide full network performance Eliminate inconsistent performance Evaluation Full network bandwidth Consistent performance ummary & Future work

30 Evaluation ettings Description torage erver Host erver CPU Intel Xeon E Intel i5-6600k RAM 64GB DRAM 16GB DRAM Ethernet 10Gbps O Linux kernel Linux kernel D Intel GB NVMe D 4 (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s)

31 Evaluation ettings Observe effect of network connection Host FTP Application 10GbE I/O request D torage server Managing technique D D D Observe effect of serial configuration GC Incurring Application torage server I/O request D Managing technique D D D

32 Effect of Network Bandwidth Transfer 10 files (respectively, 10GB) with 10 threads to storage server via FTP protocol Measurement point is the storage server Host FTP Application 10GbE I/O request D torage server Managing technique D D D

33 Effect of Network Bandwidth 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) Ideal performance of RAID 0 Typical measured performance

34 Effect of Network Bandwidth 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) 10Gb Ethernet(1.25GB/s)

35 Effect of Network Bandwidth 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) Avg. 924MB/s (4 Ds) 10Gb Ethernet(1.25GB/s) Avg. 895MB/s (Front-end, 1 D)

36 Conclusion of First Evaluation Performance is determined by network independent of performance of storage Performance of our approach is similar to that of RAID 0 with 4 Ds

37 Observe Effect of erial Configuration Performance with network effect removed Verify performance of serial configuration ynthetic workload generated by FIO benchmark tool Perform I/O for 30 minutes after aging 1200GB footprint 256KB random writes Measure performance of random write workload with 64KB I/O size torage server GC Incurring Application I/O request D Managing technique D D D

38 Verifying Consistent Performance 10Gb Ethernet(1.25GB/s) 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) Typical measured performance Aging workload

39 In Contrast to RAID 0 Configuration Ideal performance 10GbE (1.25GB/s)

40 Verifying Consistent Performance 10Gb Ethernet(1.25GB/s) 4 NVMe Ds (spec. read: 2400MB/s, write: 1200MB/s) (Measured read: 2000MB/s, write 1000MB/s) Typical measured performance Aging workload Observe consistent performance Observe no internal GC effect Observe no computing overhead for xgc

41 Outline Motivation & Observation Our Idea Provide full network performance Eliminate inconsistent performance Evaluation Full network bandwidth Consistent performance ummary & Future work

42 ummary All-Flash Array is faster but limited by network bandwidth All-Flash Array suffers from inconsistent performance due to garbage collection Proposed technique that satisfies both full network performance and consistent performance

43 Future Work Fault-tolerance Parallelization of serial configuration One serial configuration per network port calability More than 4 Ds Heterogeneous Ds Metadata management Latency performance Effect on read performance

44 Thank you