A Case for IO Determinism for Hyperscale Applications Utilizing QLC Flash Memory

Transcription

1 A Case for IO Determinism for Hyperscale Applications Utilizing QLC Flash Memory Steven Wells Data Center Architecture Fellow Jim Ulery Distinguished SSD Engineer Toshiba Memory America, Inc. August

2 Don t worry, we process this in the background August

3 Introduction Last year we introduced hyperscaler challenges with read tail latencies and offered a solution using IO Isolation This year, this presentation will expand the concept with what we are coining Hyperscale Mean Latency Technologies such as QLC intrinsically have higher latency and this presentation will demonstrate how Hyperscale Mean Latency is best mitigated with IO Isolation We demonstrate using NVMe IO Determinism solution how to mitigate internal operations such as garbage collection and background data refresh. August

4 From Last Year Read latency tails Read Latency Mixed Read/Write Workload (Simulation, us) % 99% Program Collisions 99.9% 99.99% % Erase Collisions % Simulation data: As with any test, the results and outcomes herein should not be interpreted as a guarantee August 2017 or warranty of similar results. Results may vary, depending on the circumstances and conditions. 4

5 From last year: NVM set isolation concept die die channel channel Classic SSD architecture uses bands of devices on every channel to maximize bandwidth. Maintenance is also on every die on every channel New SSD array architecture creates independent NVM Sets 5

6 From last year s POC Set Isolation Result QD1 4K Random Read Latency vs. Write Disturbances 90% % 99.9% 99.99% % ~>50x Tail Latency Improvement! % August % QD1 4KRR Latency (usec) No Sets (4KRW) Horizontal Sets Vertical Sets Lab data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of similar results. Results may vary, depending on the circumstances and conditions. Final FW Optimizations 6

7 Further justification for IO Isolation in hyperscale environments New Concept: Hyperscale Mean Latency (HML) August

8 From last year In practice, a single user request may result in thousands of subqueries, with a critical path that is dozens of subqueries long. The fork/join structure of subqueries causes latency outliers to have a disproportionate effect on total latency, and the large number of subqueries would cause slowdowns or unavailability to quickly propagate Challenges to Adopting Stronger Consistency at Scale - Ajoux et. Al., (Facebook & USC), 2015 August 2017 Topology: Thousands=Hundreds x Dozens 8 5/hotos15-paper-ajoux.pdf

9 Hyperscale Fork/Join Query Topology M parallel drive reads per Fork/Join Results Fork/Join J-latency determined by worst case latency Query F 4 M J Result Drive Reads August

10 Effects of Content Updates and Internal Refresh on Fork/Join Latencies M F J While the host is doing time critical fork/join queries, each drive element is subject to Host initiated writes to update content. Drive initiated garbage collection and internal refresh Content Updates Host Reads Internal Refresh and Garbage Collection August

11 Hyperscale Mean Latency Content Updates Host Reads Attempting to do fork/join queries in an environment with both content updates (writes) along with internal garbage collection and refresh amplify the mean latencies as seen from the perspective of the hyperscaler Single Drive vs Fork-Join (200x1) Latency Distribution (us) ,000 0% 90% 99% 99.9% 99.99% % % Single Drive At scale a singe drive tail latency turns into a hyperscale mean latency Topology: 200x1 drives Simulation data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of August 2018 similar results. Results may vary, depending on the circumstances and conditions. 11

12 Cascade Fork/Join Query Topology Cascade of N Fork/Joins M parallel drive reads per Fork/Join Fork/Join read latency determined by Tall Pole Cascade latency is sum of Tall Poles For the rest of this paper we ll assume MxN=200x24 as example Query M August N Result Content Read Access Flow F J

13 Tail Latencies: Real System Impact! N Even 1% write level impacts hyperscale mean read latency 4x! A classical ~70/30 write profile can impact mean read latencies by 10x Best system latency is when read set is quiet except host reads Solution: IO Isolation 200x24 fork/join Read Latency Distribution Against Various Write Rates (us) , ,000 0% 90% 99% 99.9% 99.99% % % ~4x mean latency for 1% writes ~10x mean latency for 30% writes Simulation data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of August 2018 similar results. Results may vary, depending on the circumstances and conditions. 13

14 Background Data Refresh (BDR) BDR continuously reads mapped content. Creates read-on-read collisions. Relocates weak content. Creates read-on-write/erase collisions. Data shows limited mean impact at a single drive level. This is what justifies an SSD designer to think its OK to call it Background Data Refresh. But 0% 90% 99% 99.9% 99.99% % % Single Drive QD2 Read Latency (us) ,000 Per-Drive (no relocations) Lab Data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of similar August 2018 results. Results may vary, depending on the circumstances and conditions. 14

15 BDR Impact at Hyperscale Level Cannot be Ignored M F M=200 0% 90% 200x1 Topology Fork/Join QD2 Read Latency (us) ,000 J 99% 99.9% 99.99% % % August 2018 Per-Drive Per-Fork/Join Lab and Simulation Data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of similar results. Results may vary, depending on the circumstances and conditions. 15

16 IOD BDR Recommendation Suspend BDR scan during DTWIN. Requires accelerated BDR scan rate during NDWIN intervals to meet coverage targets August

17 What about QLC and IOD? Assumptions (QLC vs. TLC): Bigger blocks Reads 2x-3x slower Programs 4x-5x slower Erases and suspends about the same Predictions herein are for informational purposes only and should not be interpreted as a guarantee or August 2018 warranty of similar results. Results may vary, depending on the circumstances and conditions. 17

18 0% TLC vs. QLC Per-Drive Read Latencies TLC vs. QLC Read Latency (us) Mixed Workloads ,000 90% 99% 99.9% 99.99% Significantly higher QLC program traffic increases suspend probabilities (~5x) % % Increased QLC program time, bigger blocks mask erases. Simulation Data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of August 2018 similar results. Results may vary, depending on the circumstances and conditions. 18

19 1 TLC vs QLC in a Hyperscale Environment 2 3 TLC vs. QLC 'Full-Tree 200x24 Fork/Join Read Latency vs. Background Write Level (us) , ,000 0% N 90% 2x-3x 99% 99.9% 99.99% % 200x24 hyperscaler topology 0% write mean latency increases by the increase in QLC tread % August 2018 Simulation Data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of similar results. Results may vary, depending on the circumstances and conditions. 19

20 1 TLC vs QLC in a Hyperscale Environment 2 3 TLC vs. QLC 'Full-Tree' Fork/Join Read Latency vs. Background Write Level (us) , ,000 0% N TLC with a non isolated ~70/30 workload increases 200x24 mean latencies by ~6x vs IO Isolation TLC: ~6x 99.99% % % QLC: ~18x Where as QLC without isolation with a similar workload extends the mean latency by 18x vs utilizing isolation August 2018 Simulation Data: As with any test, the results and outcomes herein should not be interpreted as a guarantee or warranty of similar results. Results may vary, depending on the circumstances and conditions. 20

21 Summary Last year we demonstrated array isolation offering ~50x latency tail improvements The concept of Hyperscale Mean Latency (HML) is explored where low probability drive read tail latencies turn into mean latency impacts for hyperscalers given the breadth and depth of fork/join operations. Applying HML concepts to a TLC SSD tells us Even 1% write rates without IOD impacts HML by 4x A classical 70/30 workload without can impact HML by ~10x NVMe IOD is an idea solution to address HML Background data refresh can meaningful impact HML and is recommended to utilize determinism modes of NVMe to mitigate QLC s longer program latencies can induce further HML latencies and values IO Determinism concepts even more than TLC August

22 Please stop by booth #307 to see the latest offerings and technology demonstrations from Toshiba Memory America NVMe is a trademark of NVM Express, Inc. Information, including product pricing and specifications, content of services, and contact August 2018 information is current and believed to be accurate on the date of the publication, but is subject to change without prior notice. Technical and application information contained here is subject to the most recent applicable Toshiba product specifications Toshiba Memory America, Inc. 22