The Need for Consistent IO Speed in the Financial Services Industry. Silverton Consulting, Inc. StorInt Briefing

Transcription

1 The Need for Consistent IO Speed in the Financial Services Industry Silverton Consulting, Inc. StorInt Briefing

2 THE NEED FOR CONSISTENT IO SPEED IN THE FINANCIAL SERVICES INDUSTRY PAGE 2 OF 5 Introduction Micron has been at the forefront of NAND and dynamic random-access memory (DRAM) technology for a long time and has recently invested in bringing to market 3D NAND non-volatile memory as the next-generation flash storage. As the latest evolution of flash technology, 3D NAND uses the vertical axis to provide more storage per square inch and larger bit storage. As a result, 3D NAND moves beyond 2D or planar NAND in offering more capacity in the same footprint while providing greater endurance and other beneficial characteristics. For example, higher capacity brings advantages such as more free write pages, which can be used to improve write IO performance and reduce response time variability. The speed, density and endurance of 3D NAND can provide a natural advantage in several application areas above and beyond the advantages offered by alternative solid state drive (SSD) technologies available on the market today. For instance, over the past decade trading in the financial services industry has turned into a race to the finish. With more than $275B being traded every day worldwide, companies that offer faster transaction processing reaping much higher profits than slower competitors. Financial services need for speed Many financial services applications use high speed as a competitive advantage, including: High-frequency trading (HFT) the trading of stocks and other securities in the fastest manner possible to stay a step ahead of the competition and gain rapid-fire profits. Many financial services organizations doing HFT have physically relocated their data centers closer to security trading platforms to shave milliseconds from data transit times. Event-based trading the trading on securities in response to adverse or favorable news in order to generate profits. Great Britain s exit from the EU, OPEC s agreement to curtail production, unexpected airline outages are examples of events that present unique opportunities for profit. Trend-based trading the monitoring of market activities for persistent changes in the value of securities. Like HFT and event-based trading, capitalizing on trends early on can lead to significant profits. HFT, event-based trading and trend-based trading all have the risk of inappropriate or improper transaction activities, so any trades need to be actively monitored for regulatory compliance. Compliance assessments depend on logs gathered during the trading period, which means real-time logging often occurs in the critical path of transaction execution and thus has the potential to slow security trades. Compliance

3 THE NEED FOR CONSISTENT IO SPEED IN THE FINANCIAL SERVICES INDUSTRY PAGE 3 OF 5 may not be a profit-generating activity, but it nonetheless demands high IO speed to be performed effectively. Banks also have an unquenchable need for IO speed. The number of credit/debit card transactions that occur on an average day is significant, but transaction volumes can stagger the imagination on Black Friday, Cyber Monday and other big shopping days. During high-volume shopping days, real-time credit transaction validation can save credit and banking organizations a great deal of money. Consistent vs. variable IO speed and application response time The above applications need to work as fast as possible to generate trading profits and reduce credit fraud, which requires a combination of high-speed compute, networking and IO activity, as described below: Compute processing speed has increased by orders of magnitude over the years, and nowadays each new generation of processor chip adds more cores which provide more compute parallelism to process even more data more quickly. Networking speed has increased, but the only way to increase transmission speed is to shrink the transmission distance. For networking speed, proximity counts. IO speed has been the slowest to improve. Over decades of disk evolution, only modest improvements have occurred in access time. However, the introduction of flash/nand technology for data storage has finally increased IO speed by orders of magnitude. Even with the speed of NAND today, achieving consistent IO access times can be challenging. The physical characteristics of NAND technology require sophisticated management of pages, blocks and tables to supply adequate write endurance while at the same time providing high performance in mixed IO environments. Consistent access times matter. Even if the average SSD can perform 100,000 IOs/sec, the variance or dispersion around this average can range from 1,000 to 2,000 IOs/sec. In other words, an SSD with a higher variance will occasionally perform much worse than one with a lower variance. Storage with consistent IO speed Micron 3D NAND SSDs optimize device management activity for performance consistency. Because they are consistency optimized, Micron s 3D NAND SSs can maintain excellent performance over a broad range of workloads To accurately measure IO consistency, it can be useful to examine the ratio of worst-case IO performance to average IO performance. Figure 1 compares three SSDs, one from Micron and two from another vendor.

4 THE NEED FOR CONSISTENT IO SPEED IN THE FINANCIAL SERVICES INDUSTRY PAGE 4 OF 5 Figure 1 SSD IO performance over 1 second in a write-intensive environment* The graph plots the 1 IO/second performance achieved by the three devices for one hour (3,600 intervals) of write-intensive IO activity. As shown by the height of their respective performance lines, the average IO performance varies significantly between the three SSDs. The Micron 5100 MAX 960GB SSD performed on average ~51.7% better than the Competitor A NVMe SSD and ~170.5% better than the Competitor A SATA SSD. By viewing the chart one can see that the Micron 5100 MAX 960GB SSD 1-second IO performance is more consistent (the scatter plot is narrower) than the Competitor A NVMe SSD, but less consistent (the scatter plot is wider) than the Competitor A SATA SSD. However, the visual dispersion comparison doesn t tell the whole story. For each device, it can be useful to divide the 99.9% worst IO performance over a 1-second interval by the device s average 1-second IO performance for the whole hour. In this analysis, the Micron 3D NAND SSD achieved 95.3% of its average IO performance, while Competitor A s NVMe SSD achieved only 90.9% of its average IO performance and Competitor A s SATA SSD achieved 93.2% of its average performance. Similar calculations using 80%, 90%, 95%, 99.5% and 100% (minimum IO performance achieved over a 1-second interval) show that in each case the Micron 5100 MAX SSD performs more IO and more consistently than the competitive SSDs. Previously, we discussed mathematical variance or dispersion of a devices IO performance. As measured, the device with the smallest statistical IO variance is the Figure 1 uses a 100% write workload, which is worst-case IO for NAND-based SSDs. This workload exercises each device for 1 hour, measuring the IOs achieved each second. Raw data and chart provided by Micron from testing in their labs. *

5 THE NEED FOR CONSISTENT IO SPEED IN THE FINANCIAL SERVICES INDUSTRY PAGE 5 OF 5 Competitor A SATA SSD, while the Micron 5100 MAX came in second and Competitor A NVMe SSD came in last. Note, this performance testing used a write-intensive workload, which is a worstcase workload for SSDs. Results may vary considerably. However, most application workloads have some amount of write activity, and inconsistent performance in write-intensive workloads will show up in more normal (mixed) workloads as well. Furthermore, IO performance typically degrades the most when a device is encountering its highest performance demand. In these situations, the sophistication of Micron s internal NAND management plays an increasingly important role in determining overall SSD IO performance. All this means that if it s Black Friday, IO performed to analyze credit validity will occur with the highest consistency and fastest manner available. And if OPEC finally agrees on production targets, event-based trading activity IOs are completed before competitors, using other vendor SSDs, have a chance to react. Summary The need for IO speed is particularly acute in the financial services industry. With more than $275B being traded every day worldwide, any slight performance advantage can reap major profits, whereas even the smallest performance degradation can result in huge financial losses. The higher and more consistent IO performance available from the Micron 5100 MAX 3D NAND SSDs are an excellent solution for applications requiring consistent IOPS performance and low average latency such as trading and credit analysis. The Micron 5100 MAX 3D NAND SSD is the solution for organizations looking for the fastest IO every time. Silverton Consulting, Inc., is a U.S.-based Storage, Strategy & Systems consulting firm offering products and services to the data storage community. Disclaimer: This document was developed with Micron Technology, Inc. funding. Although the document uses publicly available material from various sources, including Micron, it does not necessarily reflect the positions of such sources on the issues addressed.