NVMe SSDs Becoming Norm for All Flash Storage

Transcription

1 SSDs Becoming Norm for All Flash Storage Storage media has improved by leaps and bounds over the last several years. Capacity and performance are both improving at rather rapid rates as popular vendors rollout out their high-performance SSDs. Nearly every single manufacturer of storage equipment around the world has intently researched and developed non-volatile devices and has become one of the most discussed topics in the industry. forged for flash memory What exactly is? It stands for non-volatile memory express, a transmission specification developed by a work team comprising personnel from over 100 companies. So, what is the advantage of the stack? Quite simply, it delivers unprecedented speed and peak bandwidth for solid state drives. Normal SSDs cannot keep up in terms of throughput for particularly high I/O-intensive applications (like massive databases), which is where comes in to deliver unprecedentedly low latency and higher peak rates. Standard protocol interfaces are extremely important to the widening adoption of flash memory. provides a standard access method for flash able to leverage the low latency and concurrency capabilities of Peripheral Component Interconnect Express (PCIe) channels and the concurrency capabilities of modern processors, platforms, and applications. Reduction in protocol latency and increase in concurrency capabilities greatly enhance the read/write performance of SSDs while controlling storage costs. The potential of flash memory is further unleashed and all-ssd at the data center is further promoted. Conventional SAS protocols are designed for mechanical hard disks. Rapid developments in flash media and the ever-widening need for improved

2 performance across systems has brought the disadvantages of SAS protocols to the surface. The protocols are recognized as one of the weakest links in development of all-flash storage systems. Compared to SAS HDDs, throughput and peak IOPS on SSDs have greatly improved and latency has been reduced down to the ultra-low 100 μs level in some cases. SSD capacity has continued to improve, so what can be done about the weak link with the transmission protocol? The NVM Express organization has developed the protocol standard, removing the I/O scheduler and complex SCSI protocol layers in SAS HDD systems and replacing these with the lightweight stack. Application Block Layer SCSI PCIe SAS SAS HBA Disk SAS Disk protocol stack delivers huge improvements in efficiency because each I/O requires less than half the CPU cycle of SAS. Put another way, provides twice the IOPS as SAS in the same CPU cycle. The disadvantages of SAS are rather obvious:

3 1. More interaction times because protocols must be converted between the PCIe and the complex SAS/SCSI stack, and the software processing protocol overhead is rather high (making it difficult to achieve higher IOPS). disks can be directly attached to PCIe interfaces, simplifying protocol interactions for better performance. 2. SAS is a single-queue protocol whereas can support up to 64K queues (each able to support 64K concurrent commands). The multiple queues achieve high levels of concurrency in processing to match the higher performance of the multi-channel, chip-select, and Die formats found in SSDs. Lock protections must be added to the single queues of SAS in multicore environments whereas native supports lock-free mechanisms to better tap into the concurrency advantages of configuring multiple cores while reducing overhead in running software. There is no doubt that is the more efficient protocol, but it is rather new. The SAS protocol, on the other hand, has been put to the test for many years now and is a highly mature protocol, so it does have some advantages: 1. Natively supports multi-port and dual-port disk technology. The protocol is stable and reliable in multi-core systems while the PCIe dual-port technology still needs improvement. 2. Good support for hot-swap operations. I/O-based protocol mechanisms will not cause faults to spread over systems. Adding hot-swap capabilities to PCIe is technologically difficult, which means more accumulation is needed before high reliability with the protocol can be ensured. 3. SAS networking is flexible and easily scalable. Network deployment particulars where fully considered from the start when designing the SAS protocol. Basing the protocol on routing exchange makes subsequent expansions easier. In contrast, the scalability of the PCIe protocol is rather limited a when it comes to the depth of cascade and total number of

4 supported devices. SAS-based SSD storage systems are used in conventional high-performance settings while -based storage systems are prominently used in intense latency-sensitivity cases. SAS will be around for quite some time to come because has its own service particulars. Analysts predict that SSD shipments will exceed those of SAS in 2017, and the gap widen substantially as time goes by. PCIe SSD is the main direction of development and the protocol is best suited to those developments. The increased pace of data center deployments and cloud adoptions also makes SSDs more advantageous for hybrid application scenarios. New SCM media like 3D XPoint is being used in high-speed cache cases. The ultra-high requirements on performance can only be fulfilled by using PCIe channels, which is adding a further push behind the development and use of the stack. In addition to being used in local PCIe SSDs, the NVM Express organization released its over Fabrics specification in June 2016, allowing the stack to be carried on RDMA/FC protocols and thereby providing a high-performance solution for remote access to SSDs and solving the issues in sharing of cross-site and on-premise SSD resources. What are the challenges to storage systems in deploying? is a new protocol that is based on the PCIe technology. While it does deliver huge gains in performance, there are some rather daunting challenges it brings in terms of disk and system design. SSD is a type of hard drive, so it must be able to be replaced quickly and non-disruptively. However, PCIe does not have very good hot-swap capabilities, especially when it comes to PCIe surprise hot plug. As a result, some settings are prone to service interruptions and system faults. Furthermore, enterprise-

5 class storage systems require high reliability and therefore require dualcontroller redundancies at a minimum whereas PCIe-based dual port technology is difficult to effectuate in the SSD design. Some of the SSDs produced do not have native dual PCIe ports (like some of Intel's SSDs that use an internal PCIe switch to achieve the dual-port capability). Taking these types of shortcuts will inevitably lead to a reduction in system reliability. Another challenge to -based storage systems is the difficulty in implementing optimizations at the system level. Some suppose that treating the system like nothing more than an SSD we produce huge advantages in system performance. However, it is not so simple. The system I/O architecture requires a re-design to make it more suitable for the concurrent processing capabilities in multi-core configurations. The re-design allows for the advantages of to be truly leveraged. Huawei's technical accumulation in Huawei started to research flash technologies in Twelve years of accumulated expertise and continuously stepped investments have secured the many core patents in the field for the company and the most when it comes to SSDs. At present, Huawei is the only vendor to home grow its storage operating systems, controllers, and SSDs, giving it the one-of-a-kind capability in performing penetrating end-to-end optimizations on flash. Based on the comprehensive products and solutions provided by Huawei, not to mention its open ecosphere, customers can easily enjoy the great service processing capabilities brought by revolutions in flash technologies. With the most complete portfolio of products and solutions in the industry and its participation in the wide-open ecosystem, Huawei enables customers to avail themselves of the advances in service processing technologies made possible by the flash revolution.

6 Huawei's SSDs support the dual-port technology. The two PCIe 3.0x2 ports are independent from each other, eliminating cross-interference while enabling improved serviceability in system repair and exception handling. System reliability is also improved with dual-controller redundancy. Exception handling in PCIe settings is rather complicated, especially when it comes to ensuring system reliability in a surprise hot plug event. Huawei has been investing into PCIe research for many years now and has a complete grasp on link management, exception handling, and hot-swap capabilities to improve system reliability from end to end because disk replacements can be executed non-disruptively and faults are contained. CPU RP MSI INT PCIe Switch hot plug DP controller PCIe link Slot SSD hot plug SSD hot swap In the figure above, a hot swap of SSD will place the PCIe into an

7 abnormal state. SSDs can be pulled out at any time and with any method on Huawei OceanStor Dorado V3, a clear advantage for systems. The following have been implemented specifically in the design of the PCIe driver for such use cases: In the standard processing process, advanced error reporting (AER) errors will be reported until the I/O process is stopped at the service end, which could lead to an AER error storm and in turn cause complications to the processing procedure. Huawei has turned off AER in the PCIe driver and mechanisms promptly shut down the I/O processes to avoid propagation of error storms. If the pull action behind the disk extraction is not forceful enough and the link is re-established, a back pressure error will result. The links in the PCIe driver processing processes can be quickly disconnected in the Huawei system to avoid the error. When a surprise hot plug event transpires, downstream port containment (DPC) can be triggered by other fatal errors as long as the event does not produce a Surprise Down error. The link can be disconnected faster and the packets sent to the upper layer can be returned much more quickly than that of standard processing processes. Even if disks are being removed slowly, the rapid link-disconnect utility helps ensure system reliability. Huawei has applied its extensive expertise in hard disks to systems, providing comprehensive health monitoring, lifecycle predictions, hardware/software fault monitoring, and repair and isolation utilities for storage media running in these environments. All these adapted applications help ensure a smoother rollout for. The proprietary P3 pin power-on/poweroff repair function provides a highly viable solution to repairing disk failures and

8 most soft failures on SSDs, thereby effectively reducing the failure rate. Optimizations at the system level are required to accommodate the highperformance of. Huawei has designed a set of I/O scheduling mechanisms in its designs. These mechanisms completely remove disk-level mutex form the I/O paths to avoid conflicts in queue locks. This approach substantially improves processing performance at the back end. An example I/O scheduling diagram is provided below; multiple threads and queues are depicted (this layout achieves top performance). CPU CORE 1 Thread 1 CPU CORE 2 Thread 2 CPU CORE n... Thread n... Admin Qu eue IO Qu eue SQ CQ SSD 1 SSD 2 SSD 3 SSD n... I/O scheduling in systems Huawei is fully geared for the all-flash era Huawei OceanStor Dorado V3 all-flash arrays are built to carry the heavily loaded mission-critical applications for organizations of all sizes. The arrays deliver up to 4 million IOPS in performance while maintaining latency at a predictable and stable 0.5 ms. Full system reliability is ensured as organizations accelerate their core businesses. OceanStor Dorado V3 runs on an protocol and the newly designed hardware architecture support dual-port SSDs and surprise hot plug replacement capabilities. The newly designed

9 parallel software architecture brings out the full performance advantages of. Huawei OceanStor Dorado5000 V3 Huawei is fully geared for the all-flash era and is continuing to add more investments into -based storage systems. Huawei has introduced new SCM media to inject even more performance. The release of over Fabrics achieves global sharing of SSD resources and the provisioning of frontend interfaces helps optimize the system hardware and software architecture, adding to the competitiveness of all-flash in the new era.