Next Generation Architecture for NVM Express SSD

Transcription

1 Next Generation Architecture for NVM Express SSD Dan Mahoney CEO Fastor Systems Copyright 2014, PCI-SIG, All Rights Reserved 1

2 NVMExpress Key Characteristics Highest performance, lowest latency SSD interface protocol today Architected from ground up for NVM scaling from client to enterprise. Large number of parallel commands Wide queuing interface This protocol begs for unconstrained access to the media 2

3 Essential SSD Key Operations 3

4 Typical SSD Controller CPU - 1 CPU - N PCIe Interface Controller Nand Devices Buffer DMA 4

5 Fabric Based SSD Architecture Utilizes a Non-Blocking Fabric to connect: Multiple Channel Controllers PCIe Interface Memory Controller Embedded CPU Exploits PCIe high performance low latency protocol to connect to array of devices Permits Concurrent Data Flow across all modules 5

6 Fabric Based SSD Architecture DRAM Controller Host CPU ecpu FTL PCIe Interface n n Non-Blocking Fabric coupled to Individual Channel Controllers 6

7 A Highly Scalable Architecture DRAM Controller ecpu(s) NEXTFlas h Engine TM PCIe Interface The architecture of the future: 100TB & 25M IOPS! 7 8/20/2014

8 Command Processing Fetch command and data from host Assemble internal datagram with Fabric header LBA Write Data LBA Write Data Fabric Header 8

9 Message Based Architecture Messages sent to Channel Controllers for Data Movement ecpu Copy Page Data from Channel 1 to Channel 2 Message Copy Complete Send completion status to ecpu Message Message 9

10 Packet Forwarding Schedule Packets for Ingress into Fabric Forward to Destination Packets Fabric Scheduler 10

11 Packet Completions After packets are processed by channel controller, completions are sent to ecpu Permits Asynchronous completions ecpu 11

12 Concurrency Eliminates the Main Source of Latency Latency Random, un-cached (4k blocks) Fabric-based Architecture PCIe Fabric 80µS 3µS 2µS 85µS Typical & Worst Case Legacy Architectures PCIe 80µS 3µS PCIe 80µS 3µS DMA (Setup and Teardown) µS µS Optimal DMA (Setup and Teardown) µS µS Typical µs 12 8/20/2014

13 Determinism Enables SDS Unmatched Throughput & Latency Enabling SDS n n NEXT Fabric TM Storage QoS VM storage SLAs Application Optimization IOPs Carving Capacity Carving 13 8/20/2014

14 Legacy vs Fabric Summary Scaling by Controller Replication Fabric Operated ctlr NAND flash Host CPU PCIe RAID ctlr ctlr ctlr NAND flash NAND flash Host CPU DRAM ctlr NAND flash Islands of no global wear leveling Control resources must be replicated to scale the Comingled control and data planes make the controller a choke point for data movements 8/20/2014 Saturate Interface Deterministic Highly Scalable SDS enablement Low TCO 14

15 Comparison of Write Overhead Controller-Centric Fabric-Based 1. ID DMA channel 1. Fire and forget! 2. CPU issues start address 3. CPU hand over end address 4. CPU send start command to DMA controller 5. DMA transmits from memory to Five steps CPU cycles compared to one 15

16 Advantages of Fabric Based Architecture Allows asynchronous movement of datagrams within the SSD Concurrency! No Setup/Tear down of DMA channels Provide QoS within the SSD Separation of control and data planes offers scalable and deterministic performance Agnostic regarding interface and storage media aligned with emerging technologies such as MRAM, PCM, etc.. 16

17 Ultimate Architectural Potential This interface-agnostic architecture can support higher speed interfaces i.e. HMC This would allow large scale modules that look like DRAM to the host CPU HMC modules attached to an HMCenabled processor could ultimately scale to 10X Bandwidth improvement with 2.5X or greater latency reduction compared to state of the art based solutions Potential appliance capacity of 25% of an HDDbased appliance with 100X the bandwidth 17

18 HMC Attached L3 Cache Fastor HMC-based Processor Core Cluster HMC HA 160 GB/ s HMC DRAM PCIe Root Complex HMC Fastor Controller Crossbar Switch Blue arrows signify control plane traffic if separate from data plane traffic HMC Embedded CPU Arrays 18

19 Conclusion Better controllers are good, but if they remain in the SSD s data path, they remain a bottleneck Emerging storage software solutions will all benefit from faster, more deterministic, more scalable storage hardware 19