Building dense NVMe storage

Transcription

1 Building dense NVMe storage Mikhail Malygin, Principal Software Engineer Santa Clara, CA 1

2 Driven by demand Demand is changing From traditional DBs to NO-SQL Average NO-SQL DB size: 300TB Analytics is everywhere 50% of storage issues: performance Year 2022 forecast NVMe 80% of SSD market Santa Clara, CA 2

3 Motivation There is clear storage paradigm shift Established architectures can t cope with it Storage Architecture need to be revisited New solutions in HW and SW Storage should achieve balance between density, performance and availability Santa Clara, CA 3

4 Clustered storage Hardware setup with shared drives Need a symmetric A+A solution Node synchronization required Parity updates Placement metadata Background tasks Works fine on spinning drives Santa Clara, CA 4

5 NVMe challenges PCIe bus limitation Hotplug CPU usage Santa Clara, CA 5

6 Need a new architecture CPU is the bottleneck, back again Generic solutions do not work Software becomes complex Complex software is less performant Santa Clara, CA 6

7 Hardware Accelerated Architecture U2 NVRAM U2 NVRAM NVMe SSD NVMe RAID NVMe SSD NVMe RAID Storage Processor Power CPU I/O Modules PCIe Fabric U2 NVRAM U2 NVRAM NVMe RAID NVMe SSD NVMe RAID NVMe SSD Storage Processor Power CPU Santa Clara, CA U.2 NVRAM Module is the patent pending solution by YADRO 7

8 Clustering revised Cluster with PCIe Fabric All members share the same bus topology Simultaneous access to all system devices Storage, IO, Acceleration, Synchronization SR-IOV for multi host access Santa Clara, CA 8

9 Storage revised NVMe RAID Controller Dedicated CPU for EC and NVMe operations Aggregated drives, less PCIe devices on CPU Direct I/O path, from card to controller Santa Clara, CA 9

10 NVMe RAID Controller Features PCIe Switch side RAID Controller side Powerful multi-cores ARM A72 SoC PCIe Gen4 support Flexible protection algorithms SR-IOV provider for drives array Multiple namespaces support Battery-protected cache NVRAM support Santa Clara, CA 10

11 U.2 NVRAM Module Industry standard form-factor PCIe Gen4 support Powerful multi-core A72 SoC for in-situ data processing Up to 256GB of DDR4 memory backed-up with 512GB flash Unlimited RAM write endurance NVMe mode and direct memory access mode External battery support Santa Clara, CA U.2 NVRAM Module is the patent pending solution by YADRO 11

12 Solving Active-Active problem Hardware accelerated Key - Value as PCIe device Based on U.2 NVRAM Module Makes KV operations atomic Stores metadata and cache Scales up by partitioning Santa Clara, CA U.2 NVRAM Module is the patent pending solution by YADRO 12

13 PCIe Fabric NVMe Drives Module PCIe switches 16xU.2 NVMe SSDs Management sideband Dual-port drives support Optional NVMe RAID Mezzanine-attached NVMe RAID controllers Santa Clara, CA 13

14 Rear 16xFHHL AIC I/O module (optional, double-wide) PCIe Fabric Controller Up to eight unified PCIe-attached modules Side 6xHHHL AIC I/O uplinks module (4x16 or 2x16+4x8) Root PCIe fabric switches & management Santa Clara, CA 14

15 Thank you! Questions? Santa Clara, CA 15

16 Data protection challenges 10+ TB per drive Traditional RAID does not scale good enough Rebuild overhead for RAID 5/6 Spare overhead for RAID 60 Use RAID with thin provision and flexible placement Just place data somewhere in the enclosure and keep index Flexible protection scheme with erasure coding Santa Clara, CA 16

17 NVMe challenges (CPU usage) Santa Clara, CA 17

18 PCIe Fabric Controller Features Redundant PCIe topology PCIe Gen4 ready 4kW redundant PSUs Redundant management Sideband management network Internal/external hosts support Wide (PCIe 4x16) uplinks Santa Clara, CA 18

19 NVMe RAID diagram Santa Clara, CA 19

20 PCIe Fabric NVMe Drives Module 16xU.2 NVMe SSDs Management sideband Dual-port drives support Optional NVMe RAID Santa Clara, CA 20

21 NVRAM Module Features Dual mode access (NVMe & direct mapped memory) Transactional memory with atomic operations SR-IOV for sharing among multiple hosts via PCIe Fabric Acceleration for storage applications Hash calculation, compression, encryption Shadow replication (redundancy) Configuration & management via NVMe command set Santa Clara, CA U.2 NVRAM Module is the patent pending solution by YADRO 21