Fusion Engine Next generation storage engine for Flash- SSD and 3D XPoint storage system

Transcription

1 Fusion Engine Next generation storage engine for Flash- SSD and 3D XPoint storage system Fei Liu, Sheng Qiu, Jianjian Huo, Shu Li Alibaba Group Santa Clara, CA 1

2 Software overhead become critical Legacy file system and OS is heavy! New storage trend NVM (3DX Point) Flash-SSD HDD Latency 400 ns (expected) 20 us 10 ms Note: numbers come from online third-party websites 0 Latency breakdown (us) file system os hardware 0.4 hdd flash-ssd 3DX point NDIMM Santa Clara, CA 2

3 Our Solution: Customize storage engine FusionEngine Customized file-system Customized device management Customized FTL Bypass OS kernel Santa Clara, CA 3

4 Light-weight FS for NVMe SSD Why a light-weight user space file system POSIX semantic incurs unnecessary overhead Kernel bug fixes/upgrades are painful Easy to customize and add new feature Adopt to new hardware Santa Clara, CA 4

5 User Space Storage File System (USSFS) Light-weight, allocate/read/write/remove chunks User space, much easier to upgrade and customize Very low latency, make best of modern SSDs Enable run-to-completion usage model. Santa Clara, CA 5

6 How USSFS works Separation of control path and data path. SPDK user space NVMe driver USSOS and device manager Pangu Chunk Server Santa Clara, CA 6

7 USSFS memory and meta data management On-disk: Block table and chunk table In-memory: Block cache, chunk cache Chunk index DPDK user space memory library based on physical huge s. Santa Clara, CA 7

8 User-space Storage OS USSOS User-Space Storage OS (USSOS) Build on top of Intel Memory Management Utilities and Perf-counters Scheduler and QoS SPDK/DPDK BUT add tons of new Device Manager Open-channel Driver components and features SPDK DPDK Santa Clara, CA 8

9 USSOS Device Management Manage user space device drivers, NVMe, Optane and etc. Manage device, qpair and resources. Handle hot plug events, bad disks and etc. Santa Clara, CA 9

10 USSOS QoS-aware IO Scheduler Priority-aware Dispatch order Bandwidth usage align IO cost with the assigned resource quota Normalize io cost and device capability disk1 Santa Clara, CA

11 Fusion Engine Application in Alibaba Cloud Deployed widely inside Alibaba cloud infrastructure ESSD cloud disk reaches 1M IOPS, latency at μs scale 50% IOPS increase, 30% CPU utilization decrease. Santa Clara, CA 11

12 Use due-mode SSD Open-channel and legacy mode Opportunities Co-optimization Application and storage Flexibility Diversified and fast-changing workloads More Santa Clara, CA 12

13 Beyond block interface Control data placement APP1 writes Open-Channel mode Block Block Block Control GC APP2 writes Santa Clara, CA 13

14 SW/HW integrated storage architecture Customized FS Customized device management FusionEngine Storage Engine FusionEngine User-space File System FusionEngine User-space Device Management Customized FTL Open-channel mode Open Channel Mode Legacy Mode Open Channel SSD Legacy SSD NVM Santa Clara, CA 14

15 Conclusion Customized SW/HW co-optimization is required FusionEngine solves many challenges BUT more are waiting! Santa Clara, CA 15

16 Thank You! Questions? Santa Clara, CA 16