/ At the heart of a new generation of data center infrastructures and appliances Sept 2017
VIRTUALIZED DATACENTER: THE BLENDER EFFECT FOR STORAGE I/O OPERATIONS 10, 000 VMs MIOPs in random Aggregate switch Compute Nodes 10,000s of VMs generate millions of random IOPs on the storage side. Centralized storage Page 2 2017 Kalray SA All Rights Reserved
NVMe s: THE ANSWER TO THE RANDOM MIOPs DEMAND SAS HDD SAS NVMe 150 IOPs 6ms-200ms latency X 1000 more compute 200 KIOPs 140µs latency X4 more compute 800 KIOPs 115µs latency NVMe s deliver 4000x better performances than traditional SAS HDDs. Page 3 2016 Kalray SA All Rights Reserved
FROM SAS TO NVMe Top of Rack Switch (ToR) Compute Nodes Top of Rack Switch (ToR) Compute Nodes Storage Servers SAS HBA FE Fabric Storage Servers CPU CPU FE Fabric SAS Fabric SAS HBA SAS Switch A SAS Switch B JBOF (Just a Bunch Of Flash) SAS EXPANDER SAS EXPANDER SAS Fabric SAS to NVMe Point-topoint connection JBOF (Just a Bunch Of Flash) Page 4 2017 Kalray SA All Rights Reserved FMS 2017
: The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Point-topoint connection Not designed for Rack communication JBOF (Just a Bunch Of Flash) Page 5 2017 Kalray SA All Rights Reserved FMS 2017
: The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Adding JBOFs means adding head nodes Page 6 2017 Kalray SA All Rights Reserved FMS 2017
: The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Point-topoint connection 6 Head Nodes Max JBOF (Just a Bunch Of Flash) Scale storage head nodes based on services Page 7 2017 Kalray SA All Rights Reserved FMS 2017
: The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Scale storage head nodes based on services Point-topoint connection JBOF (Just a Bunch Of Flash) 1 to 6 Head Nodes per JBOF Limited compute-to-storage ratio and flexibility Page 8 2017 Kalray SA All Rights Reserved FMS 2017
NVMe JBOF: THE 2 OTHER SOLUTIONS EXISTING TODAY Compute Node Top of Rack Switch (ToR) Top of Rack Switch (ToR) NVMe-oF BE Fabric Storage Head Nodes HBA NIC HBA NIC X86 JBOF (Just a Bunch Of Flash) RNIC Switch Ethernet FE Fabric Hyperconverged nodes... Ethernet RNIC Switch X86 JBOF HYPERCONVERGED Lower Density High Cost/High Power Compute/storage ratio is fixed Page 9 2016 Kalray SA All Rights Reserved Confidential information
THE IDEAL SOLUTION: NVMe-oF JBOF Compute Node Top of Rack Switch (ToR) NVMe-oF Target Controller RNIC CPU NVMe-oF BE Fabric HBA NIC HBA NIC NVMe-OF Target Switch CPU CPU CPU CPU Ethernet FE Fabric NVMe-OF Target Switch NVMe-oF JBOF Scale head nodes based on services Scale storage as needed Leverage existing JBOF designs High Density Cost/power optimized Density of JBOF with the flexibility of x86 JBOF Page 10 2016 Kalray SA All Rights Reserved Confidential information
Page 11 2016 Kalray SA All Rights Reserved Confidential Information
NVMe-oF STORAGE SOLUTION: KALRAY TARGET CONTROLLER (KTC40/KTC80) KALRAY TARGET CONTROLLER FUNCTION JBOF Target Controller RNIC Manages all the storage functions of the new generation storage JBOF. CPU Target Controller RNIC CPU SW SW U.2 U.2 TARGET CONTROLLER FEATURE RC MODE FOR DIRECT CONTROL Standard Linux with NVMe Driver Control up to 255 endpoints Any NVMe supported no need for CMB Hot Plug Support NVMe-oF PROTOCOL OVER RoCEv1/v2 4x + performant than SAS (IOPs &throughput) Scalability: Connect up to 2048 initiator cores standard ethernet connectivity LOW ADDITIONAL LATENCY 15 µs for 4KB block transfer BOARD MANAGEMENT CONTROL (BMC) Supervise enclosure HIGH AVAILABILITY ARCHITECTURE End-to-end Multipath architecture END USER INLINE PROCESSING Compression, Encryption, Page 12 2016 Kalray SA All Rights Reserved
KTC40 & KTC80 HARDWARE SPECIFICATION KTC80 MPPA 2.2-256 (Bostan2 processor) 80 GbE of sustained throughput 2 x QSFP+ ports 16-lane Gen3 2 x DDR3-1866 with ECC (4GB) FHHL (Full-Height, Half-Length) Embedded switch with bifurcation up to 4 x 4-lane KTC40 MPPA 2.2-256 (Bostan2 processor) 40GbE 2 x QSFP+ ports 8-lane Gen3 2 x DDR3-1866 with ECC (2GB) LP (Low-profile)
KALRAY LEADS THE INDUSTRY IN NVMe-oF COMPATIBILITY OPERATION KTC40 KTC80 Ethernet 1.6 MIOPs 3.2 MIOPs (NVMe Direct/Root Complex) 67%RD / 33% WR @4KB 15 µs latency 15 µs latency Highest possible throughput. A whole family of products. Page 14 2016 Kalray SA All Rights Reserved Confidential information
4x10G / 40G Ethernet 4x10G / 40G Ethernet KALRAY I/O BOSTAN PROCESSOR OVERVIEW HIGH-SPEED INTERFACES: 2x 40GbE 2x Gen3 8-lanes (EP/RC) Quad core 0 8-lane Gen3 Quad core 1 CONNECTED TO A LARGE ARRAY OF PROCESSING Full C/C++ Programmable Dataplane execution VIA A HIGH BANDWIDTH LOW LATENCY NETWORK ON CHIP Direct packet-to-core delivery Direct core-to-core transfers Direct connect between multiple MPPAs C C C C C C C C NoC C C C C C C C C AND I/O Quad CORES Runs Linux Runs control plane Quad core 2 8-lane Gen3 Quad core 3
KTC NVMe-oF SOFTWARE STACK SES Mgmt USER CONTROL PLANE Configuration SES ports Linux USER INLINE PROCESSING (DEDUP, ENCRYPTION, ) NVME-oF RoCEv1/v2 ODP Libnoc NVMe RA ODP Firmware iscsi TCP/IP NVME-oF /iser RoCE v1/v2 NoC Driver NVME RA NVME RC Hypervisor Hypervisor Clusters Quad Cores MPPA Processor Page 16 2016 Kalray SA All Rights Reserved
END USER CUSTOMIZABLE SOLUTION CUSTOMIZABLE FUNCTIONS INLINE PROCESSING Compression Encryption Deduplication Erasure Coding BOARD MANAGEMENT CONTROL (BMC) REDFISH/SWORDFISH SES openbmc END USER READ/WRITE OPERATION POLICY Implement optimized Read/write scheduling to eliminate outliers on critical streams Achieve a low latency for 99.9999% Page 17 2017 Kalray SA All Rights Reserved Confidential information
Page 18 2016 Kalray SA All Rights Reserved Confidential Information
YOUR JBOF EASILY BECOMES AN ETHERNET JBOF WITH KALRAY TARGET CONTROLLER JBOF NVMe-oF JBOF No Modifications KTC ENABLES A FAST TIME-TO-MARKET TO BUILD NVMe-oF JBOF Page 19 2016 Kalray SA All Rights Reserved
INITIATOR BOTTLENECK SOLUTION JBOF NVMe-oF JBOF MIOPs Initiator cores KIOPS/ Core 12 432 28 MIOPs Initiator cores KIOPS/ Core 12 24, 576 1 X6 Up to 6 x 72 Initiator cores Up to 6 x 255 End points X170 X6 Up to 6 x 2048 Initiator cores (170 x 72 cores) Up to 6 x 255 End Points NVMe-oF KTC connects to 28x more initiator cores than adapters. This solves the initiator bottleneck issue! Page 20 2017 Kalray SA All Rights Reserved
NVMe-oF JBOF: Scales the storage capacity JBOF NVMe-oF JBOF Top of Rack Switch (ToR) Compute Node Point-topoint connection Compute Nodes Storage Servers JBOF (Just a Bunch Of Flash) FE Fabric NVMe-oF BE Fabric Top of Rack Switch (ToR) HBA NIC HBA NIC KTC Switch CPU CPU CPU CPU Ethernet FE Fabric KTC Switch NVMe-oF KTC Page 21 2017 Kalray SA All Rights Reserved
KTC ENABLES HIGH AVAILABILITY ARCHITECTURE END-TO-END REDUNDANT PATH Dual port U.2 NVME Dual Trees Dual KTC40/80 connectivity MULTIPATH HANDLED AT THE INITATOR SIDE Standard feature available in Linux Kernel Support Active-Active or Active-Standby modes Page 22 2017 Kalray SA All Rights Reserved
SCALE PERFORMANCE UP TO PEAK CAPABILITIES 6.4 MIOPs 17.6 GB/s 15 MIOPs 48 GB/s Scale up to peak performances Global performances 6.4 MIOPs 17.6 GB/s 19.2 MIOPs 52.8 GB/s 15 MIOPs 48 GB/s Global performances 15 MIOPs 48 GB/s MIOPS : BANDWIDTH : RANDOM - 66% RD / 33% WR - 4KB RANDOM - 100% RD 4KB Page 23 2017 Kalray SA All Rights Reserved
x86-based JBOF Versus KTC-based JBOF: performance optimized X86 JBOF architecture Same density. = KTC-based architecture DENSITY: 24 s in 2U (77TB) CPU+ NIC FUNCTION 2 x XEON E5-2667v4 8 x 16GB DDR4 3 x 100G NIC POWER: 309 W PERFORMANCE: 9.4 MIOPs More power efficient. 32% Higher performance 60% DENSITY: CPU + NIC FUNCTION 6 x KTC80 POWER: 24 s in 2U (77TB) 210 W PEFORMANCE: 15 MIOPS ELIMINATE THE HIGH COST/ HIGH POWER x86 SYSTEM (CPU, MEMORY, ) WHILE INCREASING THE PERFORMANCES BY 60%
x86-based JBOF Versus KTC-based JBOF: density optimized X86 JBOF architecture Better performance. 20% KTC-based architecture DENSITY: 154TB in 2U SPECIFICATION 2 x XEON E5-2667v4 8 x 16GB DDR4 2 x 100G NIC POWER: 294 W PERFORMANCE: 6.25 MIOPs More Power effective. 64% Greater density. 58 % DENSITY: 240 TB in 2OU SPECIFICATION CHASSIS WITH 250 M.2 in 2OU 3 x KTC80-LP POWER: 105 W PEFORMANCE: 7.5 MIOPS ELIMINATE THE HIGH COST/ HIGH POWER x86 SYSTEM (CPU, MEMORY, ) WHILE INCREASING DENSITY AND OPTIMZING COST AND POWER
STORAGE: PAY AS YOU GROW WITH KALRAY TARGET CONTROLLER Top of Rack Switch (ToR) NVMeoF BE Fabric Storage Server HBA NIC KTC KTC KTC JBOF (Just a Bunch Of Flash) Ethernet FE Fabric KALRAY UNIQUE ADVANTAGE: CHAIN NVMe-oF JBOFs KEEP THE SAME INFRASTRUCTURE ToR switch Number of storage servers PAY AS YOU GROW! Pay only for additional storage capacity Not for additional storage servers or Top of Rack Switch The chaining equivalent to SAS protocol. Page 26 2016 Kalray SA All Rights Reserved Confidential information
NVMe-oF JBOF ENABLES DISAGREGATED HYPERCONVERGED ARCHITECTURE Top of Rack Switch (ToR) Top of Rack Switch (ToR) Hyperconverged nodes Ethernet Hyperconverged nodes... Converged Ethernet (NVMe-oF)...... Target Controller Target Controller Hyperconverged /SDS Hyperconverged/SDS scales naturally Compute/storage ratio is fixed DAS is expansive Disaggregated Hyperconverged/SDS Scale compute & storage independently Leverage existing JBOF designs Page 27 2017 Kalray SA All Rights Reserved FMS 2017
Page 28 2016 Kalray SA All Rights Reserved Confidential Information
NVMe-oF : THE SOLUTION FOR NEW GENERATION OF STORAGE SYSTEMS NVMe-oF TARGET $$$ THE SOLUTION FOR NVMe-oF JBOF 4X HIGHER IOPS THAN SAS SCALABLE & FLEXIBLE FAST TIME TO MARKET Eliminate the need of X86 and associated system memory End-to-end NVMe/NVMe-oF capabilities ensure 4X more IOPS Scale the Head Nodes and Storage capacity independently Plug NVMe-oF Target controller in your standard JBOF Page 29 2016 Kalray SA All Rights Reserved
KALRAY S.A. - GRENOBLE - FRANCE 445 rue Lavoisier, 38 330 Montbonnot - France Tel: +33 (0)4 76 18 09 18 email: info@kalray.eu KALRAY INC. - LOS ALTOS - USA 4962 El Camino Real Los Altos, CA - USA Tel: +1 (650) 469 3729 email: info@kalrayinc.com MPPA, ACCESSCORE and the Kalray logo are trademarks or registered trademarks of Kalray in various countries. All trademarks, service marks, and trade names are the marks of the respective owner(s), and any unauthorized use thereof is strictly prohibited. All terms and prices are indicative and subject to any modification without notice.