At the heart of a new generation of data center infrastructures and appliances. Sept 2017

Size: px

Start display at page:

Download "At the heart of a new generation of data center infrastructures and appliances. Sept 2017"

Norma Johns
6 years ago
Views:

1 / At the heart of a new generation of data center infrastructures and appliances Sept 2017

2 VIRTUALIZED DATACENTER: THE BLENDER EFFECT FOR STORAGE I/O OPERATIONS 10, 000 VMs MIOPs in random Aggregate switch Compute Nodes 10,000s of VMs generate millions of random IOPs on the storage side. Centralized storage Page Kalray SA All Rights Reserved

3 NVMe s: THE ANSWER TO THE RANDOM MIOPs DEMAND SAS HDD SAS NVMe 150 IOPs 6ms-200ms latency X 1000 more compute 200 KIOPs 140µs latency X4 more compute 800 KIOPs 115µs latency NVMe s deliver 4000x better performances than traditional SAS HDDs. Page Kalray SA All Rights Reserved

4 FROM SAS TO NVMe Top of Rack Switch (ToR) Compute Nodes Top of Rack Switch (ToR) Compute Nodes Storage Servers SAS HBA FE Fabric Storage Servers CPU CPU FE Fabric SAS Fabric SAS HBA SAS Switch A SAS Switch B JBOF (Just a Bunch Of Flash) SAS EXPANDER SAS EXPANDER SAS Fabric SAS to NVMe Point-topoint connection JBOF (Just a Bunch Of Flash) Page Kalray SA All Rights Reserved FMS 2017

5 : The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Point-topoint connection Not designed for Rack communication JBOF (Just a Bunch Of Flash) Page Kalray SA All Rights Reserved FMS 2017

6 : The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Adding JBOFs means adding head nodes Page Kalray SA All Rights Reserved FMS 2017

7 : The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Point-topoint connection 6 Head Nodes Max JBOF (Just a Bunch Of Flash) Scale storage head nodes based on services Page Kalray SA All Rights Reserved FMS 2017

8 : The issues Top of Rack Switch (ToR) Compute Nodes Storage Servers FE Fabric Primary designed for CPU-to-peripheral Scale storage capacity (pay as you grow) Scale storage head nodes based on services Point-topoint connection JBOF (Just a Bunch Of Flash) 1 to 6 Head Nodes per JBOF Limited compute-to-storage ratio and flexibility Page Kalray SA All Rights Reserved FMS 2017

9 NVMe JBOF: THE 2 OTHER SOLUTIONS EXISTING TODAY Compute Node Top of Rack Switch (ToR) Top of Rack Switch (ToR) NVMe-oF BE Fabric Storage Head Nodes HBA NIC HBA NIC X86 JBOF (Just a Bunch Of Flash) RNIC Switch Ethernet FE Fabric Hyperconverged nodes... Ethernet RNIC Switch X86 JBOF HYPERCONVERGED Lower Density High Cost/High Power Compute/storage ratio is fixed Page Kalray SA All Rights Reserved Confidential information

10 THE IDEAL SOLUTION: NVMe-oF JBOF Compute Node Top of Rack Switch (ToR) NVMe-oF Target Controller RNIC CPU NVMe-oF BE Fabric HBA NIC HBA NIC NVMe-OF Target Switch CPU CPU CPU CPU Ethernet FE Fabric NVMe-OF Target Switch NVMe-oF JBOF Scale head nodes based on services Scale storage as needed Leverage existing JBOF designs High Density Cost/power optimized Density of JBOF with the flexibility of x86 JBOF Page Kalray SA All Rights Reserved Confidential information

12 NVMe-oF STORAGE SOLUTION: KALRAY TARGET CONTROLLER (KTC40/KTC80) KALRAY TARGET CONTROLLER FUNCTION JBOF Target Controller RNIC Manages all the storage functions of the new generation storage JBOF. CPU Target Controller RNIC CPU SW SW U.2 U.2 TARGET CONTROLLER FEATURE RC MODE FOR DIRECT CONTROL Standard Linux with NVMe Driver Control up to 255 endpoints Any NVMe supported no need for CMB Hot Plug Support NVMe-oF PROTOCOL OVER RoCEv1/v2 4x + performant than SAS (IOPs &throughput) Scalability: Connect up to 2048 initiator cores standard ethernet connectivity LOW ADDITIONAL LATENCY 15 µs for 4KB block transfer BOARD MANAGEMENT CONTROL (BMC) Supervise enclosure HIGH AVAILABILITY ARCHITECTURE End-to-end Multipath architecture END USER INLINE PROCESSING Compression, Encryption, Page Kalray SA All Rights Reserved

KTC40 & KTC80 HARDWARE SPECIFICATION KTC80 MPPA 2.

QSFP+ ports 16-lane Gen3 2 x DDR3-1866 with ECC (4GB) FHHL

13 KTC40 & KTC80 HARDWARE SPECIFICATION KTC80 MPPA (Bostan2 processor) 80 GbE of sustained throughput 2 x QSFP+ ports 16-lane Gen3 2 x DDR with ECC (4GB) FHHL (Full-Height, Half-Length) Embedded switch with bifurcation up to 4 x 4-lane KTC40 MPPA (Bostan2 processor) 40GbE 2 x QSFP+ ports 8-lane Gen3 2 x DDR with ECC (2GB) LP (Low-profile)

14 KALRAY LEADS THE INDUSTRY IN NVMe-oF COMPATIBILITY OPERATION KTC40 KTC80 Ethernet 1.6 MIOPs 3.2 MIOPs (NVMe Direct/Root Complex) 67%RD / 33% 15 µs latency 15 µs latency Highest possible throughput. A whole family of products. Page Kalray SA All Rights Reserved Confidential information

4x10G / 40G Ethernet 4x10G / 40G Ethernet KALRAY I/O BOSTAN PROCESSOR OVERVIEW HIGH-SPEED INTERFACES: 2x 40GbE 2x Gen3 8-lanes (EP/RC) Quad core 0 8-lane Gen3 Quad core 1 CONNECTED TO A LARGE ARRAY

15 4x10G / 40G Ethernet 4x10G / 40G Ethernet KALRAY I/O BOSTAN PROCESSOR OVERVIEW HIGH-SPEED INTERFACES: 2x 40GbE 2x Gen3 8-lanes (EP/RC) Quad core 0 8-lane Gen3 Quad core 1 CONNECTED TO A LARGE ARRAY OF PROCESSING Full C/C++ Programmable Dataplane execution VIA A HIGH BANDWIDTH LOW LATENCY NETWORK ON CHIP Direct packet-to-core delivery Direct core-to-core transfers Direct connect between multiple MPPAs C C C C C C C C NoC C C C C C C C C AND I/O Quad CORES Runs Linux Runs control plane Quad core 2 8-lane Gen3 Quad core 3

KTC NVMe-oF SOFTWARE STACK SES Mgmt USER CONTROL PLANE Configuration SES ports Linux USER INLINE PROCESSING (DEDUP, ENCRYPTION, ) NVME-oF RoCEv1/v2 ODP Libnoc NVMe RA ODP

16 KTC NVMe-oF SOFTWARE STACK SES Mgmt USER CONTROL PLANE Configuration SES ports Linux USER INLINE PROCESSING (DEDUP, ENCRYPTION, ) NVME-oF RoCEv1/v2 ODP Libnoc NVMe RA ODP Firmware iscsi TCP/IP NVME-oF /iser RoCE v1/v2 NoC Driver NVME RA NVME RC Hypervisor Hypervisor Clusters Quad Cores MPPA Processor Page Kalray SA All Rights Reserved

END USER CUSTOMIZABLE SOLUTION CUSTOMIZABLE FUNCTIONS INLINE PROCESSING Compression Encryption Deduplication Erasure Coding BOARD MANAGEMENT CONTROL (BMC) REDFISH/SWORDFISH SES openbmc END USER

17 END USER CUSTOMIZABLE SOLUTION CUSTOMIZABLE FUNCTIONS INLINE PROCESSING Compression Encryption Deduplication Erasure Coding BOARD MANAGEMENT CONTROL (BMC) REDFISH/SWORDFISH SES openbmc END USER READ/WRITE OPERATION POLICY Implement optimized Read/write scheduling to eliminate outliers on critical streams Achieve a low latency for % Page Kalray SA All Rights Reserved Confidential information

YOUR JBOF EASILY BECOMES AN ETHERNET JBOF WITH KALRAY TARGET CONTROLLER JBOF NVMe-oF JBOF No

INITIATOR BOTTLENECK SOLUTION JBOF NVMe-oF JBOF MIOPs Initiator cores KIOPS/ Core 12 432 28 MIOPs

Up to 6 x 2048 Initiator cores (170 x 72 cores) Up to 6 x 255 End Points NVMe-oF KTC connects to 28x more

20 INITIATOR BOTTLENECK SOLUTION JBOF NVMe-oF JBOF MIOPs Initiator cores KIOPS/ Core MIOPs Initiator cores KIOPS/ Core 12 24, X6 Up to 6 x 72 Initiator cores Up to 6 x 255 End points X170 X6 Up to 6 x 2048 Initiator cores (170 x 72 cores) Up to 6 x 255 End Points NVMe-oF KTC connects to 28x more initiator cores than adapters. This solves the initiator bottleneck issue! Page Kalray SA All Rights Reserved

21 NVMe-oF JBOF: Scales the storage capacity JBOF NVMe-oF JBOF Top of Rack Switch (ToR) Compute Node Point-topoint connection Compute Nodes Storage Servers JBOF (Just a Bunch Of Flash) FE Fabric NVMe-oF BE Fabric Top of Rack Switch (ToR) HBA NIC HBA NIC KTC Switch CPU CPU CPU CPU Ethernet FE Fabric KTC Switch NVMe-oF KTC Page Kalray SA All Rights Reserved

2 NVME Dual Trees Dual KTC40/80 connectivity MULTIPATH HANDLED AT THE

22 KTC ENABLES HIGH AVAILABILITY ARCHITECTURE END-TO-END REDUNDANT PATH Dual port U.2 NVME Dual Trees Dual KTC40/80 connectivity MULTIPATH HANDLED AT THE INITATOR SIDE Standard feature available in Linux Kernel Support Active-Active or Active-Standby modes Page Kalray SA All Rights Reserved

23 SCALE PERFORMANCE UP TO PEAK CAPABILITIES 6.4 MIOPs 17.6 GB/s 15 MIOPs 48 GB/s Scale up to peak performances Global performances 6.4 MIOPs 17.6 GB/s 19.2 MIOPs 52.8 GB/s 15 MIOPs 48 GB/s Global performances 15 MIOPs 48 GB/s MIOPS : BANDWIDTH : RANDOM - 66% RD / 33% WR - 4KB RANDOM - 100% RD 4KB Page Kalray SA All Rights Reserved

24 x86-based JBOF Versus KTC-based JBOF: performance optimized X86 JBOF architecture Same density. = KTC-based architecture DENSITY: 24 s in 2U (77TB) CPU+ NIC FUNCTION 2 x XEON E5-2667v4 8 x 16GB DDR4 3 x 100G NIC POWER: 309 W PERFORMANCE: 9.4 MIOPs More power efficient. 32% Higher performance 60% DENSITY: CPU + NIC FUNCTION 6 x KTC80 POWER: 24 s in 2U (77TB) 210 W PEFORMANCE: 15 MIOPS ELIMINATE THE HIGH COST/ HIGH POWER x86 SYSTEM (CPU, MEMORY, ) WHILE INCREASING THE PERFORMANCES BY 60%

x86-based JBOF Versus KTC-based JBOF: density optimized X86 JBOF

20% KTC-based architecture DENSITY: 154TB in 2U SPECIFICATION 2

PERFORMANCE: 6.25 MIOPs More Power effective.

58 % DENSITY: 240 TB in 2OU SPECIFICATION CHASSIS WITH 250 M.

25 x86-based JBOF Versus KTC-based JBOF: density optimized X86 JBOF architecture Better performance. 20% KTC-based architecture DENSITY: 154TB in 2U SPECIFICATION 2 x XEON E5-2667v4 8 x 16GB DDR4 2 x 100G NIC POWER: 294 W PERFORMANCE: 6.25 MIOPs More Power effective. 64% Greater density. 58 % DENSITY: 240 TB in 2OU SPECIFICATION CHASSIS WITH 250 M.2 in 2OU 3 x KTC80-LP POWER: 105 W PEFORMANCE: 7.5 MIOPS ELIMINATE THE HIGH COST/ HIGH POWER x86 SYSTEM (CPU, MEMORY, ) WHILE INCREASING DENSITY AND OPTIMZING COST AND POWER

26 STORAGE: PAY AS YOU GROW WITH KALRAY TARGET CONTROLLER Top of Rack Switch (ToR) NVMeoF BE Fabric Storage Server HBA NIC KTC KTC KTC JBOF (Just a Bunch Of Flash) Ethernet FE Fabric KALRAY UNIQUE ADVANTAGE: CHAIN NVMe-oF JBOFs KEEP THE SAME INFRASTRUCTURE ToR switch Number of storage servers PAY AS YOU GROW! Pay only for additional storage capacity Not for additional storage servers or Top of Rack Switch The chaining equivalent to SAS protocol. Page Kalray SA All Rights Reserved Confidential information

27 NVMe-oF JBOF ENABLES DISAGREGATED HYPERCONVERGED ARCHITECTURE Top of Rack Switch (ToR) Top of Rack Switch (ToR) Hyperconverged nodes Ethernet Hyperconverged nodes... Converged Ethernet (NVMe-oF) Target Controller Target Controller Hyperconverged /SDS Hyperconverged/SDS scales naturally Compute/storage ratio is fixed DAS is expansive Disaggregated Hyperconverged/SDS Scale compute & storage independently Leverage existing JBOF designs Page Kalray SA All Rights Reserved FMS 2017

system memory End-to-end NVMe/NVMe-oF capabilities ensure 4X more IOPS Scale the Head Nodes and Storage

29 NVMe-oF : THE SOLUTION FOR NEW GENERATION OF STORAGE SYSTEMS NVMe-oF TARGET $$$ THE SOLUTION FOR NVMe-oF JBOF 4X HIGHER IOPS THAN SAS SCALABLE & FLEXIBLE FAST TIME TO MARKET Eliminate the need of X86 and associated system memory End-to-end NVMe/NVMe-oF capabilities ensure 4X more IOPS Scale the Head Nodes and Storage capacity independently Plug NVMe-oF Target controller in your standard JBOF Page Kalray SA All Rights Reserved

30 KALRAY S.A. - GRENOBLE - FRANCE 445 rue Lavoisier, Montbonnot - France Tel: +33 (0) info@kalray.eu KALRAY INC. - LOS ALTOS - USA 4962 El Camino Real Los Altos, CA - USA Tel: +1 (650) info@kalrayinc.com MPPA, ACCESSCORE and the Kalray logo are trademarks or registered trademarks of Kalray in various countries. All trademarks, service marks, and trade names are the marks of the respective owner(s), and any unauthorized use thereof is strictly prohibited. All terms and prices are indicative and subject to any modification without notice.

Cisco HyperFlex HX220c M4 and HX220c M4 All Flash Nodes

Data Sheet Cisco HyperFlex HX220c M4 and HX220c M4 All Flash Nodes Fast and Flexible Hyperconverged Systems You need systems that can adapt to match the speed of your business. Cisco HyperFlex Systems