SmartNICs: Giving Rise To Smarter Offload at The Edge and In The Data Center

Transcription

1 SmartNICs: Giving Rise To Smarter Offload at The Edge and In The Data Center Jeff Defilippi Senior Product Manager Arm #Arm Tech Symposia

2 The Cloud to Edge Infrastructure Foundation for a World of 1T Intelligent Devices 2018 Arm Limited

3 SmartNIC and Smart Offload Definition Smart Includes application processing capable of running rich operating system (Linux) and distributed cloud stacks. Offload Offload special purpose hw that offloads a specific task (ex: network, storage, security) to maximize efficiency and performance. NIC - class of offload for ethernet controllers. 3

4 Agenda Evolving beyond the SmartNIC, a datacenter story Arm platform for smart offload and accelerated edge compute 4

5 Evolving beyond the SmartNIC, a datacenter story 5

6 Traditional server deployment Server Runs all OS, virtualization and application stacks IO stacks for fixed function off-load Server Application Fixed function off-load NIC Network Interface Controller Storage HDDs, SDDs, etc Ethernet Hypervisor Storage NIC HDD 6

7 Web and mobile traffic drove fundamental shift to IO Global IP traffic hit 1 Zettabyte in 2016 Will increase to >3 Zettabytes by 2021 Server Application Direct impact on server network bandwidth 10Gb Ethernet ->100Gb Ethernet++ Hypervisor Orders of magnitude more storage requests NVMe/SSDs - millions of operations per second Ethernet Storage Impact on Server: Increased CPU utilization for network/storage IO NIC HDD/SDD NVMe 7

8 Storage and memory disaggregation Pull data from anywhere, quickly Over Fabric such as NVMe-oF Software defined network/storage (SDN/SDS) Reconfigurable virtual networks Server Application Hypervisor RDMA fast path to remote memory requests RDMA - Remote Direct Memory Access Ethernet SDN/SDS RDMA Storage Impact on Server: SDN, SDS, RDMA CPU utilization and increased east/west traffic NIC HDD/SDD NVMe 8

9 Rise of cloud computing New business models built on as a service SaaS, IaaS,PaaS, FaaS Required new management and security Virtual machine orchestration, management Virtual machine switching Protecting shared resources from guest applications Server Guest App Guest App Guest App Hypervisor vswitch VM Mgmt Security SDN/SDS RDMA Ethernet Storage Impact on Server: More CPU cycles devoted to VM/container management and security NIC HDD/SDD NVMe 9

10 Significant overhead cost for offload and management $3.5B lost revenue opportunity per 1M servers/yr 16 vcpus (1) consumed for offload tasks Cloud CPU rate = $0.025 (2) per vcpu/hr $220/vCPU/yr Server Guest App Guest App Guest App Hypervisor 1M servers * 16 vcpu * $220/vCPU/year = $3.5B (1) Estimated based on 50GbE NIC with 96 vcpu instance (2) Based on average Amazon AWS M5 and Microsoft Azure Dv3 reserved pricing per core vswitch VM Mgmt SDN/SDS RDMA Ethernet Security Storage NIC HDD/SDD NVMe 10

11 SmartNIC Offload Solution a better approach Add the Smart in NIC Add compute capable of running Linux OS & offload SW stacks But it s more than a NIC Orchestration, VM management, security, storage Run vswitch, SDN/SDS software stacks Guest App Server Guest App Guest App Guest App Guest App Guest App Thin Hypervisor vswitch VM Mgmt Security SDN/SDS RDMA HDDs Impact on Server: Frees all CPUs for applications Network NIC Storage SSDs 11

12 Smart offload becomes the datacenter orchestration point CPU/ Server Smart Offload 100GbE Datacenter Network Datacenter orchestration: Decide where to deploy applications/vms/containers Map/re-map storage to applications Map/re-map datacenter network topologies 12

13 Smart offload as a host for cloud storage CPU/ Server TPU TPU Storage/ TPU NVMe/ Memory Smart Offload 100GbE Smart Offload 100GbE Host for disaggregated storage and memory: Pools of HDDs, SDDs, NVMe and DRAM memory Operates cloud object, file and block storage Datacenter Network Datacenter Network 13

14 Smart offload as a host for cloud acceleration Accelerator platform as a service CPU/ Server TPU TPU Storage/ TPU NVMe/ Memory TPU TPU TPU GPU TPU TPU TPU TPU TPU TPU TPU FPGA Smart Offload Smart Offload Smart Offload Smart Offload Smart Offload 100GbE 100GbE 100GbE 100GbE 100GbE Datacenter Network Datacenter Network Datacenter Network Datacenter Network Datacenter Network 14

15 Challenges of moving compute to smart offload 10x efficiency gain required OCP 3.0 Mezz Card 20W ASIC budget Servers: 100W~200W processors focus on compute perf Large footprint, large heat sink, sophisticated cooling Offload devices: ~20W-35W ASIC budget for mainstream Attached via PCIe slots or Mezz card Smaller footprint, smaller heat sink, much less sophisticated cooling NIC4.0-v add-note.pdf 15

16 The path to 10x efficiency gain Arm = better Perf/W Network Custom Acceleration Storage Security Move functions from server CPUs to more efficient accelerators PCIe CCIX Arm RDMA Packet Inspection De-duplication Compression Crypto IPSec Customize with secret sauce 100GbE DDR TCP/IP offload NVMe-OF Offload Secure Enclave 16

17 From smart offload in the datacenter to accelerated edge compute 17

18 4G infrastructure latency is ok for today s mobile broadband Cloud Data Centers Media Content Billions of Phones Create & Distribute Consume 18

19 Data consumption and latency driving future designs Cloud Data Centers Critical Data Edge Edge 5G Edge Massive Amounts of Data Trillions of Devices Analyze & Store Filter & React Local Decisions 19

20 Accelerated edge compute example deployment Cloud server (100W~200W) Edge server gateway (<100W) Edge Wireless Access Power over Ethernet (<30W) Cloud Data Centers Core Network Local Edge Network 20

21 Accelerated edge compute example systems Edge server gateway <100W Wireless Access Power over Ethernet (<30W) Edge arm PCIe CCIX Smart Offload 100GbE Local Edge Network 10GbE arm 5G Radio ML Crypto DDR ML Cache Video DDR Imagine 21

22 Arm compute and IO subsystem Arm CPU A72, A75, Ares CPUs A55, Helios CPUs GIC/MMU Virtualization A72, A75, Ares, A55, Helios Arm SoC Backplane Memory IO CMN Coherent Mesh Network Power Control Kit DDR4 DDR5 PCIe CCIX GIC, MMU Virtualization CryptoIsland Secure Enclave HBM GbE CryptoIsland Secure Enclave Power Control Kit DDR CMN PCIe/CCIX Common Software Platform, SBSA, SBBR, Arm ServerReady Arm Architecture v8.x-a, AMBA 22

23 Accelerator integration Arm CPU A72, A75, Ares CPUs Accelerators Arm SoC Backplane Memory A55, Helios CPUs CMN Coherent Mesh Network Power Control Kit DDR4 DDR5 GIC, MMU Virtualization CryptoIsland Secure Enclave HBM RDMA Packet Inspection TCP/IP offload De-dupe Compression NVMe-OF Offload Bulk Crypto IPSec GPU ML Processor A/V transcode 4G/5G Wireless FPGA IO PCIe CCIX GbE Common Software Platform, SBSA, SBBR, Arm ServerReady Arm Architecture v8.x-a, AMBA 23

24 Smart offload requires scalability SoC Multi chip module Disaggregated Platform Scalability from fully integrated SoC to multichip modules to fully disaggregated Edge HW Platform Arm Compute Subsystem IO Accelerators Memory 24

25 Accelerated edge cloud stack Test suites to ensure major OS vendor compatibility SBSA: Server Base System Architecture SBBR: Server Base Boot Requirements Device Drivers Arm Trusted Firmware Linux OS Firmware Device Management Foundation for cloud deployments Edge HW Platform Arm Compute Subsystem IO Accelerators Memory 25

26 Accelerated edge cloud stack Infra Orchestration and Installer Apps Apps Apps Bare metal Container VM vswitch SDN Controller Orchestration and installation Automated deployment, discovery, management Edge stack blueprints Network management, NFV Data Plane SR-IOV VirtIO Vertical deployments (ex: IIoT) Device Drivers Linux OS Smart City and edge cloud Arm Trusted Firmware Firmware Device Management Edge HW Platform Arm Compute Subsystem IO Accelerators Memory 26

27 Arm powering smart offload and the accelerated edge Scalable, secure, virtualized compute subsystems Combine with custom acceleration for target offload functions Ready for scale-out deployments in the datacenter and at the accelerated edge 27

28 Trademark and copyright statement The trademarks featured in this presentation are registered and/or unregistered trademarks of Arm Limited in the EU and/or elsewhere. All rights reserved. All other marks featured may be trademarks of their respective owners. Thank You! Copyright