CHALLENGES FOR NEXT GENERATION FOG AND IOT COMPUTING: PERFORMANCE OF VIRTUAL MACHINES ON EDGE DEVICES WEDNESDAY SEPTEMBER 14 TH, 2016 MARK DOUGLAS

Transcription

1 CHALLENGES FOR NEXT GENERATION FOG AND IOT COMPUTING: PERFORMANCE OF VIRTUAL MACHINES ON EDGE DEVICES WEDNESDAY SEPTEMBER 14 TH, 2016 MARK DOUGLAS

2 Agenda Challenges to Intelligent Edge devices what s driving an intelligent edge what the current devices do not have enter intelligent edge platform Fog or Intelligent Edge Features isolation of services performance of these consolidated features performance optimized I/O Proposed New Architecture for Intelligent Edge why virtualization? feature partitioning Single VM performance test setup iperf over time progressive improvements I/O via virtio (Preliminary Data) Iperf and FIO performance on 4VMs based on Linux v4.1.8 Future investigations highlighted

3 Current Needs at the Edge What s driving an intelligent edge? increasing bandwidth requirements at the edge The endless struggle to get as much content as possible as quickly as possible available instant content delivery 3 rd party applications providing isolated operations we are seeing a consolidation at the edge combining 3 separate boxes or features in one SoC

4 Fog computing and Intelligent edge features: The Intelligent Edge Evolution: feature isolation combining seemly disjoint features normally in separate appliances current edge devices don t have HW/SW architectures to support this. if isolated, performance in their respective sandboxes is essential VMs must be optimized for performance performance optimized I/O direct mapped Network interfaces virtionet is an option, yet shared these are exclusively owned by the guest file I/O also is possibly needed virtiodataplane used compute performance too

5 Proposed New Architecture What s different? currently appliances are single function: example => routers and switches single mission software and single mission device Partitioning is needed proper isolation between combined features Physical resources (I/O) will be either shared or dedicated to each VM Slower resources can be shared high performance can be dedicated VMs are used for isolation physical to virtual CPU mappings SMP guests can be used application application application application operating system operating system operating system VM (kvm) VM (kvm) VM (kvm) Linux ARMv8 ARMv8 ARMv8.. ARMv8 acceleration

6 Looking at the Test Setup What s tested and why? using iperf for bandwidth and overhead iperf/snort Iozone and FIO later LS2085ARDB LS2085ARDB Iperf client 10GXFI 10GXFI Iperf server LS2085ARDB LS2085ARDB Iperf client 10GXFI 10GXFI Iperf server KVM Virtual Machine

7 LS2085A Iperf absolute benchmark (Mb/Sec) Iperf absolute benchmark data (LS2085A) Blue is Bare Metal Linux Red is VM Guest Linux Units are Megabits/Second Polling 3.19 virtio_ net windo w size 45K x86 Polling 4.0 virtio_ net windo wn size 45K x86 Polling 3.19 polling 1 Networ k I/O windo w size 45K x86 Polling 3.19 polling 1 Networ k I/O windo w size 45K LS2 Polling 4.0 polling 1 Networ k I/O windo w size 45K LS2 Polling 4.0 polling 1 Networ k I/O windo w size 45K x86 Polling 4.0 polling 1 Networ k I/O windo w size 90K x86 Interru pt: Initial result Networ k I/O windo w size 90K ls2 Interru pt: optimiz ation patch 4.0, 90K, ls2 Interru pt: optimiz ation patch fsl_qb man: Add interru pt coalesc ing suppor t 75us > Interru pt: optimiz ation patch 75us > irq_coa lescing 4.0, 90K, ls2 Interru pt optimiz ation patch 50us > irq_coa lescing 4.0, 90K, ls2 Interru pt optimiz ation patch 25us > irq_coa lescing 4.0, 90K, ls2 optimiz ation patch irq_coa lescing: 15us; client host 15us, client guest 25us 4.0, 90K, optimiz ation patch irq_coa lescing: 15us; client host 15us, client guest 25us 64KB page size, Bare Metal Linux VM Guest Linux , 90K, ls2 Project ed once Eratta is fixed

8 Iperf data v4.1.8 Linux LTS 750 flows (iperf c P 750 t 600 w 90K) Data Projection (based on errata ERR fix) 750 flows (iperf c P 750 t 600 w 90K) DPAA 2.x Direct Assignment (v4.0.x Vs.4.1.8) 1 Network Interface 1CPU Server throughput (Mbps) throughput (Mbps) Bare metal (Linux host only) kvm LS2 Kvm/host % (8CPU) ~ 91% ~91% This single core VM Vs. single core native represents a significant milestone in: a) Maximizing the dedicated network interface throughput and b) Reducing the VM overhead incurred to < 10% of the host!

9 SSD performance (virtio) (IZone) Device: SATA3.0 link up 6.0 Gbps, the capacity is 256G Command: iozone Rab /mnt/result.xls i 0 i 1 f /mnt/testfile n 8G g 16G r 64 Note: Guest virtioblk(data plane) > Guest virtioblkdataplane Guest virtioblk(pci) > Guest virtioblk Ls2 linux.git B: G SSD 8G DDR on host and guest side IOZONE(maximum file size = 2 * DDR size) 1CPU and 1VCPU read (Mbytes/sec) write (Mbytes/sec) 1GB hugepage 2MB hugepage 1GB hugepage 2MB hugepage host Guest virtioblkdata plane Guest/host (%) 79.11% 79.87% 87.57% 87.59% Guest virtioblk Guest/host(%) 78.38% 78.13% 86.01% 87.38%

10 Iperf summary performance on 4VMs based on kernel V4.1.8

11 4VMs based on kernel V4.1.8 LS2085ARDB LS2085ARDB Iperf client KVM 1/Cluster 2 10GXFI 10G XFI Iperf client KVM 2/Cluster 3 10GXFI 10G XFI Iperf server Iperf client KVM 3/Cluster 4 10GXFI 10G XFI FIO SATA KVM 4/Cluster 1

12 Resource interrupts assigned in each VM Core# 4 Packet I/O #4 Iperf client vcpu#1 Core# 4 Core# 5 Packet I/O #5 vcpu#2 Core# 5 KVM 1/Cluster 3 DPNI#1 Core# 4&5

13 LS2080A MultiVM Iperf data v4.1.x Linux LTS Iperf result with 4VMs based on kernel V4.1 Host: 2 CPUs 8DPIO 1 iperf instance 2G memory (64K page size and 48bit VA, 25us coalescing time) Guest: 4 VMs, each VM has 2 cores, 2 DPIO and 1 DPNI, 2G memory (64K page size and 48bit VA,25us coalescing time) Server:LS2085ARDB 8Cores+8DPIO+3DPNI (4K page size and 48bit VA,15us coalescing time) 3xVM iperf +1xVM FIO 3xVM iperf + no FIO 2xVM iperf +2xVM FIO 2xVM iperf + no FIO iperf in VM iperf in VM Guest Mb/Sec iperf in VM This multivm data shows: We see data scaling across all the core clusters. Measuring: Mega Bits / Second (Mb/Sec)

14 Latest LS2088A MultiVM Iperf data v4.1.x Linux LTS Iperf result with 4VMs based on kernel V4.1 Host: 2 CPUs 8DPIO 1 iperf instance 2G memory (64K page size and 48bit VA, 15us coalescing time) Guest: 4 VMs, each VM has 2 cores, 2 DPIO and 1 DPNI, 2G memory (64K page size and 48bit VA,25us coalescing time) Server:LS2088ARDB 8Cores+8DPIO+3DPNI (4K page size and 48bit VA,15us coalescing time) 1xVM iperf 2xVM iperf 3xVM iperf iperf in VM iperf in VM 2 N/A Guest Mb/Sec iperf in VM 3 N/A N/A 7304 This multivm data shows: We see data scaling across all the core clusters. Measuring: Mega Bits / Second (Mb/Sec)

15 Implications and Configurations Down at the raw VM constructed level Virtual CPU (vcpu) to Physical CPU (pcpu) mapping key: matching of virtual to physical CPUs Avoid L2 cache pollution affixing QEMU to a cluster (two pcpus / CPU cluster) QEMU threads can be partitioned across available cores VM threads assigned to cores in the cluster IRQ assignment Clever IRQ assignment is key to offloading the pcores number of pcpus per VM Powerpack your vcpus with pcpus

16 Conclusions and Future Work Conclusions Fundamentally the overhead of the guest is 10%15% range Virtionet showing low overhead MultiVM performance scales Future Work More variations with multivms Varying the IRQs, vcpus and pcpu partitioning InterVM bandwidth and latency testing for vnfs Add more and different I/O to each VM utilize virtionet with and without hardware offload