HPC Performance in the Cloud: Status and Future Prospects

Transcription

1 HPC Performance in the Cloud: Status and Future Prospects ISC Cloud 2012 Josh Simons, Office of the CTO, VMware 2009 VMware Inc. All rights reserved

2 Cloud Cloud computing is a model for enabling ubiquitous, convenient, on demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction. National Institute of Standards and Technology, U.S. Department of Commerce 2

3 Research Group 1 Research Group m Users IT VMware vcloud Director Public Clouds User Portals Catalogs Security VMware vcloud API Research Cluster 1 Research Cluster n VMware vshield Programmatic Control and Integrations VMware vcenter Server VMware vcenter Server VMware vcenter Server VMware vsphere VMware vsphere VMware vsphere Secure Private Cloud 3

4 Run Any Software Stacks application a operating system a application b operating system b 4

5 Separate workloads Secure multi-tenancy Fault isolation application a operating system a application b operating system b 5

6 Separate workloads Secure multi-tenancy Fault isolation and sometimes Performance application a operating system a application b operating system b 6

7 Use Resources More Efficiently application c operating system a application a application b application a application c operating system a operating system b operating system a operating system b 7

8 Use Resources More Efficiently application c application a application b application a application c operating system a operating system a operating system b operating system a operating system b 8

9 Protect Applications from Hardware Failures Reactive Fault Tolerance: Fail and Recover application a operating system 9

10 Protect Applications from Hardware Failures Reactive Fault Tolerance: Fail and Recover application a operating system Virtualization Hardware 10

11 Protect Applications from Hardware Failures Proactive Fault Tolerance: Move and Continue mpi rank 0 operating system mpi rank 1 operating system mpi rank 2 operating system 11

12 Protect Applications from Hardware Failures Proactive Fault Tolerance: Move and Continue mpi rank 0 operating system mpi rank 1 operating system mpi rank 2 operating system Virtualization Hardware 12

13 Virtualization Performance 13

14 When a Picture Is NOT Worth a Thousand Words application operating system 14

15 Linear Algebra L. Youseff, K. Seymour, H. You, J. Dongarra, and R. Wolski. The impact of paravirtualized memory hierarchy on linear algebra computational kernels and software. In HPDC, pages ACM,

16 Biosequence Analysis: BLAST C. Macdonell and P. Lu, "Pragmatics of Virtual Machines for High-Performance Computing: A Quantitative Study of Basic Overheads, " in Proc. of the High Perf. Computing & Simulation Conf.,

17 Biosequence Analysis: HMMer C. Macdonell and P. Lu, "Pragmatics of Virtual Machines for High-Performance Computing: A Quantitative Study of Basic Overheads, " in Proc. of the High Perf. Computing & Simulation Conf.,

18 Molecular Dynamics: GROMACS C. Macdonell and P. Lu, "Pragmatics of Virtual Machines for High-Performance Computing: A Quantitative Study of Basic Overheads, " in Proc. of the High Perf. Computing & Simulation Conf.,

19 Other Single-process Workloads Also generally within 1 5% of native performance Electronic Design Automation (EDA) Digital Content Creation (DCC) GPGPU About 98% of native performance, based on simple CUDA testing 19

20 vhadoop Benchmarking Study Seven-node Hadoop cluster (AMAX ClusterMax) Standard tests: PI, DFSIO, Teragen / Terasort Configurations: native, one VM per host, two VMs per host Details: two-socket Intel X5650, 96 GB, Mellanox 10 GbE, 12x 7200rpm SATA RHEL 6.1, 6- or 12-vCPU VMs, vmxnet3 Cloudera CDH3U0, replication=2, max 40 map and 10 reduce tasks per host Each physical host considered a rack in Hadoop s topology description ESXi 5.0 w/dev Mellanox driver, disks passed to VMs via RDMs A Benchmarking Case Study of Virtualized Hadoop Performance on VMware vsphere 5, J. Buell 20

21 Ratio to Native Ratio to Native, Lower is Better VM 2 VMs

22 Intel 2009 Experiments Hardware Eight two-socket 2.93GHz X5570 (Nehalem-EP) nodes, 24 GB Dual-ported Mellanox DDR InfiniBand adaptor Mellanox 36-port switch Software vsphere 4.0 (current version is 5.1) Platform Open Cluster Stack (OCS) 5 (native and guest) Intel compilers 11.1 HPCC STAR-CD V _x86 22

23 HPCC Virtual to Native Run-time Ratios (Lower is Better) n16p 4n32p 8n64p Data courtesy of Marco Righini, Intel Italy 23

24 Point-to-point Message Size Distribution: STAR-CD 24 Source:

25 Collective Message Size Distribution: STAR-CD 25 Source:

26 STAR-CD Virtual to Native Run-time Ratios (Lower is Better) STAR-CD A-Class Model (on 8n32p) Physical ESX4 (1 socket) ESX4 (2 socket) Data courtesy of Marco Righini, Intel Italy 26

27 Bandwidth (MB/s) InfiniBand Bandwidth with Passthrough (VM DirectPath I/O) Send: Native Send: ESXi RDMA Read: Native RDMA Read: ESXi K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M 8M Message size (bytes) 27

28 Half roundtrip latency (µs) Latency with VM DirectPath I/O (RDMA Read, Polling) MsgSize (bytes) Native ESXi ExpA Native ESXi ExpA K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M 8M Message size (bytes) 28

29 Half roundtrip latency (µs) Latency with VM DirectPath I/O (Send/Receive, Polling) MsgSize (bytes) Native ESXi ExpA Native ESXi ExpA K 2K 4K 8K 16K 32K 64K 128K 256K 512K 1M 2M 4M 8M Message size (bytes) 29

30 Summary and Conclusions Virtualization can enable new capabilities not available in traditional bare-metal HPC environments Throughput-oriented applications generally run very well in virtual environments with typical slowdowns of 1-5%, depending on I/O characteristics Latency sensitive MPI applications will generally experience noticeable slowdowns, dependent on application messaging profiles. 15% seen for industrial scale CFD code in 2009 Further latency reductions will improve MPI performance over time Both the value of benefits and the performance cost will vary by organization Today s cloud is not tomorrow s cloud we are at the very beginning 30