HPC 2 Informed by Industry

Transcription

1 HPC 2 Informed by Industry HPC User Forum October 2011 Merle Giles Private Sector Program & Economic Development mgiles@ncsa.illinois.edu National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

2 CSL Digital Computer Lab Basic & Applied Research College of Engineering NCSA ECE Computer Science Imaginations unbound Beckman Institute Institute for Genomic Biology Biotechnology Center

3 NCSA Bridges Basic Research and Commercialization with Application Over the horizon development Phase 0 Concept/ Vision Phase 1 Feasibility Phase 2 Design/ Development Phase 3 Prototyping Phase 4 Production/ Deployment Product Life Cycle Theoretical & Basic Research Applied Prototyping & Development NCSA Bridges the Gap BETWEEN Basic Research & Commercialization Optimization & Robustification Commercialization & Production (.com or.org) Universities & Labs Application Private Industry

4

5 2011 Private Sector Partners

6 Discovery is No Longer Sufficient FIDELITY today TIME Industrial Design Cycle tomorrow s HPC Challenge Source: Rolls-Royce Shyam Neerarambam COMPLEXITY

7 PLM Demands a Data Cloud Bills Of Materials CAD/CAE Information Platform Product Specification Product Design Technical Regs and Standards Product Modeling Production Planning Manufacturing Assembly Quality Assurance Shipping PLM Process Platform Supply Chain Mgmt Cost Accounting data integration

8 DARPA Open Manufacturing 1. Introduce advanced MSA on factory floor 2. Develop data feedback loop for as-builts 3. Maintain data ecosystem for multiple users 4. Support data inbound, outbound, shared analytics 5. Provide optimization environment (UQ) 6. Reduce uncertainty AND increase fidelity 7. Achieve plug-and-play: mutliphysics, PLM, etc. 8. Hand-off from SME to OEM

9 NDEMC 1. U.S. public-private partnership 2. Midwest pilot - manufacturing supply chain 3. Multi-agency attention led by Dept. of Commerce 4. WH, OSTP, DOC, EDA, NSF, NASA, DOE, NIST 5. Investors: P&G, GE, Lockheed Martin, Deere 6. Supply chain firms: SMEs 500 employees 7. Providers: NCSA, OSC, NCMS, Purdue 8. Parallel efforts with Intel s Alliance 9. Objective increase SME adoption of MS&A

10 Purpose-built HPC National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

11 iforge NCSA s first supercomputer built for and dedicated to industry 25 teraflops of x86 processing power Optimized for widely-used FEA and CFD applications No academic users; 100% industrial user queue Strategic HPC initiative, not just a machine Ideal for engineering applications HPC > 64 cores Built in collaboration with Dell and other leading vendors Opportunity for both NCSA and Dell to learn from each other and better serve industry HPC users Vendor partners also include AMD, Intel, Mellanox, Adaptive, DDN

12 Heterogeneous x86 Configuration Platform 1: Fastest dual-socket Intel Xeon CPUs for computebound workloads Platform 2: Quad-socket Intel Xeons CPUs for cache memoryintensive workloads Platform 3: Highest core-count and fastest AMD Opteron CPUs for SMP, AMD-optimized, and atypical HPC workloads Attached to GPU-based Forge sister system 40 Gb QDR Infiniband fabric; high-throughput network storage at 15 GB/sec

13 NCSA Forge/iForge

14 Platform x Dell M610 blades 2 x Intel Xeon x core CPUs 3.46 GHz (3.73 GHz Turbo) 96GB DDR MHz RAM 292 GB HDD or 200 GB SSD Fastest 6-core Intel CPU Comprises ~75% of iforge Ideal for workloads dependent on raw compute, clock speed ~2-5X faster than NCSA Abe across same number of cores

15 Platform 2 3 x Dell M910 servers 4 x Intel Xeon x core CPUs 1.83 GHz (2.53 GHz Turbo) 192GB DDR MHz RAM 292 GB HDD Highest L3 cache (24 MB) for Intel 8-core CPUs Comprises ~5% of iforge Ideal for cache memory-dependent workloads and Intel-specific large memory codes

16 Platform 3 3 x Dell C6145 servers; 2 nodes in each 2U chassis 96 cores AMD Magny-Cours at 2.50 GHz; 256 cores AMD Interlagos at Ghz Up to 64 cores and 256GB DDR MHz RAM per node (!!) Fastest AMD 12 and 16-core CPUs Comprises ~20% of iforge Ideal for large SMP workloads, and AMD-optimized codes Cutting-edge AMD Bulldozer architecture

17 Forge 36 Dell C6145 & C410x servers Features NVIDIA M2070 Tesla GPUs and AMD Opteron CPUs 2 CPUs for every 1 GPU ~153 TF of peak, double-precision performance Replaces NCSA Lincoln Standard NCSA shared queue 10% usage by NCSA industrial partners Ideal for visualization, GPU code development and testing

18 Interconnect & Network Storage Mellanox QDR Infiniband 40 GB/sec 4x faster than NCSA ABE s IB Shared fabric across all iforge platforms and GPU-based Forge DataDirect Networks system ~700 TB storage pool 15 GB/sec throughput Shared storage across all iforge platforms and GPU-based Forge

19 iforge Summary Platform 1 Platform 2 Platform 3 Total Cores CPU Manufacturer Intel Intel AMD CPU Type Xeon x Xeon L7555 Opteron 6180SE 2, Cores/Node / 64 CPU Frequency GHz 1.86 GHz-2.53 GHz 2.5 GHz / GHz CPU L3 Cache 12 MB 24 MB 12 MB / 16 MB Memory Size 96 GB 192 GB 256 GB =8GB/core =6GB/core =5.3/4.0GB/core Storage Type 108 HDD, 8 SSD HDD HDD Storage Size 292 GB (HDD) 200 GB (SSD) 292 GB 600 GB Storage Size File system Disk Throughput Interconnect Operating Systems Network Storage and OS Environments 700 TB IBM GPFS 15 GB/sec Mellanox QDR Infiniband at 40GB/sec RHEL 5/6, Windows HPC Server 2008 R2 SP2 1 Fastest 6-core Intel processor 2 Fastest 12-core AMD processor 3 New 16-core AMD microarchitecture

20 Performance Breakthroughs Industry Standard & Real World Benchmarks Credits: Dr. and Adjunct Professor Seid Koric Dr. Ahmed Taha Ansys Inc. Simulia University of Manchester National Center for Supercomputing Applications University of Illinois at Urbana-Champaign

21 ABAQUS (Implicit FEA) Heavy Benchmark Complex Physics "Torture 96 cores (Lower = Better) 4,800 6, ,740 iforge XSEDE TOP200 System Abe Time (seconds) Heavy is an NCSA partners single most complex ABAQUS simulation; much tougher than S4B standard benchmark; material, geometrical and contact nonlinearities combined Brings the best HPC system to its knees; I/O, communication and memory bandwidth intensive iforge is 75% faster than a TOP200 system used by XSEDE!! iforge is one of the very fastest systems for this partner in the world

22 Scalability Breakthroughs h"p://parafem.org.uk/ ParaFEM The open source FE library

23 Superior scaling at > 1,000 cores Parallel Efficiency for Equations Solution; 12 M dofs benchmark (Higher = Better) 84% scaling efficiency at 1,032 cores

24 Scalability Study Distributed ANSYS Manufacturing Partner B 512-core license for performance, scalability testing on real world engineering cases

25 Stunning Breakthroughs: iforge NCSA Partner A: Has always used ANSYS Mechanical (FEA) shared memory solver NCSA has tested Distributed Ansys on iforge for partner A ~160 hours to completion à 2.6 hours to completion!! 62x speedup!! NCSA Partner B: Partner B s most extreme power user (he breaks everything!) Uses ANSYS Fluent (CFD) to simulate complex gear rotations Has run on x86, POWER5+, POWER7 ~14-21 days to completion à ~6-8 hours to completion!! ~20-80x speedup!!

26 Stunning Breakthroughs: SGI UV U.S. National Cancer Institute Large-scale shared memory xgb per core required Scientific insight through pre-processing Camera technology is outpacing HPC usage (70k x 70k) Similarities to end-to-end engineering challenges Speedup metrics are irrelevant Production environment requires advanced: Workflow, Optimization, Insight, HPC, Data storage

27 Computational Advantage Ecosystem Petascale, ISV code, data management, architecture, etc. DOMAIN Materials, fluids, thermal, bio-xyz, genomics, manufacturing, etc. HPC Application/ Demonstration/ Research Computer Science Shared strategy, bilateral services, partnerships, etc. Consulting Parallelization, code bottlenecks, multiphysics, algorithms, data, etc.

28 HPC 2 Ecosystem Productive HPC is not limited to petascale Capability AND Capacity accelerates: Discovery Time-to-Solution ROI End-to-End solutions need bridges: Personnel Hardware & Software & Data Research Development Application Production Pre-processing Numerical Analysis Post-processing

29 National Petascale Computing Facility $72.5M 25MW 88,000 ft 2 National Petascale Computing Facility

30 Thank You!! HPC User Forum October 2011 Merle Giles Private Sector Program & Economic Development National Center for Supercomputing Applications University of Illinois at Urbana-Champaign