Big Data in HPC John Shalf Lawrence Berkeley National Laboratory 1
Evolving Role of Supercomputing Centers Traditional Pillars of science Theory: mathematical models of nature Experiment: empirical data about nature Scientific computing connects theory to experiment Computational models are used to test theories involving complex phenomena that cannot be matched directly against experiments Enable comprehension of complex experimental data by distilling complex data down to understandable information There is no scientific discovery without a theory about underlying mechanism There is no scientific discovery without experimental validation of model Theory Simulation Mechanism Experiment Mathematical Models of Nature Evidence Confirm/Violate Analysis Empirical Data about Nature
Breakthrough science will occur at the interface of observation and simulation Environment and Climate Cosmology Materials and Chemistry Combustion Subsurface science New research challenges to make these work together -3-
Data Intensive Arch for 2017 (as imagined in 2012) Compute On-Package DRAM Capacity Memory On-node-Storage In-Rack Storage Interconnect Global Shared Disk Off-System Network Compute Intensive Arch n.a. 5TF/socket 1-2 sockets 64-128GB HMC or Stack - 4-1TB/s 200TB/s.5-1TB CDIMM (opt.) Organized for Burst Buffer Spatially-oriented e.g. 3D-5D Torus ~1% nodes for ~1% nodes for IP Gateways 100TB/s 50GB/s/node 10TB/s/rack 10TB/s bisect 0.5TB/s aggregate 4GB/s per node Data Intensive Arch /! (root)! Dataset0! time =0.2345! validity =None! Dataset1! type, space! author =JoeBlow! Dataset0.1! subgrp! Dataset0.2! + + 5TF/sock 8+ sockets 1-4 TB Aggregate Chained-HMC 5-10TB Memory Class NVRAM 10-100TB SSD Cache or local FS 1-10PB Dist.Obj. DB (e.g whamcloud) All-to-All oriented e.g. Dragonfly or 3T ~10-20% nodes for 40GBe Ethernet to Direct from each node
Data Intensive Arch for 2017 (as imagined in 2012) Compute On-Package DRAM Capacity Memory On-node-Storage In-Rack Storage Interconnect Global Shared Disk Off-System Network Compute Intensive Arch 5TF/socket 1-2 sockets 64-128GB HMC or Stack - 5-1TB/s 200TB/s.5-1TB CDIMM (opt.) Goal: Maximum Computational Density and local n.a. bandwidth for given power/cost constraint. Organized for Burst Buffer Maximizes bandwidth density near compute Spatially-oriented e.g. 3D-5D Torus ~1% nodes for ~1% nodes for IP Gateways 100TB/s 50GB/s/node 10TB/s/rack 10TB/s bisect 0.5TB/s aggregate 4GB/s per node Data Intensive Arch 5TF/sock + + 8+ sockets Goal: Maximum Data Capacity and 1-4 TB global Aggregate Chained-HMC bandwidth for given power/cost 5-10TB constraint. Memory Class NVRAM Bring more 10-100TB storage SSD capacity near Cache compute or local (or FS conversely embed more /! author =JoeBlow! (root)! compute into the storage). Dataset0! Dataset1! subgrp! type, space! time =0.2345! validity =None! Dataset0.1! Dataset0.2! 1-10PB Dist.Obj. DB (e.g whamcloud) All-to-All oriented e.g. Dragonfly or 3T Requires software and programming environment support for such a paradigm shift ~10-20% nodes for 40GBe Ethernet to Direct from each node
Old/New Conception of Cloud/Datacenters (Simplified Conceptual Model of Interconnect Convergence) Old Conception Designed for externally facing TCP/IP Nearly 100% Std. TCP/IP ethernet inside and out The Datacenter/Cloud DMZ Router 90+% traffic Backbone 6
Old/New Conception of Cloud/Datacenters (Simplified Conceptual Model of Interconnect Convergence) New Old Conception Need Designed to Handle for Internal externally Data facing Mining/Processing TCP/IP Nearly 100% Design Std. for TCP/IP 80+% ethernet internal traffic inside and out Cloud/Datacenter The Datacenter/Cloud 80+% of traffic DMZ Router 90+% traffic Backbone High-Performance Fluid Center Low Overhead, High Bandwidth, Semi-custom internal interconnect Crunchy TCP/IP Exterior 7