Interactive HPC: Large Scale In-Situ Visualization Using NVIDIA Index in ALYA MultiPhysics

Transcription

1 Interactive HPC: Large Scale In-Situ Visualization Using NVIDIA Index in ALYA MultiPhysics Christopher Lux (NV), Vishal Mehta (BSC) and Marc Nienhaus (NV) May 8 th 2017

2 Barcelona Supercomputing Center Marenostrum PetaFlop/s General Purpose Computing 3400 nodes of Xeon, 11 PF/s Emerging Technologies Power 9 + Pascal 1.5 PF/s Knights Landing and Knights Hill 0.5 PF/s 64bit ARMv8 0.5 PF/s 2

3 COMPUTER SCIENCES To influence the way machines are built, programmed and used: programming models, performance tools, Big Data, computer architecture, energy efficiency LIFE SCIENCES To understand living organisms by means of theoretical and computational methods (molecular modeling, genomics, proteomics) Research at BSC EARTH SCIENCES To develop and implement global and regional stateof-the-art models for shortterm air quality forecast and long-term climate applications CASE To develop scientific and engineering software to efficiently exploit supercomputing capabilities (biomedical, geophysics, atmospheric, energy, social and economic simulations) 3

4 ALYA System: Large Scale Computational Mechanics 4

5 5

6 ALYA HPC Context 6

7 ALYA HPC Context 7

8 ALYA RED 8

9 Computational Cardiac Model Applications Pacemaker applications Computational analysis of malfunctioning tissue patches Computational Drug testing on cardiac tissue. 9

10 Computational Cardiac Model 10

11 Pacemaker Application 11

12 Pacemaker Application 12

13 Computational Drug Testing 13

14 INTERACTIVE HPC: LARGE SCALE IN-SITU VISUALIZATION USING NVIDIA INDEX IN ALYA MULTIPHYSICS Christopher Lux (NV), Vishal Mehta (BSC) and Marc Nienhaus (NV) May 8 th 2017

15 NVIDIA INDEX Scalable, Interactive Visual Computing GPU-cluster aware solution High-quality and scalable visualization of large-scale datasets In-situ visualization Commercial software Available and deployed in production 15

16 SCALABILITY AS ENABLER NVIDIA HPC Clusters NVIDIA Quadro VCA or DGX-1 NVIDIA Quadro Workstation Performance, dataset size, number of pixels, visual quality 16

17 Scientific Data Visualization 17

18 Time-Varying Data Visualization 18

19 Time-Varying Data Visualization Simulation data source: A Numerical Study of High-Pressure Oxygen/Methane Mixing and Combustion of a Shear Coaxial Injector, Nan Zong & Vigor Yang, AIAA

20 In-Trans and In-Situ Visualization 20

21 Computational Heart 21

22 BACKGROUND 22

23 DISTRIBUTED PARALLEL RENDERING Sort-Last Rendering (multi-gpu) [..] [..] * image compositing 23

24 DISTRIBUTED PARALLEL RENDERING Sort-Last Rendering (Cluster of multi-gpu Nodes) Cluster of VCAs [..] [..] * image compositing 24

25 DISTRIBUTED DATATYPES Various Application Domains Volume datatypes Regular Sparse Unstructured/Irregular Surface-geometry datatypes Height field Triangle mesh 25

26 IN-TRANS AND IN-SITU VISUALIZATION 26

27 TRADITIONAL VISUALIZATION PIPELINE Simulation Cluster Data Storage e.g. Unstructured Data NVIDIA IndeX Visualization Cluster 5/15/2 27

28 TIME-SERIES DATA VISUALIZATION Visualize Pre-calculated ALYA Simulation Results Visualize and Animate Stream Interact and Explore Terabyte time-varying simulation data of nasal system 28

29 IN-SITU (IN-TRANS) VISUALIZATION PIPELINE Unstructured Data Unstructured Data Unstructured Data Simulation Cluster Network NVIDIA IndeX Visualization Cluster 29

30 IN-SITU VISUALIZATION PIPELINE Combined Simulation and Visualization Cluster 5/15/2 30

31 IN-SITU/IN-TRANS SUPPORT Compute Result Integration Parallel jobs executed locally or remotely Direct access to local host and device data Fast RDMA memory transfers User-defined affinity and spatial subdivision Application-driven updates Push updated data when ready Rendering-driven updates Request computation updates for active data Clustered neuron activity 31

32 IN-TRANS RESULT TRANSFERS Fast Data Transfers to Rendering Nodes/GPUs Page-Locked System Memory RDMA (over InfiniBand) Page-Locked System Memory CUDA GPU Memory CUDA GPU Memory GPUDirect RDMA NVLink high-speed interconnect between system memory and GPU (IBM and NVIDIA) 32

33 COMPUTATIONAL HEART In-Situ Simulation and Visualization Simulate/Compute (ALYA) Visualize Parameterize and Steer Interact and Explore 33

34 NVIDIA INDEX AVAILABILITY 34

35 NVIDIA INDEX 1.4 In-Situ/In-Trans Visualization Support Support of 32bit, 16bit, 8bit fixed point, 32bit floating point and RGBA (8bit) regular volumes Dynamic streaming and GPU caching of time-varying volume data NVIDIA IndeX 1.4 (released 07/2016) Irregular volumes and sparse volumes Built-in volume shading capabilities Multi-view capabilities Zero-copy RDMA/GPUDirect compute integration infrastructure User-defined affinity and spatial subdivision support Architecture and API for in-situ/in-trans visualization (compute integration) Dynamic workload balancing Advanced CUDA memory management, error handling and logging MPI/NVIDIA IndeX interprocess coupling (CUDA IPC and shared memory) 35

36 NVIDIA INDEX 1.5 OUTLOOK User-defined Rendering Kernel Components 36

37 NVIDIA INDEX 1.5 OUTLOOK User-defined Rendering Kernel Components 37

38 IN-SITU VISUALIZATIONS 38

39 Challenges Simulation Index Rendering MPI Parallel Operations Maintain frame rates Steering Simulation MPI Unstructured Mesh Uneven Spatial partitions Balanced computations MPI.. MPI Data Affinity In-trans approach MPI.. Cubical Scene region Even spatial partitions MPI Balanced rendering load 39

40 Multi-code Coupling in ALYA All spatial interpolation on spatial domain in structured and unstructured meshes Allows setting send and receive frequencies to synchronize simulation times. Allows coupling with third party codes Parallel and Asynchronous MPI coupling 40

41 Ingredients of Coupling in ALYA mpirun -np 8 Alya.x fluid : -np 4 Test.x : -np 4 Alya.x solidz WHAT The underlying variables WHERE Surface, Volume, etc. WHEN Time step, iteration step HOW Algorithmic interpolation 41

42 Coupling for IN-SITU Visualizations Allows optimizing resources for compute and render Simulation Index Rendering Application Driven Updates, push simulation data. MPI MPI Allows inter-operability between coarser and finer meshes, adjusting data updates. Maintains high frame rates and allows interaction with the volume. Can couple multiple physics apps to a single rendering app. MPI.. MPI MPI.. MPI 42

43 Steering Simulations Steering is Application Specific Coupling Steering simulations requires handling interrupts. Interrupt communicated through backward coupling. Simulation Time S1 Rendering Time S1 Interrupt handler A general approach by scalar/vector interrupts, and user defined function to handle the variables of simulation. Function applied to fields Time S2 Time S3.. Time S2 Time S3.. 43

44 Summary: In-situ Visualization Index enables better insights into simulation data through professional visualization techniques Scalability is the enabler for HPC in-situ visualization. Multiphysics coupling is the key to scalability, and resource management for in-situ. 44

45 SELF-PACED LABS Interactive HPC Volume Visualization in ParaView NVIDIA IndeX for ParaView plugin hands-on Location: Self-paced lab area on lower level Dates: Monday 1:00 5:00pm Tuesday 9: am Wednesday 1:00 5:00pm 45

46 INTERACTIVE DEMO Interactiver HPC: Large Scale In-Situ Visualization using NVIDIA IndeX in ALYA MultiPhysics Live demonstration of in-situ visualization Interactive steering of simulation parameters Location: NVIDIA demo-booth in exhibit hall 1 46

47 Christopher Lux, NVIDIA Vishal Mehta, BSC Marc Nienhaus, NVIDA