Interactively Visualizing Science at Scale

Transcription

1 Interactively Visualizing Science at Scale Kelly Gaither Director of Visualization/Senior Research Scientist Texas Advanced Computing Center November 13, 2012

2 Issues and Concerns Maximizing Scientific Impact Managing Data at Scale Providing Resources at Scale Ensuring Broad Accessibility/Developing Ubiquitous Tools

3 Maximizing Scientific Impact Image: Greg P. Johnson, Romy Schneider, TACC Image: Adam Kubach, Karla Vega, Clint Dawson Image: Karla Vega, Shaolie Hossain, Thomas J.R., Hughes Greg Abram, Carsten Burstedde, Georg Stadler, Lucas C. Wilcox, James R. Martin, Tobin Isaac, Tan Bui-Thanh,and Omar Ghattas

4 Not Just Simulation Any More Vastly more powerful instruments and computers have led to an explosion of new data. Modern science and engineering therefore is about managing and analyzing this data as well as modeling and simulation.

5 Visualization of Large Scale Turbulent Flow Kelly Gaither, Hank Childs, Greg Johnson, Karl Schulz, Cyrus Harrison, Diego Donzis, Texas A&M; P.K. Yeung, Georgia Tech Remote interactive visualization of 17 time-steps (34 TB) of the largest turbulent flow simulation computed to date ( ). First time this had been visualized interactively at this scale. Equal parts data mining and remote interactive visualization goal was to characterize flow behavior over time. Gaither, K., Childs, H., Schulz, K., Harrison, C., Barth, W., Donzis, D., and Yeung, P.K., Using Visualization and Data Analysis to Understand Critical Structures in Massive Time Varying Turbulent Flow Simulations, IEEE Computer Graphics and Applications, 32(4), Jul/Aug 2012.

6 Stellar Magnetism Greg Foss, TACC; Ben Brown, University of Wisconsin, Madison A Sun-like star undergoes magnetic cyclic reversal shown by field lines. Shifts in positive and negative polarity demonstrate largescale polarity changes in the star. Wreath-like areas in the magnetic field may be the source of Sun spots. Terabytes of data to mine through and visualize.

7 TACC Visualization Pipelines Post-processing User generates a data set either through simulation or through measurement and saves the data set for later post-processing analysis and visualization. Reasonable model when interactive query is of primary interest and the data set size is manageable. In Situ Data sets are growing at a staggering rate and the traditional model of post-processing visualization becomes too costly to manage, particularly in those instances in which time to insight is of value. Requires instrumentation of the simulation code and hooks in the visualization software. We have facilitated this using VisIt and ParaView.

8 Managing Data at Scale HPC System Large-Scale Visualization Resource Pixels Mouse Display Data Archive Remote Site Wide-Area Network Local Site

9 Longhorn First NSF XD Visualization Resource 256 Nodes, 2048 Cores, 512 GPUs, 14.5 TB Memory 256 Dell Dual Socket, Quad Core Intel Nehalem Nodes 240 with 48 GB shared memory/node (6 GB/core) 16 with 144 GB shared memory/node (18 GB/core) 73 GB Local Disk 2 Nvidia GPUs/Node (FX GB RAM) ~13.5 TB aggregate memory QDR InfiniBand Interconnect Jobs launched through SGE ~6GB/s to scratch filesystem ~6GB/s to Ranger filesystem Kelly Gaither (PI), Valerio Pascucci, Chuck Hansen, David Ebert, John Clyne (Co-PI), Hank Childs

10 Supporting Visualization on Stampede Leverage 128 Kepler GPUs for interactive remote visualization using VNC and VirtualGL. Working with Intel graphics group to facilitate remote interactive visualization: Porting OpenGL to MIC Real time raytracing

11 Visualization Usage Modalities: Remote/Interactive Visualization Highest priority jobs Remote/Interactive capabilities facilitated through VNC Run on 3 hour queue limit boundary GPGPU jobs Run on a lower priority than the remote/interactive jobs Run on a 12 hour queue limit boundary CPU jobs with higher memory requirements Run on lowest priority when neither remote/interactive nor GPGPU jobs are waiting in the queue Run on a 12 hour queue limit boundary

12 Queue Structure Example: qsub -q normal -P vis

13 Visualization Portal portal.longhorn.tacc.utexas.edu Developed to provide easy access to remote visualization systems and abstract away complexities involved with command line access Leverages XSEDE user portal codebase and employs a fraction of XUP developers to ensure continuity Used for all in-person remote visualization training

14 Visualization Portal portal.longhorn.tacc.utexas.edu >5000 jobs submitted through the portal

15 Visualization Portal portal.longhorn.tacc.utexas.edu Specify type of session Specify resolution of vnc session Specify number of nodes needed and the wayness of the nodes Provides graphic of machine load

16 Visualization Software on All TACC Systems Programming APIs: OpenGL, vtk (Not natively parallel) OpenGL low level primitives, useful for programming at a relatively low level with respect to graphics VTK (Visualization Toolkit) open source software system for 3D computer graphics, image processing, and visualization IDL Visualization Turnkey Systems VisIt free open source parallel visualization and graphical analysis tool ParaView free open source general purpose parallel visualization system VAPOR free flow visualization package developed out of NCAR EnSight commercial turnkey parallel visualization package targeted at CFD visualization Amira commercial turnkey visualization package targeted at visualizing scanned medical data (CAT scan, MRI, etc..)

17 Thoughts Towards Exascale: Data will get larger and more unwieldy we will stop moving it around High performance computing environments will become high performance science environments that provide computing and analytics Rendering will continue to get less and less expensive. We will see a real blend in high performance environments of physical modeling and computer graphics.

18 Thank You Kelly Gaither

19 Thoughts Towards Exascale: Data will get larger and more unwieldy we will stop moving it around High performance computing environments will become high performance science environments that provide computing and analytics Rendering will continue to get less and less expensive. We will see a real blend in high performance environments of physical modeling and computer graphics.