Large Scale Remote Interactive Visualization

Transcription

1 Large Scale Remote Interactive Visualization Kelly Gaither Director of Visualization Senior Research Scientist Texas Advanced Computing Center The University of Texas at Austin March 1, 2012

2 Visualization at TACC 10 Years in the Making Bioinformatics Orbital Debris Turbulent Flow CT Models Gravity Map Quantum Chemistry GeoSciences Natural Convection

3 TACC Visualization Group Provides resources/services to a growing local and national user community to enable scientific discovery and insight. Researches and develops tools/techniques for the next generation of problems facing the user community. Trains the next generation of scientists to visually analyze datasets of all sizes.

4 Visualization Group at TACC Areas of Expertise Scientific & Information Visualization Data Mining & Feature Detection Large Scale GPU Clusters Large Scale Tiled Displays Remote & Collaborative Visualization Tools

5 Spur Remote, Interactive Visualization System Directly Connected to Ranger 128 cores, 1 TB aggregate memory, 32 GPUs spur.tacc.utexas.edu 1 fat memory node Sun Fire X4600 server 8 AMD Opteron dual-core 3 GHz 256 GB memory 4 NVIDIA FX5600 GPUs 7 other nodes Sun Fire X4440 server 4 AMD Opteron quad-core 2.3 GHz 128 GB memory 4 NVIDIA FX5600 GPUs

6 Longhorn First NSF extreme Digital (XD) Visualization Resource 256 Nodes, 2048 Cores, 512 GPUs, 14.5 TB Memory 256 Dell Dual Socket, Quad Core Intel Nehalem Nodes 240 with 48 GB shared memory/node (6 GB/core) 16 with 144 GB shared memory/node (18 GB/core) 73 GB Local Disk 2 Nvidia GPUs/Node (FX GB RAM) ~14.5 TB aggregate memory QDR InfiniBand Interconnect Direct Connection to Ranger s Lustre Parallel File System 10G Connection to 210 TB Local Lustre Parallel File System Jobs launched through SGE Kelly Gaither (PI), Valerio Pascucci, Chuck Hansen, David Ebert, John Clyne (Co-PI), Hank Childs

7 Hadoop on Longhorn PI: Weijia Xu (UT LIFT Grant) Local Storage Expansion GB 7.2k drives are installed on 48 R610 nodes on Longhorn (96 usable TB) GB 15k drives are installed on 16 R710 nodes on Longhorn. (16 usable TB) /hadoop file system Went in to production December 2010 Projects include text mining and information retrieval

8 Longhorn Usage Modalities: Remote/Interactive Visualization Highest priority jobs Remote/Interactive capabilities facilitated through VNC Run on 3 hour queue limit boundary GPGPU jobs Run on a lower priority than the remote/interactive jobs Run on a 12 hour queue limit boundary CPU jobs with higher memory requirements Run on lowest priority when neither remote/interactive nor GPGPU jobs are waiting in the queue Run on a 12 hour queue limit boundary

9 Longhorn Queue Structure Example: qsub -q normal -P vis

10 Software Available on Longhorn Programming APIs: OpenGL, vtk (Not natively parallel) OpenGL low level primitives, useful for programming at a relatively low level with respect to graphics VTK (Visualization Toolkit) open source software system for 3D computer graphics, image processing, and visualization IDL Visualization Turnkey Systems VisIt free open source parallel visualization and graphical analysis tool ParaView free open source general purpose parallel visualization system VAPOR free flow visualization package developed out of NCAR EnSight commercial turnkey parallel visualization package targeted at CFD visualization Amira commercial turnkey visualization package targeted at visualizing scanned medical data (CAT scan, MRI, etc..)

11 Connecting to Longhorn/Spur Using VNC laptop or workstation ssh longhorn qsub /share/sge/default/pe_scripts/job.vnc touch ~/vncserver.out tail f ~/vncserver.out contains vnc port info after job launches ssh L <port>:longhorn.tacc.utexas.edu:<port> <user>@longhorn.tacc.utexas.edu laptop or longhorn workstation establishes secure tunnel to longhorn vnc port laptop or workstation vncviewer localhost::<port> localhost connection forwarded to longhorn via ssh tunnel longhorn automatic port forwarding to vis node VNC server on vis node ivis[1-7 big]

12 Longhorn Visualization Portal portal.longhorn.tacc.utexas.edu

13 Some Lessons Learned Over the Past 10 Years Close collaborations with the science partners are key User support Minimize data transfers if possible Data stays in single location Scale resources effectively based on use cases Easy accessibility to and interaction with technologies encourages diverse communities participation

14 Large Scale Visualization in HPC Issues and Trends Erik Engquist DAVinCI Visualization Manager Rice University Rice Oil & Gas High Performance Computing Workshop March 1, 2012

15 What is a Visualization Facility? Visualization rooms Also known as: vislabs, decision spaces, visualization theaters, collaboration rooms,... Visualization clusters We will also accept: gpu clusters, render farms, remote graphics clusters,...

16 Visualization Rooms Specialized, common features include: 3D stereo display Multi-wall, floor, ceiling Specialized interaction devices, tracking Multipurpose, common uses include: Remote collaboration and data sharing Data exploration Presentations VR-KUB

17 Specialized Visualization Rooms Space Requirements Multi-floor buildings Custom construction Maintenance Requirements Regular calibration and alignment Consumables, lamps, glasses,... Often custom, exotic parts C6

18 Specialized Visualization Rooms Usability Interface familiarity Physical: wands, gloves, tablets,... Software: applications, file formats,... Training and support Access Availability Distance COVISE

19 Multipurpose Visualization Rooms Enhanced conference and meeting rooms Remote Collaboration Data sharing Session sharing HD video interaction Infrastructure Familiarity Extend existing facility General software environment Mostly COTS equipment Flexibility Respond to evolving needs Utilization

20 Visualization Clusters Data center visualization: no more shelves full of workstation towers Thank the GPGPU folks: Server friendly graphics Passive cooling, environmental monitoring, error reporting PCI-e x16 in 1U servers and blades, GPUs on the motherboard Thank the Virt/Cloud folks: Graphics friendly servers Decent selection and vendor competition in big memory and big I/O servers Up to 2TB RAM, 8GPUs in a single system image

21 Visualization Clusters Hybrid clusters Tightly coupled, mixed compute and graphics Configurations are compute dominated. Often less memory, more compute than ideal Target: coupled compute+visualization and data+visualization Dedicated clusters Scheduling for interactive, not batch Less traditional HPC users Target: remote graphics workstations

22 Current Data Center Visualization Issues User interface Visualization rarely a batch process Select resources Launch remote environment Manage connections Persistent sessions Application portals? Systems Virtual machines with graphics Graphics hardware sharing Communication/Video/Image acceleration

23 Thank you Questions?