High. Visualization. Performance. Enabling Extreme-Scale. E. Wes Bethel Hank Childs. Scientific Insight. Charles Hansen. Edited by

Transcription

1 High Performance Visualization Enabling Extreme-Scale Scientific Insight Edited by E. Wes Bethel Hank Childs Charles Hansen Taylor & Francis Group Boca Raton London New York CRC Press is an imprint of the Taylor & Francis Croup, an informa business A CHAPMAN & HALL BOOK

2 Foreword v Preface vii Contributor List ix List of Figures xiii List of Tables xxi Acknowledgments xxxiii 1 Introduction 1 E. Wes Bethel 1.1 Historical Perspective Moore's Law and the Data Tsunami Focus of this Book Book Organization and Themes Conclusion 6 1 Distributed Memory Parallel Concepts and Sys tems 7 2 Parallel Visualization Frameworks 9 Hank Childs 2.1 Introduction Background Parallel Computing Data Flow Networks Parallelization Strategy Usage Advanced Processing Techniques Contracts Data Subsetting Parallelization Artifacts Scheduling Conclusion 22

3 xxiv Remote and Distributed Visualization Architectures 25 E. Wes Bethel and Mark Miller 3.1 Introduction Visualization Performance Fundamentals and Networks Send-Images Partitioning Send-Data Partitioning Send-Geometry Partitioning Hybrid and Adaptive Approaches Which Pipeline Partitioning Works the Best? Case Study: Visapult Visapult Architecture: The Send-Geometry Partition Visapult Architecture: The Send-Data Partition 3.9 Case Study: Chromium Rcnderserver Case Study: Visit and Dynamic Pipeline Reconfiguration How Visit Manages Pipeline Partitioning Send-Geometry Partitioning Send-Images Partitioning Automatic Pipeline Partitioning Selection Conclusion 45 4 Rendering 49 Charles Hansen, E. Wes Bethel, Thiago he, and Carson Brownlee 4.1 Introduction Rendering Taxonomy Rendering Geometry Volume Rendering Real-Time Ray Tracer for Visualization on a Cluster Load Balancing Display Process Distributed Cache Ray Tracing DC BVH DC Primitives Results Maximum Frame Rate Conclusion 66 5 Parallel Image Compositing Methods 71 Tom Peterka and Kwan-Liu Ma 5.1 Introduction Basic Concepts and Early Work in Compositing Definition of Image Composition Fundamental Image Composition Algorithms Image Compositing Hardware Recent Advances Swap 77

4 xxv Radix-k Optimizations Results Discussion and Conclusion Conclusion Directions for Future Research 85 6 Parallel Integral Curves 91 David Pugmire, Tom Peterka, and Christoph Garth 6.1 Introduction Challenges to Parallelization Problem Classification Approaches to Parallelization Test Data Parallelization Over Seed Points Parallelization Over Data A Hybrid Approach to Parallelization Algorithm Analysis Hybrid Data Structure and Communication Algorithm Conclusion Ill II Advanced Processing Techniques Query-Driven Visualization and Analysis 117 Oliver Rubel, E. Wes Bethel, Prabhat, and Kesheng Wu 7.1 Introduction Data Subsetting and Performance Bitmap Indexing Data Interfaces Formulating Multivariate Queries Parallel Coordinates Multivariate Query Interface Segmenting Query Results Applications of Query-Driven Visualization Applications in Forensic Cybersecurity Applications in High Energy Physics Linear Particle Accelerator Laser Plasma Particle Accelerator Conclusion Progressive Data Access for Regular Grids 145 John Clyne 8.1 Introduction Preliminaries Z-Order Curves Constructing the Curve 149

5 xxvi Progressive Access Wavelets Linear Decomposition Scaling and Wavelet Functions Wavelets and Filter Banks Compression Boundary Handling Multiple Dimensions Implementation Considerations Blocking Wavelet Choice Coefficient Addressing A Hybrid Approach Volume Rendering Example Further Reading In Situ Processing 171 Hank Childs, Kwan-Liu Ma, Hongfeng Yu, Brad Whitlock, Jeremy Meredith, Jean Favre, Scott Klasky, Norbert Podhorszki, Karsten Schwan, Matthew Wolf, Manish Parashar, and Fan Zhang 9.1 Introduction Tailored Co-Processing at High Concurrency Co-Processing With General Visualization Tools Via Adaptors Adaptor Design High Level Implementation Issues In Practice Co-Processing Performance Concurrent Processing Service Oriented Architecture for Data Management in HPC The ADaptable I/O System, ADIOS Data Staging for In Situ Processing Exploratory Visualization with Visit and Paraview Us ing ADIOS In Situ Analytics Using Hybrid Staging Data Exploration and In Situ Processing In Situ Visualization by Proxy In Situ Data Triage Conclusion Streaming and Out-of-Core Methods 199 David E. DeMarle, Berk Geveci, Jon Woodring, and Jim Ahrens 10.1 External Memory Algorithms Taxonomy of Streamed Visualization Streamed Visualization Concepts 204

6 xxvii Data Structures Repetition Algorithms Sparse Traversal Survey of Current State of the Art Rendering Streamed Processing of Unstructured Data General Purpose Systems Asynchronous Systems Lazy Evaluation Conclusion 215 III Advanced Architectural Challenges and Solu tions GPU-Accelerated Visualization 223 Marco Ament, Steffen Prey, Christoph Muller, Sebastian Grottel, Thomas Ertl, and Daniel Weiskopf 11.1 Introduction Programmable Graphics Hardware High-Level Shader Languages General Purpose Computing on GPUs GPGPU Programming Languages GPU-Accelerated Volume Rendering Basic GPU Techniques D Texture-Based Rendering D Texture-Based Rendering Ray Casting Advanced GPU Algorithms Scalable Volume Rendering on GPU-Clusters Sort-Last Volume Rendering Sort-First Volume Rendering Particle-Based Rendering GPU-Based Glyph Rendering Large Molecular Dynamics Visualization Iterative Surface Ray Casting GPGPU High Performance Environments New Challenges in GPGPU Environments Distributed GPU Computing Distributed Heterogeneous Computing Large Display Visualization Flat Panel-Based Systems Projection-Based Systems Rendering for Large Displays 246

7 xxviii Hybrid Parallelism 261 E. Wes Bethel, David Camp, Hank Childs, Christoph Garth, Mark Howison, Kenneth I. Joy, and David Pugmire 12.1 Introduction Hybrid Parallelism and Volume Rendering Background and Previous Work Implementation Shared-Memory Parallel Ray Casting Parallel Compositing Experiment Methodology Results Initialization Ghost Data/Halo Exchange Ray Casting Compositing Overall Performance Hybrid Parallelism and Integral Curve Calculation Background and Context Design and Implementation Parallelize Over Seeds Parallelize Over Blocks Experiment Methodology Factors Influencing Parallelization Strategy Test Cases Runtime Environment Measurements Results Parallelization Over Seeds Parallelization Over Blocks Conclusion and Future Work Visualization at Extreme Scale Concurrency 291 Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther Weber, and E. Wes Bethel 13.1 Overview Pure Parallelism Massive Data Experiments Varying over Supercomputing Environment Varying over I/O Pattern Varying over Data Generation Scaling Experiments Study Overview Results Pitfalls at Scale Volume Rendering All-to-One Communication 303

8 xxix Shared Libraries and Start-up Time Conclusion Performance Optimization and Auto-Tuning 307 E. Wes Bethel and Mark Howison 14.1 Introduction Optimizing Performance of a 3D Stencil Operator on the GPU Introduction and Related Work Design and Methodology Results Algorithmic Design Option: Width-, Height-, and Depth-Row Kernels Device-Specific Feature: Constant Versus Global Memory for Filter Weights 314 Parameter: Thread Block Tunable Algorithmic Size Lessons Learned Optimizing Ray Casting Volume Rendering on Multi-Core GPUs and Many-Core GPUs Introduction and Related Work Design and Methodology Results Tunable Parameter: Image Tile Size/CUDA Block Size Algorithmic Optimization: Early Ray Termi nation Algorithmic Optimization: Z-Ordered Mem ory Lessons Learned Conclusion The Path to Exascale 331 Sean Ahern 15.1 Introduction Future System Architectures Science Understanding Needs at the Exascale Research Directions Data Processing Modes In Situ Processing Post-Processing Data Analysis Visualization and Analysis Methods Support for Data Processing Modes Topological Methods Statistical Methods Adapting to Increased Data... Complexity 343

9 and XXX I/O and Storage Systems Storage Technologies for the Exascale I/O Middleware Platforms Conclusion and the Path Forward 347 IV High Performance Visualization Implementa tions Visit: An End-User Tool for Visualizing and Analyzing Very Large Data 357 Hank Childs, Eric Brugger, Ahem, David Pugmire, Brad Whitlock, Jeremy Meredith, Sean Kathleen Biagas, Mark Miller, Cyrus Harrison, Gunther H. Weber, Hari Krishnan, Thomas Fogal, Allen Sanderson, Christoph Garth, E. Wes Bethel, David Camp, Oliver Rubel, and Marc Durant, Jean M. Favre, Paul Navrdtil 16.1 Introduction Focal Points Enable Data Understanding Support for Large Data Provide a Robust and Usable Product for End Users Design Architecture Parallelism User Interface Concepts and Extensibility The Size and Breadth of Visit Successes Scalability Successes A Repository for Large Data Algorithms Supercomputing Research Performed with Visit User Successes Future Challenges Conclusion IceT 373 Kenneth Moreland 17.1 Introduction Motivation Implementation Theoretical Limitations... How to Break Them Pixel Reduction Techniques Tricks to Boost the Frame Rate Application Programming Interface Image Generation Opaque versus Transparent Rendering Conclusion 379

10 xxxi 18 The ParaView Visualization Application 383 Utkarsh Ayachit, Berk Geveci, Kenneth Moreland, and John Patchett, Jim Ahrens 18.1 Introduction Understanding the Need The ParaView Framework Configurations Parallel Data Processing The ParaView Application Graphical User Interface Scripting with Python Customizing with Plug-ins and Custom Applications Co-Processing: In Situ Visualization and Data. Analysis ParaViewWeb: Interactive Visualization for the Web ParaView In Use Identifying and Validating Fragmentation in Shock Physics Simulation ParaView at the Los Alamos National Laboratory Analyzing Simulations of the Earth's Magnetosphere Conclusion The ViSUS Visualization Framework 401 Valerio Pascucci, Giorgio Scorzelli, Brian Summa, Peer-Timo Bremer, Attila Gyulassy, Cameron Christensen, Sujin Philip, and Sidharth Kumar 19.1 Introduction ViSUS Software Architecture Applications The VAPOR Visualization Application 415 Alan Norton and John Clyne 20.1 Introduction Features Limitations Progressive Data Access VAPOR Data Collection Multiresolution Visualization-Guided Analysis Progressive Access Examination Discussion Conclusion The EnSight Visualization Application 429 Randall Frank and Michael F. Krogh 21A Introduction EnSight Architectural Overview 430

11 xxxii 21.3 Cluster Abstraction: CEIShell Virtual Clustering Via CEIShell Roles Application Invocation CEIShell Extensibility Advanced Rendering Customized Fragment Rendering Image Composition System Conclusion 440 Index 443