Parallel Visualization in the ASCI Program

Transcription

1 AA220/CS238 - Parallel Methods in Numerical Analysis Parallel Visualization in the ASCI Program Lecture 27 November 26, 2003

2 Overview Visualization of large-scale datasets generated with massively parallel machines is a very compute intensive task: Large datasets Usually time-dependent Complex solution features yield large I/O requirements Floating point operations needed to render the image Advancements are required in several areas Basic improvements in visualization algorithms Parallel implementation of visualization algorithms Parallel visualization hardware (scalable and cost-effective)

3 Overview - Cont d Examples in this lecture drawn from: Stanford ASCI work in unsteady turbomachinery flow simulations University of Utah Scientific Computing and Imaging Institute Collaboration with MIT on parallel pv3 A number of research groups are working on parallel visualization techniques (both hardware and software): Stanford University U. of Utah DoE National Laboratories Etc

4 Large-Scale Scientific Visualization Chris Johnson Scientific Computing and Imaging Institute University of Utah

5 Interactive Large-Scale Visualization Medical Scientific Computing GeoScience

6 The Visualization Pipeline Visualization Process Generate Render Dynamic extraction of isosurfaces Rapid extractions Preprocess Search Construct Render Isovalue Offline Online

7 Isosurface Extraction Marching Cubes Octree Extrema Graphs Sweeping Simplices The Span Space Livnat, Shen, Johnson Maximum Isovalue min = max Isovalue Minimum

8 Isosurface Extraction Marching Cubes Octree Extrema Graphs Sweeping Simplices The Span Space Livnat, Shen, Johnson Maximum Isovalue min = max Isovalue Minimum

9 Isosurface Extraction Marching Cubes Octree Extrema Graphs Sweeping Simplices The Span Space Livnat, Shen, Johnson NOISE: O( n+k) Maximum min = max Minimum

10 The Visualization Pipeline Reduce the amount of data Reduce during the search... View point Preprocess Search O(V(k)) O(k) construct O(k) O(V(k)) Render Isovalue Offline Online

11 A View-dependent Approach Attractive for: Large datasets High depth complexity Remote visualization

12 A View-dependent Approach Three step method 1) Traverse front to back Project To Graphics Hardware 2) Project onto a virtual screen 3) Render triangles on graphics hardware Traverse Front to back

13 A View-dependent Approach Object Space Value Space Flow chart Software Front to Back traversal Image Space Prune non - intersecting cells Prune non-visible cells Visibility Part I Hardware Graphics Engine Z-buffer Rendering Final Image Visibility Part II

14 Visible Woman Full View Isosurface depend Polys 2,246, ,000 Create 177 sec 72 sec Render 2.32 sec 0.25 sec

15 Why Not Always Use Polygons? Marching cubes and similar algorithms can generate millions of polygons for large data sets Reduce by decimation (e.g. Shekhar et. al 96) View dependent (e.g. Livnat and Hansen 98)

16 Real-Time Ray Tracer

17 Real-Time Ray Tracer (RTRT) Implemented on SGI Origin 3000 ccnuma architecture - up to 512 processors (now working on a distributed version) Approximately linear speedup Load balancing and memory coherence are key to performance

18 Algorithm - 3 Phases Traversing a ray through cells that do not contain an isosurface Analytically computing the isosurface when the intersecting volume contains an isosurface Shading the resulting intersection point

19 Frames/second (32 processors) Frame Number (time)

20 Real-Time Ray Tracer - Scalability

21 RTRT Time Varying Visualization

22 Real-time Volume Rendering

23 Volume Rendering enamel / background dentin / background dentin / enamel dentin / pulp 1D: not possible 2D: specificity not as good

24 Volume Rendering - 3D Transfer Function

25 Vector Fields ZIB UofU

26 LIC Flow (Banks and Interrante)

27 Illuminated Lines - C. Hege, ZIB

28 Tensor Visualization - Hesselink

29 Brush Strokes (Laidlaw `98)

30 AA220/CS238 - Parallel Methods in Numerical Analysis Large-Scale Visualization of Turbomachinery Flows Using pv3 Lecture 27 November 26, 2003

31 Objectives Utilize existing software and hardware technologies to visualize large datasets with proper scalability in both Display size / resolution Rendering speed Interactive visualization of large-scale datasets for useful investigation of simulation results Understand what can be done with the kind of visualization systems that will be available on the desktop in 2-3 years

32 Motivation At Stanford, in the DoE ASCI (Accelerated Strategic Computing Initiative) we are trying to simulate very large scale flows in turbomachinery. The visualization of these flows is rather difficult and time consuming. Our CS group has a lot of expertise in software and hardware for parallel rendering. Can we leverage these tools in the context of an engineeringusable visualization package?

33

34 Objective - Demonstrate Potential of Hi-Fi Gas Turbine Engine Simulation Integrated fan/compressor/ combustor/turbine/secondaries unsteady flow and turbulent combustion simulation RANS Turbomachinery Combustor RANS (NASA-NCC) LES (CITS) Multi-Code Interface Complex code coupling Will require 100 TFLOPS Have industry and NASA participation and interest P&W 6000 Engine

35 Flamlet-progress variable model for combustion LES Mixture fraction Product mass fraction

36 P&W combustor 2.5D grid 1

37 Stanford-ASCI TFLO Project Goals To develop a scalable code (TFLO) that is capable of: tackling large-scale unsteady flow simulations of multistage turbomachinery, as well as interactions between compressor, combustor, and turbine rapid and cost-effective steady and unsteady analyses required in a design environment (single blade passages, multiple stage simulation with low blade counts) comparable to existing industrial practice incorporate advanced turbulence models with corrections to account for effects typical in turbomachinery (streamline curvature, rotation, etc.) To contribute to the development of numerical simulation techniques that make this type of calculations computationally affordable To demonstrate integrated calculations simulating the interaction between the compressor, combustor, and HP/LP turbine

38 Gas-Turbine Components

39 TFLO performance on P&W 6000 turbine

40 0.21 Unsteady Simulation of Aachen Turbine Rig (TFLO) Simulation Completed, AIAA Paper Presented 13.5 M Points 374 Blocks 187 Processors 2,800 Time-Steps (w/ 30 inner iterations per timestep) Required 1,985 Hours (clock time), 371,000 Hours (cpu Secondary time) Required Velocity Field pressure envelope (p/p ref ) L.E Aachen Blade Unsteady Pressure Envelopes passage count passage count x/c Amplitude of Harmonics T.E L.E S.S P.S Frequency Spectrum Entropy Measured Plane No. 00 Time Index: 1 Predicted Pressure Amplitude (Pa) Estimated T.E. vortex shedding frequency: 10 BPF Frequency/BPF

41 Unsteady Flow Simulation of P&W Turbine Rig (TFLO) Vane/Blade Potential Interaction One Global (1/6 Circumference) (33% of Total) Completed 31.2 M Points 652 Blocks 196 Processors 4,200 Time-Steps (w/ 30 inner iterations per time-step) Required Shock/Blade Interaction: Reflected Waves from Vane 4,125 Hours (clock time), 808,500 Hours (cpu time) Required Pressure Loading Compares Well with Experiment and PW Prediction Viscous Wake/Blade Interaction Blade Trailing Edge Shocks Pressure Predicted Aerodynamic Losses Compare Favorably with PW Prediction Entropy

42 Unsteady Flow Simulation of PW6000 Turbine (TFLO) LPT (5,6,7) 63% of One Global Cycle (1/6 Circumference) (21% of Total) Completed 93.8 M Points 2192 Blocks Processors 5,700 Time-Steps (w/ 30 inner iterations per timestep) Required 5,970 Hours (clock time), 3,060,000 Hours (cpu time) Required Entropy HPT (1,2,3)

43 Main/Secondary Flow Path Integration Direct Coupling (SPMD) Temperature and Streamlines (projected in constant ) Simulation Complete 9.4 M Points, 238 Blocks, 144 Processors, 1-200,000 Time-Steps Required 3,700 Hours (clock time), 532,800 Hours (cpu time) Required Pressure and Streamlines (projected in constant ) Temperature and 3D Blade- Relative Streamlines

44 Key Technologies Hardware High resolution displays Powerwall Super-high resolution displays (5000x3000 kind) High speed network interconnects for commodity clusters Software Support for tiled / high resolution displays with wiregl Parallel software implementation for scalable rendering using pv3

45 Why pv3? pv3 is already setup for Parallel feature extraction Concurrent visualization Distributed visualization Computational steering Work to be done Use of wiregl for tiled displays (completed) Parallelization of renderer (almost completed)

46 pv3 clients CPU 1 CPU 2 Current Feature ExtractionLarge-Scale Visualization Setup CPU 3 CPU 4 CPU 5 CPU 6 CPU 7 pv3 server (wiregl) Rendering CPU 8 CPU 9 CPU 10 CPU 11 CPU 12 CPU 13 CPU 14 CPU 15 CPU 16 WAN CPU 1 CPU 2 CPU 3 CPU 4 GR 1 GR 2 GR 3 GR 4 wiregl Bottlenecks in WAN (avoidable), single renderer (in progress), internal network

47 pv3 clients CPU 1 CPU 2 Future Feature Extraction Large-Scale Visualization Setup CPU 3 CPU 4 CPU 5 CPU 6 CPU 7 pv3 server (wiregl) Rendering CPU 8 CPU 9 CPU 10 CPU 11 CPU 12 CPU 13 CPU 14 CPU 15 CPU 16 CPU 1 CPU 2 CPU 3 CPU 4 GR 1 GR 2 GR 3 GR 4 wiregl WAN

48 Advantages / Expected Outcome Rendering speed 12 x on current display (best case scenario) High resolution images for flow details Large degree of interactivity for turbomachinery flow visualizations Parallel I/O will be necessary for unsteady flow visualizations