Introduction to Scientific Visualization in the WestGrid Environment

Transcription

1 Introduction to Scientific Visualization in the WestGrid Environment Jon Johansson Academic Information and Communication Technologies University of Alberta Slide 1 of 75 What is Scientific Visualization? convert scientific data into visual form use computers to process data numerical algorithms such as marching cubes to extract features add color, lights, camera view (create a 3D scene) graphics techniques to render the data sets can be large many gigabytes or more the resulting image is presented to the human visual processing system for the direct analysis and interpretation of the information Slide 2 of 75 1

2 What is Scientific Visualization? a successful visualization is a representation of data which allows the user to gain some insight into the data communicates the insight to others frequently deal with data that has a natural geometric structure MRI, CAT data (measured) weather simulations (computed) Slide 3 of 75 What is Scientific Visualization? the key point is that we can understand data in a picture much better than we can understand a bunch of numbers WestGrid resources can generate a lot of numbers!! Slide 4 of 75 2

3 What is Scientific Visualization? use well understood algorithms to extract features from data sets goals understand the data (exploration) interactively examine the data looking for the parts that are interesting communicate the data to peers publications conferences NOTE: the exploration phase is usually interactive so we need to generate images in (almost) real time humans in the loop have limited tolerance for delay Slide 5 of 75 What is Scientific Visualization? once we find interesting structures in the data we have the freedom to arrange colors lights transparency camera views this should clarify what we are looking at Slide 6 of 75 3

4 WestGrid Visualization Resources WestGrid provides a visualization server located at Simon Fraser University it the machine can be accessed as: hydra.westgrid.ca vizserver.westgrid.ca IP address: Hydra is an SGI Onyx UltimateVision graphics supercomputer 24 processors 14 GB RAM 10 graphics pipes Slide 7 of 75 WestGrid Visualization Resources You should have access to hydra automatically by being granted access to WestGrid resources Info about accounts: Apply for an account at: p// / pp / / ppy g_ for _ an _ a ccount Slide 8 of 75 4

5 WestGrid Visualization Resources Web links for more information: WestGrid Collaboration/Visualization Visualization Visualization Server Page on_server/using_vizserver WestGrid Visualization Facilities on_facilities Slide 9 of 75 Visual Serving OpenGL Vizserver 1.1 released by SGI in April 2000 users are able to interact with visualization supercomputers from anywhere in the world connect individually connect as a collaborative community rebranded as Visual Area Networking (VAN) in 2002 hardware coupled with OpenGL Vizserver 2.0 Slide 10 of 75 5

6 WestGrid Visual Area Network Slide 11 of 75 SGI OpenGL Vizserver SGI Vizserver is client-server software that allows you to run fully hardwareaccelerated graphics applications on a remote machine and have images delivered to your desktop the server is installed on hydra.westgrid.ca the client is installed on your workstation Slide 12 of 75 6

7 SGI OpenGL Vizserver OpenGL VizServer the server software runs on the WestGrid Visualization Server machine the client runs on the users desktop machine there are currently 5 licenses for the Vizserver software on hydra Slide 13 of 75 SGI OpenGL Vizserver Application-transparent OpenGL Vizserver runs existing OpenGL APIbased applications without modification Shared application control OpenGL Vizserver instantly turns existing stand-alone applications into collaborative applications with up to five remote sites sharing control of a single interactive application Slide 14 of 75 7

8 SGI OpenGL Vizserver Platform/system independent OpenGL Vizserver gives existing workstations, personal computers, and even wireless handheld devices the power of a Silicon Graphics Prism or SGI Onyx family system the application uses the resources on the server CPUs and graphics pipes images are sent to the client machine no need for much graphic power Slide 15 of 75 SGI OpenGL Vizserver you can run nearly any graphics or visualization application on the remote SGI host (hydra.westgrid.ca), and have the resulting imagery delivered to your desktop the client machine simply decompresses and displays the image created by the server Slide 16 of 75 8

9 SGI OpenGL Vizserver why the Vizserver approach is better: X doesn't support hardware accelerated 3D graphics OpenGL function calls are passed to the graphics card on your desktop the X protocol is horribly inefficient Vizserver uses a much more streamlined (custom) protocol to move events and payload image data data compression is available Vizserver lets multiple users interact simultaneously with a single application running on the graphics server each connection has a dedicated graphics pipe Slide 17 of 75 SGI OpenGL Vizserver OpenGL Vizserver clients are available for SGI IRIX OS-based systems, such as Silicon Graphics Octane2, Silicon Graphics Fuel, Silicon Graphics O2+, and the SGI Onyx family Linux Solaris Windows NT, Windows 2000, Windows XP, and Windows XP Tablet PC Edition Mac OS X Slide 18 of 75 9

10 SGI OpenGL Vizserver the OpenGL Vizserver client is a free download from the SGI website note that you must create a Supportfolio ID to be able to download the client this is free Slide 19 of 75 Connection to Hydra you need to create an SSH tunnel to hydra to be able to use it. from a Linux/Unix machine: ssh -L 7051:hydra.westgrid.ca:7051 -N username@hydra.westgrid.ca replace username with your WestGrid username and you will be prompted for your password 7051 is the port that vizserver uses to make connections Slide 20 of 75 10

11 Connection to Hydra look at this connection command: ssh -L 7051:hydra.westgrid.ca:7051 -N -L 7051:hydra.westgrid.ca:7051 specifies that port 7051 on the local (client) machine is to be forwarded to the given host and port on the remote machine this is an outgoing tunnel -N means do not execute a remote command for an incoming tunnel (the given port on the remote (server) machine is to be forwarded to the given port on the local machine) use -R instead of -L Slide 21 of 75 Connection to Hydra you will need to set an environment variable to be able start a vizserver session: VSX_SESSION_HOST = hydra.westgrid.ca launch the vizserver client connect to localhost not to hydra the connection is handled by the tunnel you still need to authenticate to Hydra with your WestGrid username and password. on/vizserver_tunnel.pdf Slide 22 of 75 11

12 Connection to Hydra Sample session start set up the SSH tunnel ssh -L 7051:hydra.westgrid.ca:7051 -N set the environment variable (Bash shell) export VSX_SESSION_HOST=hydra.westgrid.ca launch the client vizserver -h localhost Slide 23 of 75 OpenGL Vizserver Sample session start (cont.) enter your WestGrid user id and password to login to hydra start a session select parameters for the session launch applications from the vizserver console window Slide 24 of 75 12

13 SSH tunneling also called port forwarding or port mapping forward a network port from one network node to another used to carry insecure network traffic over the Internet in a secure way users outside a firewall can access a service inside the firewall Slide 25 of 75 SSH tunneling ssh connects using port 22 connection is encrypted and secure vizserver connects using port 7051 connection is not encrypted to make a vizserver session secure, wrap the vizserver session inside ssh Slide 26 of 75 13

14 SSH tunneling on the source machine ssh listens for packets addressed to port 7051 all those packets are encrypted and addressed to the destination using port 22 on the destination machine when the packets are received, ssh decrypts them packets are then sent to the destination port Slide 27 of 75 SSH tunneling We can use a GUI and manually assign port forwarding port 7051 on localhost is forwarded to 7051 on the remote host this is an outgoing tunnel outgoing tunnels protect the data that your local computer sends to the remote host computer we don t need an incoming (reverse) tunnel for port only an outgoing tunnel to the server incoming tunnels protect the data that your local computer is receiving i from the remote host computer. our main concern is that the username and password used to authenticate to hydra be encrypted Slide 28 of 75 14

15 Scientific Visualization Having opened a connection to hydra we would like to use a visualization application what s available? Package Name AVS/Express 6.2 OpenDX VTK Start Command express dx vtk (the tcl shell for vtk) if something is not configured correctly, please send a message to support@westgrid.ca Slide 29 of 75 Viz Packages OpenDX, VTK and AVS Express use a pipelined, component-based architecture a user can quickly assemble modular software components into a finished application. these systems are flexible - components can be combined in a multitude of ways, thereby allowing an application developer to accomplish a wide variety of visualization tasks they are extensible - they offer the means for developers to add new components to the system, thereby extending the system s functionality Slide 30 of 75 15

16 OpenDX to be able to run OpenDX on hydra: set the environment variable DXROOT to setenv DXROOT /usr/local/dx (tcsh) export DXROOT=/usr/local/dx (bash) add the following directory to your library path setenv LD_LIBRARY_PATH {$LD_LIBRARY_PATH}:/usr/local/lib/ (tcsh) add the bin directories to your path setenv PATH {$PATH}:/usr/local/dx/bin:/usr/local/dx/bin_sgi Slide 31 of 75 Scientific Visualization In order to create a visualization: import your data into the package explore the data looking for the interesting features lots of work here having found interesting features set up the view to show them clearly. Select: colors/surface properties camera view lights (these have some properties to adjust) export images/animations Slide 32 of 75 16

17 Scientific Visualization color choice can help to clarify a visualization a handy guide is the online paper A A Rule-based Tool for Assisting Colormap Selection by Lawrence D. Bergman, Bernice E. Rogowitz and Lloyd A. Treinish, all from IBM pravda/index.htm lots of interesting references at the end Slide 33 of 75 Scientific Visualization another very nice online publication is How NOT to Lie with Visualization by Bernice E. Rogowitz and Lloyd A. Treinish, from IBM pravda/truevis.htm PRAVDA is intellectual property (IBM s) separate from OpenDX and isn t freely available the papers p are still worth reading Slide 34 of 75 17

18 Scientific Visualization Slide 35 of 75 Source: Sample Data Set We have the electric potential due to a dielectric cylinder introduced into a constant electric field E 0 the parameters I used are: E 0 = 50 V/m dielectric constant = 5 F/m R cyl = 10 m -50 m x, y, z 50 m Slide 36 of 75 18

19 Sample Data Set Slide 37 of 75 OpenDX Data Import a data file might consist of ascii values, each line containing 4 values it s important know how the file was written: row major: for( i = 0; i < nx; i++){ for( j = 0; j < ny; j++){ for( k = 0; k < nz; k++){ fprintf( outfh, "%12.8f %12.8f %12.8f %12.8f \n", x, y, z, V ); } } } row major is defined by the last index varying the fastest Slide 38 of 75 19

20 OpenDX Data Import We have a data set that has regular spacings in the x, y and z directions We don t need to read the coordinates for each point from a file The general e header file to read in this data a is: file =./pot_cart.dat grid = 101 x 101 x 101 format = ascii interleaving = field majority = row header = lines 0 field = potential structure = scalar type = float dependency = positions positions = regular, regular, regular, -50.0, 1.0, -50.0, 1.0, -50.0, 1.0 end Slide 39 of 75 Sample Data Set the connections between the points are inferred when you tell OpenDX that the data are row major these are Cartesian connections Slide 40 of 75 20

21 Sample Data Set you can also work in non-cartesian coordinates the grid is row major in cylindrical coordinates (r, phi, z) write Cartesian coordinates in the file and connections are inferred from the cylindrical grid Slide 41 of 75 Sample Data Set Slide 42 of 75 21

22 Sample Data Set Slide 43 of 75 Data Import Start with a FileSelector module to browse to the general file describing the data set and connect it to an import module which actually does the reading Double click on the FileSelector to get a widget to browse the file system When the data is in the package we can start applying algorithms to it and look for interesting features Slide 44 of 75 22

23 Isovalues of the electric potential The lower slice shows the electric potential with contour lines to give a sense of the shape change close to the cylinder Use a slab module to extract a slice of the potential field Use the Isosurface module to extract contour lines at values -1750, -1250, -750, -250, 250, 750, 1250, 1750 volts Slide 45 of 75 In the Slab module set the parameters: dimension: z position: 0 thickness: (0 or 1) this is the default The AutoColor module generates a color map so that we have something to see Slide 46 of 75 23

24 Slide 47 of 75 Take the output of the Slab module and attach it to the input of an Isosurface module The 2D slab data will result in contour lines Set the isovalues at -1750, -1250, -750, -250, 250, 750, 1250, 1750 volts The contour lines are then passed to a Tube module Set the diameter of the tube to 1.0 Slide 48 of 75 24

25 Slide 49 of 75 Electric field lines The middle slice is color mapped to the magnitude of the electric field and shows some field lines The lines are diverted in the region of the cylinder Use the Gradient module and the Compute module to calculate the electric field from the electric potential E = - grad ϕ Slide 50 of 75 25

26 Use a Slab module to extract a slice of the electric field Use the Streamline module to extract contour lines at values The Streamline module needs the vector field as one input, and a set of seed points as the next input Use the Tube module to produce lines with some thickness Slide 51 of 75 Use the Compute module after the gradient to negate the vector components In the Compute module set expression to [-a.x, -a.y, -a.z] In Slab set dimension: z position: 50 thickness: (0 or 1) this is the default Slide 52 of 75 26

27 Slide 53 of 75 To calculate streamlines connect the Electric Field (output of Compute) to the left input of the Streamlines module Use another Slab module to extract a line of points to use as seeds for the stream line Dimension: x Position: 0 Use Reduce to decrease the number of points by 4 Use Tube with diameter 1.0 Slide 54 of 75 27

28 Slide 55 of 75 The top slice shows vector glyphs at some grid points of the electric field The glyphs show the direction of the field The size of the glyphs is scaled to the magnitude of the field Slide 56 of 75 28

29 Insert a Reduce module and a Glyph module between Slab and Collect The Slab module parameters: Dimension: z Position: 100 The Reduce module has Factor: 4.0 In the Glyph module Type: rocket Shape: 1.0 Scale: 0.15 Ratio: 0.1 Slide 57 of 75 Slide 58 of 75 29

30 We have applied some visualization techniques to the data and created 3 slices pretty, but what does it mean?? this lacks context need some information to help the viewer understand what we are looking at Slide 59 of 75 Annotations add these to provide the context for the visualization ShowBox lets the viewer know the volume of space occupied by the data Text: titles, labels explain a bit Axes provide orientation and scale ColorBar shows how the numbers are mapped to color Draw the cylinder so we see the cause of all the trouble in the field Slide 60 of 75 30

31 The ShowBox module draws a box around the extents of the data set Use a Tube module to get nicely rendered lines Caption modules allow you to place text in the 2D plane of the screen Set the text and location in the interface Slide 61 of 75 Slide 62 of 75 31

32 A very useful annotation in this visualization is some indication of where the cylinder actually is It is, after all, the source of all the disruption in the field Slide 63 of 75 Use the Construct module to create a line Origin: {[0 0-50]} Deltas: {[0 0 1]} Counts: [ ] Data: {1} The Tube module creates a cylinder around the line Diameter: 20.0 Ngon: 25 The Color module does just that Color: purple Opacity: 0.3 Slide 64 of 75 32

33 Slide 65 of 75 When we put it all together and select the camera angle that we want we hopefully have what we want My goals Show a variety of techniques that I can apply to this data set Provide enough annotation to orient the viewer Make it look nice Slide 66 of 75 33

34 Slide 67 of 75 Debugging Debugging Sometimes things aren t working and you just don t know why OpenDX s equivalent of the print statement is the Describe module Attach the Describe module to the output port of any module that is providing a field and Describe will give you a summary of what the module is providing in the Message Window. Slide 68 of 75 34

35 Debugging Slide 69 of 75 Scientific Visualization lots of info on the web we have some introductory t pages at Slide 70 of 75 35

36 WestGrid Future Visualization nothing definite yet vendors are supplying Viz Clusters to provide the functionality of the VAN a Viz Cluster is: a bunch of PC parts graphics processing units (GPUs) for rendering software to tie it all together a Viz Cluster combines cpus for numerical computations combines gpus for graphics rendering Slide 71 of 75 WestGrid Future Visualization a viz cluster is generally running a version of Linux with Message Passing Interface (MPI/Pro, Mpich or LAM/MPI) for inter-process communication Distributed MultiHead X (DMX) to create a single X Windows session across multiple displays Chromium is used to distribute OpenGL work VNC for remote access to the server Slide 72 of 75 36

37 WestGrid Future Visualization Distributed Rendering Tile Display Remote Collaboration Slide 73 of 75 Vendors HP Scalable Visualization Array html IBM Deep Computing Visualization 03.ibm.com/systems/deepcomputing/visualiz ation/ Sun Visualization System Slide 74 of 75 37

38 The End Questions? Slide 75 of 75 38