VRX: Virtual Reality explorer Toolkit v A brief system specification -

Transcription

1 VRX: Virtual Reality explorer Toolkit v A brief system specification - Michal Koutek, M.Koutek@ewi.tudelft.nl VR and Visualization Group, Faculty of Electrical Engineering, Mathematics and Computer Science, Delft University of Technology 1) Summary: Virtual Laboratory for e-science (VL-e) Consortium VRX is a toolkit for interactive 3D visualization of data using Virtual Reality (VR). This toolkit has been submitted for the contribution area of the VL-e proof-of-concept environment (pre-poc). VRX toolkit has been developed at the TU Delft in the VR & Visualization section of the Computer Graphics Group. This software is made available to the interested project partners of VL-e consortium and to the public as open source, under conditions explained later in the license paragraph. The primary goal of this software submission is to demonstrate the concepts of interactive visualization of three-dimensional data using VR in a virtual laboratory setting. Object oriented and modular structure of the VRX toolkit allows making extensions for interactive visualization of the data that can be processed on the GRID using grid computing. In such a situation the VRX client will run locally at a graphic workstation. The current VRX toolkit (version 2.0) is based on visualization toolkit VTK v.5.0, a publicly available open source toolkit. VRX is a visualization module of the VRmeer Library that has been developed in a course of several projects by the VR group at the TU Delft. The VRmeer Library provides a Virtual Reality API on top of the scene graph library OpenSceneGraph (OSG v.2.4) and the tracking library VRPN v Several visualization techniques in VRX use GPU and require libfloat from IlmBase of OpenEXR library. OSG, VRPN, OpenEXR libraries are publicly available as open source. VRmeer is an ongoing research project and some parts of VRm library are not part of the software distribution. The VL-e pre-poc submission of the VRX includes only the core of the VRmeer library as Open Source Edition (OSE). For the use in the VL-e project the VL-e PoC software submission contains the depending libraries as a whole bundle that can be built together. External users can download VRmeer OSE only without dependencies. In such a case they can download the actual versions of VTK, OSG, VRPN, and OpenEXR from the internet; see the links in the references. 1

2 VRmeer (shortly VRm) is a high-level VR API library that provides the necessary VR functionality for the applications, such as setting up the VR rendering and display pipeline for a specific VR system (workbenches, power walls, personal VR systems, and desktop displays), supporting direct and remote 3D interaction (selection and manipulation) with the objects in the 3D scene, hybrid 3D/2D graphical user interfaces, and a number of VR optimizations of the scene graph processing. It has been developed on top of the OpenSceneGraph library that provides generic 3D scene handling and basic scene-graph operations. VRPN library is used as an interface to the tracking and 3D interaction devices. VR applications can be either implemented using C++, optionally using Python, or as a combination of both. Note that Python version of VRm library as Python wrappings of the C++ library code are not part of the VRm OSE, but can be eventually requested from the Visualization group of TU Delft. 2) VRX Architecture: VRX uses a dataflow visualization pipeline approach of VTK on the back-end. On the front-end it uses VRm and OSG libraries to provide an interactive immersive application environment for 3D data exploration. The multiple data fields can be attached to VRX. These data fields can be either read from a file or can be an output of data processing pipeline, performed for example on the computational GRID. The spatial domain of the data fields is in VRX covered by the SystemSpace object. The SystemSpace is a VRm object in the Virtual Environment (VE) that can be interactively manipulated and oriented. The user can attach multiple DataSources (data fields) to it. Various visualization tools can be applied on the active DataSources in the SystemSpace. VRX primarily supports rectangular, regularly structured 3D grids of data; in the terms of VTK: vtkstructuredpoints, vtkimagedata. This has to do with the fact that these types of grid are most suitable for GPU acceleration of the direct visualization tools that are executed on the VR graphics workstation. Other types of data grids, such as structured, but not rectangular, or rectilinear, hybrid, or even unstructured, are supported through VTK. When a fast, direct visualization of such data is needed in the virtual environment, then the slower grid can be converted into vtkstructuredpoints. The VTK data processing pipeline can also provide 3D visualizations of data. In that case VRX has on the input the geometric data coming from the VTK pipeline (vtkpolydata). This geometry will be encapsulated into a VR object and placed scene graph hierarchy under the SystemSpace object. This way VRX implements for example iso-surface extraction, streamline advection, vector probes, etc. 2

3 VRX has been designed under consideration that 3D visualization and data exploration using VR has to operate in highly interactive and responsive fashion. Therefore the computationally intensive data and visualization processing should be decoupled from the run-time VR application. Further the VRX toolkit utilizes high-end graphics workstation with modern GPU s to provide fast and direct visualization processing where possible on the client side. The use of multi-processing, multi-threading, and shared memory makes the VR part of the system independent of the data / visualization processing pipeline, which would otherwise introduce latency in update of the visualized graphical primitives in the scene and significantly lower the VR application interactivity and responsiveness. 3) VRX Modules: The VRX toolkit has object oriented structure and the visualization toolbox contains the following modules: vrxsystemspace: This is a central class that visually represents the 3D volume of the data in the virtual environment (VE). The data typically originate from computer simulations or measurements. The 3D-VIS application experiment using VRX provides visualization of typically several scalar and vector data components. Since the datasets don t have directly a visual representation (i.e. 3D geometry) they have to be first added to the vrxsystemspace as vrxdatasources and then visualized, probed or measured by a tool of class category vrxvistool (abstract class). The instance of vrxsystemspace has to be initialized with a vrxdatasource, which holds the correct spatial data extent, such as the one that VTK is using. Note that vrxsystemspace will normalize the data volume / extents in such a way that the maximum extent will become 1.0 v.u. (virtual units) and the data volume will keep the original (vtk) aspect ratio. In VRX application the user can specify the relative scale in virtual units of the maximal data extent, for example 20. vrxdatasource: This class captures the 3D data on the 3D grid (supported are vtkstructuredpoints, vtkimagedata), either read from a file or originating from any VTK processing pipeline. The vrxdatasource is managing a SharedMemory copy of the data array from the vtkdataset for the purposes of several (GPU-) accelerated tools that run on the graphics VR client to ensure high refresh rate of the VE. This enables a fast (on the VTK independent) exploration, visualization, and probing of the copy of the data by the vrxtools. Note that when the original vtkdataset changes, for example with a next time step, then also the vrxdatasource must be updated. vrxmappercolor: This class provides the color mapping schemes for the (scalar) data. The vrxdatasource must be assigned to an instance of vrxmappercolor. If vector data is provided VRX can generate vector magnitude scalar data or can extract any vector component as a scalar data. The vrxmappercolor class internally works with the vtklookuptable, but also provides several predefined color schemes. The actual color-mapping visualization step is provided during execution of the vrxvistools. vrxactors: This class is derived from the generic vrxvistool and the vtkactor. It enables embedding results of a VTK-processing pipeline as geometrical entities into the vrxsystemspace. The vrxsystemspace has to be correctly initialized with an appropriate vrxdatasource that defines the grid dimensions and the data volume, consistently with the VTK spatial data extent. The geometrical primitives are converted by vtkactortoosg everytime the VTK- pipeline produces a new content of a certain vtkactor. The vrxactors adds VR interaction functionality to these objects, and adds the (3D GUI) controls for the attached VTK-pipeline. 3

4 vrxprocessedgeometry: It is a generic visualization tool that is typically connected to any 3D geometry producing vtk-pipeline. For example you can have a vtk pipeline that reads or generates vtkpolydata and converts them into vtkactor. vrxprocessedgeometry is the easiest way to show the geometry in VR. If more complex interaction behavior is needed, consider to use vrxactor instead. Keep in mind that the processed geometry will be added to vrxsystemspace, and the size of the vtk-geometry should be kept in proportion with the size of vrxsystemspace. If the default sizing behavior provided by "fitvtkactorobject()" function doesn't make your custom geometry fit into the systemspace correctly consider to overload fitvtkactorobject() function and make it fit yourself. vrxvistool: It is a generic class of tools that inside of the 3D volume given by the vrxsystemspace operates on a specific, attached vrxdatasource. It works on 3D grid of data or in conjuction with a vrxactor. For faster performance of these tools in the VR environment, some of the tools are implemented using the GPU programming (GLSL - OpenGL Shading Language, tested on Nvidia GPU s). vrxslicers: It is a generic class of slicing tools that makes visualization of cross-sections of 3D grids or any geometry. Such as vrxslicer can be used slice 3D mesh geometry generated by vrxisosurf or any geometry provided by vrxprocessedgeometry tool. Color slicing and 3D mesh slicing is accelerated by GPU programming (GLSL). VRX Visualization Tools - Overview: vrxcolorslicer: - Arbitrary oriented or constrained orthogonal colored slicing planes. - Operates on 3D grid of scalar data. vrxisosurf: - VTK pipeline for extraction of an ISO-surface. - Operates on 3D grid of scalar data. vrxvectorslicer - Arbitrary oriented or constrained orthogonal slicing planes with colored arrows. - Operates on 3D grid of vector data. vrxstreamliner - VTK pipeline for extraction of stream-lines,-tubes, -ribbons, or particles. - Operates on 3D grid of vector data. vrxstreamslicer - Multiple streamline advection from an arbitrary oriented or constrained orthogonal slicing planes. Internally it uses vtkplanesource. - Operates on 3D grid of vector data. Basic probing tools: These tools are used for probing the data values on data slices (vrxslicer), iso-surfaces, any other geometry inside the vrxsystemspace or arbitrary in the 3D space. The following set shows basic examples and extension ideas of point-probing. line-probing, line-profiling, and specialized data measurement tools can be easily further implemented by the VRX users for specific application. 4

5 vrxprobepoint - Places a 3D marker on a point of interest and displays the values of the data field(s). - Has a simple GUI to add and remove probepoints. - The ProbePoints can be translated/ manipulated by means of 3D interaction provided by the VRmeer library. - Tool operates on 3D grid of scalar and vector data. vrxprobestream - Derived class from the vrxprobepoint. - Besides displaying the data value and possibility of changing position of the probe by the user, vrxprobestream will advect a stream-line(-tube, etc.) from the position of the probe. - Tool operates on 3D grid of vector data. vrxprobeiso - Derived class from vrxprobepoint. - Can be used to directly change/select the ISO-value in the 3D space of the data instead of using a slider widget. - Tool operates on 3D grid of scalar data. GUI controls and widgets: VRmeer Library provides a basic set of GUI widgets, such as buttons, sliders, dials, windows, menus to control the VR-VIS application. 3D display widgets exist to show results of the point, line, rectangle probes, showing the position + the value, or eventually 2D plot of value along the line, histogram, or showing for example min/max/mean values. Possible future extensions: Network/GRID communication interfaces in not part of the current VRX submission, but can be developed in the future for data intensive visualization. Time support in VRX (and VTK) is still rather experimental. More advanced handling of time-dependent datasets will be incorporated in the future. The following list will be built into the next version of VRX: vrxdvr (Direct 3D Volume Renderer, GPU acceleration) vrxstreamlineemitter (Streamline GPU Emitter, stationary fields) vrxparticleemitter (Particle GPU Emitter, stationary fields) Data markers and data annotators Measurement tools 4) Software licenses, availability, and support: VRX version 2.0 and VRmeer Library OSE are provided as they are, with module documentation and a number of example applications and tutorials. This software is provided as Open Source, on the basis the BSD license. VRmeer Library and VRX are in a continuous development by a small research community of the Visualization group at the TU Delft and their relations. The source code is provided with no guaranties, and without any commitment of support. At this moment there exists only a project-based support for the project-relations of the development team. An example of that is the Dutch Virtual Laboratory for E-science (VL-e) consortium. New potential users of VRmeer & VRX are welcome. Generally speaking, the support can be (only) provided on a project basis or if there is a common scientific interest. Eventual questions and requests can be directed to supportvrx@gmail.com or check the project website: 5

6 Here is a summary of licenses of the VRmeer library dependencies: - VTK v.5 is open source software; BSD licensed, and it is a part of the current release of the VL-e PoC. - OpenSceneGraph v.2.4 is also open source software, OSGPL licensed, derived from LGPL (Lesser GNU Public License). - OpenEXR - ILMBASE (libhalf), open source software, modified BSD licensed. - VRPN v.7.17 is public domain software, not licensed. 5) Hardware Requirements: - High-end graphics workstation, multi-core high frequency CPU; - RAM >2GB, 3D-stereo capable graphics card (for example Nvidia Quadro FX 1700, 4600); - Stereo graphics display (VR PowerWall, VR Workbench, Desktop VR) - 3D tracking system (must be supported by VRPN); - Operating system LINUX; Tested with Ubuntu 8.04 and RedHat 6) Acknowledgment: VRmeer library with the number of its modules have been developed primarily by the VR- Visualization section at the TU Delft, under supervision of Frits Post, associate professor. This section belongs to the Computer Graphics and CAD/CAM group at the EWI Faculty of TU Delft, headed by prof.dr.ir. F.W. Jansen. Several researchers, master and PhD. students have contributed to the VRmeer project; in alphabetical order: Dylan Dussel, Eric Griffith, Gerwin de Haan, Michal Koutek, Rene Molenaar, and Frits Post as the VR group leader. The development of VRX and VRmeer project has been (partially) financially supported by the project grants: VL-e BSIK, BRICS, and NWO Open Competition awarded project Cumulus Clouds Visualization in VR. 7) References: Last document update: 27 February, 2009 by Michal Koutek. 6

7 Appendix: VRX Tutorial Image Gallery The following figures demonstrate VRX en VRmeer Library in action. The C++ source code can be found in [VRX2.0 install directory]/modules/vrm/app/vrxtutorial. vrxtutorialscalar1: Upper image iso-surface visualization, probing a scalar value, colored cross-section (colorslicer) also serves as a cutting plane of the iso-surface mesh. Left image iso-surface probing; directly in the 3D space probing a data value, which is used for iso-surface extraction. vrxtutorialscalar2: Iso-surface visualization with 3 vrxcolorslicers, demonstrating the use of different vrxmappercolor modules. The cut-off by the 3 slicers is drawn in wireframe so that the user can see the interior. Note: Without 3D tracking and interaction device the VR interaction is emulated using keyboard-mouse, press h for help. ( 1 viewpoint interactor emulator, 2 stylus interactor emulator, CTRL stylus button, left-middle-right mouse button navigate the tracker-emulator) 7

8 Appendix: VRX Tutorial Image Gallery vrxtutorialvector1,.2: Upper image vector plot visualization, vector slicer constrained in Z-axis motion. Right images interactive streamline slicer (stream-tubes). vrxtutorialvector3: Left image vector field visualization, axis-constrained colored slicers (VRX black&white banded color mapping), streamlines, and probing of data values. Right image streamline probes in action; tracing individual streamlines from a probe-point. 8

9 Appendix: VRX Tutorial Image Gallery vrxdemoflow: Airflow in an office shown as experimental visualization with iso-surface, colored slicers, and stream-tubes. vrxtutorialvector3 - extension: Tornado dataset, several visualization tools combined. vrxdemocoral: Visualization and exploration of a coral skeleton and an iso-surface of a coral. Right image a view in detail. 9