Visualization Toolkit(VTK) Atul Kumar MD MMST PhD IRCAD-Taiwan

Transcription

1 Visualization Toolkit(VTK) Atul Kumar MD MMST PhD IRCAD-Taiwan

2 Visualization What is visualization?: Informally, it is the transformation of data or information into pictures.(scientific, Data, Information) Application of interactive 3D computer graphics to the understanding data. visual data analysis interactive viewing, understanding and reasoning process Conversion of numbers into images humans are generally poor at raw numerical data analysis human visual reasoning allows robust analysis of visual stimuli convert numerical analysis into visual analysis

3 Visualization 3 main sources of visualisation data: Scientific / Medical Visualisation visualising results of simulations, experiments or observations Frequently data is multi-dimensional Information Visualisation Visualisation of abstract, usually discrete data e.g financial data, web site hits etc. Real-world Data 3D capture technologies are reaching maturity Application : Virtual Reality, Games

4 Terminology Imaging or image processing is the study of 2D pictures or images. Computer graphics is the process of creating images using computer. Visualization is the process of exploring, transforming and viewing data as images ( or other sensory forms ) to gain understanding and insight into the data.

5 Visualization:Examples

6 Visualization:Examples

7 Visualization: Examples

8 Visualization: Examples

9 Visualization presenting data in a convenient and efficient visual form for human visual analysis Combining multi-dimensional data into a single image so that it is easy to understand Extracting important features Extracting important surface data from volume data Coloring it in various ways so that it is easy to understand

10 Visualization

11 Visualization

12 Visualization

13 Visualization The visualization process Computational methods -Finite element method -boundary element Data Measured Data -CT, MRI, Ultrasound -satellite Transform Map Display

14 VTK: Introduction A provider of a computer graphics architecture for visualization VTK is a set of methods (toolkit) that implement a variety of visualization operations Implements a visualization pipeline ( series of steps) Platform independent (we use Linux, Windows) Object-orientated visualization Program in C++ or Java or use an interpreted language such as Tcl/Tk or Python VTK also implements basic tools for visualization: 3D computer graphics output & basic interactive user input Resources:

15 VTK: Introduction Graphical objects in VTK

16 VTK: Introduction Actors are rendered in the scene by the renderer object Controls camera and lighting properties The renderer draws to a render window object Controls window size Can display or capture to an image file

17 VTK: Introduction Computer Graphics Objects in VTK To convert a data structure into graphical data (object) in VTK use an object called a mapper Graphics objects in vtk are known as actors Position, rotation and surface properties also specified by actor methods Controls graphics properties such as colour and shading transformation from object to world co-ordinates

18 VTK: Introduction Graphical objects in VTK

19 VTK: Introduction Example: Demo of program for Cone.cxx

20 VTK: Introduction Example: Cone: Its Visualization pipeline

21 VTK: Introduction Example: Demo of program for Cone3.cxx

22 VTK: Introduction VTK : window interactor Functions available (vtkrenderwindowinteractor): Rotate ( left mouse button ) Zoom ( Right mouse button ) Pan ( left mouse + shift key ) w Draw as a wireframe mesh s Draw as a surface mesh r Reset camera view u user defined command (For TCL/TCk). Here, bring up window command box ren AddObserver UserEvent {wm deiconify vtkinteract} e exit p pick actor underneath mouse pointer

23 Visualization: Computer Vision Visualisation & Computer Vision Computer Vision (CV): computer analysis of visual information automated visual reasoning Visualization: computer presentation of that visual information commonly part of the output process in the CV pipeline (especially 3D capture) also used for evaluation in CV (i.e. to provide insight into CV process itself)

24 Object Oriented Programming Terminology: Object Class Encapsulation Interface Inheritance Polymorphism Overloading Abstract class

25 Computer graphics primer Computer graphics is the process of generating images using computers. Also called as Rendering Data converted into graphics primitives and then rendered.

26 Computer graphics primer Simulation of light behavior in 3D Effective simulation requires to model: object representation (geometry) object illumination (lighting) camera model (vision) world to image plane projection rendering: converting graphical data into an image

27 Computer graphics primer Data representation: 3D shape Approximate smooth surfaces with flat, planar polygons Polygons formed of edges and vertices Vertex: positional point (2D or 3D) Edge: joins 2 vertices Polygon: enclosed within N edges mesh: set of connected polygons forming a surface (or object)

28 Computer graphics primer Data representation:

29 Computer Graphics Primer Image-order and object-order There are two types of rendering algorithms Image order, e.g., ray-tracing (or ray-casting) Object order, render one object at a time.

30 Computer Graphics Primer Surface versus volume rendering Surface-rendering: somethings are very hard to do with that kind of technique, e.g., clouds, smoke or translucent surfaces. Volume rendering: Done with ray-tracing or raycasting techniques, where rendering is done one ray at the time.

31 Computer Graphics Primer Color There are three types of cones in the human retina, with characteristic response.

32 Computer Graphics Primer Color models: Models common in use RGB specifies colors as an additive mix of Red, Green and Blue HSV Specifies Hue (color), Saturation, and Value to specify a color. CMY species colors as a subtractive mix of Cyan, Magenta and Yellow.. This color model is mainly used for printing purposes.

33 Computer Graphics Primer Light interaction with surfaces Simple 3 parameter model The sum of 3 illumination terms: Ambient : 'background' illumination Specular : bright, shiny reflections Diffuse : non-shiny illumination and shadows

34 Computer Graphics Primer Ambient Lighting Light from the environment light reflected or scattered from other objects simple approximation to complex 'real-world' process Result: globally uniform colour for object Rc = resulting intensity curve Lc = light intensity curve Oc = colour curve of object

35 Computer Graphics Primer Ambient Lighting

36 Computer Graphics Primer Diffuse Lighting Also known as Lambertian reflection considers the angle of incidence of light on surface (angle between light and surface normal) Result: lighting varies over surface with orientation to light

37 Computer Graphics Primer Diffuse Lighting

38 Computer Graphics Primer Specular Lighting Direct reflections of light source off shiny object specular intensity n = shiny reflectance of object Result: specular highlight on object

39 Computer Graphics Primer Specular Lighting

40 Computer Graphics Primer Combined Lighting Models R c = w a (ambient) + w d (diffuse) +w s (specular) for relative weights w a, w d, w s also specular power n

41 Computer Graphics Primer Computation of lighting at the points over the surface depends on the shading model Flat Shading (once per polygon) Gouraud shading ( for all the vertex of the polygon) Phong Shading ( all the points)

42 Computer Graphics Primer Phong shading is good but computationally costly Flat shading is easy but results are too bad Gouraud shading is usually used for simple applications

43 Computer Graphics Primer Cameras: Position, Focal point, Direction of projection, Roll Elevation, Azimuth, Yaw, Pitch, View up, View plane normal, front and back clipping plane.

44 Computer Graphics Primer View Frustrum 3D space viewable to camera bound by clipping planes (front, back); by camera view angle (top, bottom) clipping planes eliminate data that is too near or too distant from camera

45 Model Computer Graphics: Coordinate Systems local coordinate system World Where the models are placed View Logical Image Plane Display X,Y Pixel locations

46 Coordinate System

47 Computer Graphics Primer Coordinate systems:

48 Computer Graphics Primer Coordinate systems: Homogeneous coordinates

49 Computer Graphics Primer Graphics primitives: Polygon, Triangle strip, Line, Polyline, Point Typical graphics interface hierarchy Your program Application Library-vtk Graphics Library Graphics Hardware Display Hardware

50 Computer Graphics Primer Rasterization /Scan conversion Z-buffer

51 Computer Graphics Primer The graphics model of VTK: Seven basic objects vtkrenderwindow: manages a window on the display device; one or more renderers draw into an instance of vtkrenderwindow. vtkrenderer: coordinates the rendering process involving lights, cameras, and actors. vtklight: a source of light to illuminate the scene vtkcamera: defines the view position, focal point, and other viewing properties of the scene.

52 Computer Graphics Primer The graphics model of VTK: Seven basic objects (cont.) vtkactor: represents an object rendered in the scene, both its properties and position in the world coordinate system. vtkproperty: defines the appearance properties of an actor including color, transparency, and lighting properties such as specular and diffuse. vtkmapper: the geometric representation for an actor. More than one actor may refer to the same mapper.

53 Examples Render a cone: Cone.cxx Creating multiple renderers: Cone2.cxx Introducing vtkrenderwindowinteractor: Cone3.cxx Properties and Transformations: Cone4.cxx

54 Data Representation in Visualization For (computer) visualization data is discrete (in representation) structured / unstructured (e.g. grid / cloud) of a specific dimension (1, 2, 3, n) Organizing structure (architecture) for data: Topology How things (data elements) fit together Geometry Particular values that create (instantiate) a particular object Data Attributes

55 Data: Discrete vs. Continuous Real World is continuous Data for visualization is discrete Computers are good in handling discrete data discrete representation of a real world Text example of y = x 2, and f(x,y) = x 2 - y 2 continuous To create digital / visual representation must sample points E.g., built in method for sampling over function However, difficult to visualize continuous shape from raw discrete sampling? Requires topology and interpolation

56 Topology and Geometry Topology: Just, the way the data points are put together As lines, polygons, etc. More formally: Relationships within the data invariant under geometric transformation (e.g., orientation, translation) i.e., the information which vertex is connected to which edge, which face is composed of which edges, etc. Transformation : Classes of transformations preserve certain properties E.g., affine transformations for computer graphics Geometry: Putting particular values in for the elements of particular topology Creating an instance of (or instantiating) a particular form Topology is invariant under geometric transformation

57 Interpolation & Topology For various reasons interpolation is important in visualization Storage efficiency Representing change over known (or ascribed) values Interpolation: Increasing resolution of a discrete representation by producing intermediate samples Use interpolation to shade whole cube: in example: producing intermediate color pixels over cube topology from discrete vertex samples Example: Vertices represent spatial temperature at the 8 discrete points? e.g. a color scale with blue(=cold) -> green -> red( = hot) How to visualize temperature field over whole cube?

58 Detail: Interpolate over a Rectangle Will the red color dot on the right lower corner affect the color of the point near the left top? What happens if we change the representation? Discrete data samples remain the same topology has changed => affects interpolation => effects visualization

59 Interpolation in Digital Images Some digital cameras use interpolation to produce a larger image than the sensor captured Also using the topology of data E.g. below: top, just increase size original pixels evident, bottom, interpolate From computer graphics Various smoothing (and antialiasing) algorithms, effectively interpolation, as used here

60 Topological Dimension, or, Dimensionality Data has an inherent topological dimension I.e., minimum number of independent continuous variables needed to specify the location inside the data Or, just, dimensionality Points : 0D, Lines, curves : 1D, Surfaces : 2D, Volumes : 3D Time dependent volumes : 4D and, n-dimensional

61 VTK Data Representation Data objects: structure + value A dataset

62 Dataset Dataset consists of 2 main components Structure of the data Structure gives spatial meaning to the attributes Value Attributes associated to particular parts of the structure Example: values = { blue, green, green, green, green, green, turquoise, red} attributes alone are meaningless without structure

63 Structure of Data Structure has 2 main parts Topology : determines interpolation required for visualization shape of data Geometry instantiation of the topology specific position of points in geometric space

64 Concrete Representation of Datasets Points specify where the data is known specify geometry Cells allow us to interpolate between points specify topology of points

65 VTK Cells Quite similar in all graphics applications Fundamental building blocks of shapes Defined by topological dimension 1,2,3d Specified as an ordered point list Primary or composite cells composite : consists of one or more primary cells

66 0 and 1 Dimensional Cell Types Zero-dimensional Vertex Primary zero-dimensional cell Definition: single point Polyvertex Composite zero-dimensional cell composite : comprises of several vertex cells Definition: arbitrarily ordered set of points One-dimensional Line Primary one-dimensional cell type Definition: 2 points, direction is from first to second point. Polyline Composite one-dimensional cell type Definition: an ordered set of n+1 points, where n is the number of lines in the polyline

67 Two-Dimensional Cell Types Triangle Primary 2D cell type Definition: counter-clockwise ordering of 3 points order of the points specifies the direction of the surface normal Triangle strip Composite 2D cell consisting of a strip of triangles Definition: ordered list of n+2 points n is the number of triangles Quadrilateral Primary 2D cell type Definition: ordered list of four points lying in a plane constraints: convex + edges must not intersect Pixel Primary 2D cell, consisting of 4 points topologically equivalent to a quadrilateral constraints: perpendicular edges; axis aligned numbering is in increasing axis coordinates Polygon Primary 2D cell type Definition: ordered list of 3 or more points constraint: may not self-intersect

68 Three-Dimensional Cell Types Tetrahedron Definition: list of 4 non-planar points Six edges, four faces Hexahedron Definition: ordered list of 8 points six quadrilateral faces, 12 edges, 8 vertices constraint: edges and faces must not intersect Voxel Volume pixel Topologically equivalent to Hexahedron constraint: each face is perpendicular to a coordinate axis 3D pixels

69 Example: Hexahedron Cell In fact each cell type represented by set/list of points From computer graphics, vertex list, etc. Definition: ordered list of 8 points six quadrilateral faces, 12 edges, 8 vertices constraint: edges and faces must not intersect

70 Attribute Data Information associated with data topology usually associated to points or cells Examples : temperature, wind speed, humidity, rain fall (meteorology) heat flux, stress, vibration (engineering) surface normal, color (computer graphics) Usually categorized into specific types: scalar (1D) vector (commonly 2D or 3D; ND in RN) tensor (N dimensional array)

71 Attribute Data : Scalar Single valued data at each location simplest and most common form of visualisation data

72 Attribute Data: Vector Data Magnitude and direction at each location 3D triplet of values (i, j, k) Also Normals vectors of unit length (magnitude = 1)

73 Dataset Types Defined by structure type: regular or irregular If there is a mathematical relationship between the point and cell positions it is regular if the points are regular, the geometry is regular if the cells are regular, the topology is regular Regular data can be implicitly represented saves storage and computation (e.g. grid based representation) Datasets with implicit topology Structured points (regular grid) Rectilinear grid Structured grid Datasets without implicit topology Unstructured grid (has topology) Polygonal data (might have topology) Unstructured points (does not have topology)

74 Regular: Structured Points & Rectilinear Grid Structured points Points & cells arranged in a regular grid axis aligned grid for line elements (1D), pixels (2D) or voxels (3D) e.g. images (2D), medical scanners (3D) regular topology and geometry uniform, equally spaced, axis aligned cells Rectilinear Grid Points and cells arranged in a regular grid topology is regular geometry is partially regular points are arranged along the axes but the spacing may vary e.g. log-scale

75 Semi-Regular : Structured Grid Regular topology Irregular geometry (completely) Surface triangulations are structured grids common topology = triangles Not all polygon surfaces have regular topology

76 Irregular: Unstructured Points and Grids Unstructured Points Points irregularly located in space No topology No structured geometry e.g. sparse measurements of temperature etc. Difficult to visualize e.g., unstructured point clouds Can be from 3D scanners Need surface reconstruction from unstructured points clouds topology recovery Unstructured Grid Both topology and geometry unstructured can range from 0D to 3D topologies general, flexible, inefficient to store e.g. finite element analysis Using higher resolution grids near where precise simulation is needed,rough grids for regions less important Same verticies might be both a structured (implicit topology) and unstructured grid

77 Polygonal Data - Rendering Essential representation for visualization (and computer graphics) Consists of points, lines, polygons graphics primitives irregular geometry irregular topology Usually irregular (modelled by humans, or sticking regular data together) used for rendering

78 Examples: Polygon.cxx SGrid.cxx Rgrid.cxx