So what is visualisation? UG4 / M.Sc. Course 2006 toby.breckon@ed.ac.uk Computer Vision Lab. Institute for Perception, Action & Behaviour Introducing 1 Application of interactive 3D computer graphics to the understanding and communication of data. visual data analysis interactive viewing, understanding and reasoning process Conversion of numbers images intelligent amplification [Brooks '94] humans are generally poor at raw numerical data analysis human visual reasoning allows robust analysis of visual stimuli convert numerical analysis into visual analysis Introducing 2 OED Definition Question? Q: So how can we, 1. The action or fact of visualizing; the power or process of forming a mental picture or vision of something not actually present to the sight; a picture thus formed. as computer scientists, make information 2. The action or process of rendering visible. (or knowledge) Oxford English Dictionary http://www.oed.com visible? Introducing 3 Introducing 4
Simple Example : Maps Simple Example : Maps Ben Nevis Scotland's Highest Mountain (1343m / 4409ft) Maps: Crown copyright Ben Nevis visualisation of 3D satellite data http://earth.google.com Ben Nevis Fly Through: http://www.ordnancesurvey.co.uk Contours represent changes in height on a 2D map so what is the shape of this famous mountain? Introducing 5... but with 3D graphics we can represent the shape of the mountain directly we can improve the visualisation of this height data by viewing it in 3D Introducing 6 The scientific process The visualisation process the creation of knowledge Knowledge the effective presentation of knowledge Observations (Scientific papers, business strategy, a medical diagnosis etc). Some analysis? (could be scientific, medical or business sales figures etc). Knowledge Observations (Scientific papers, business strategy, a medical diagnosis etc). algorithms (could be scientific, medical or business sales figures etc). Interaction Human expert 3D Computer Numbers Human expert Introducing 7 Graphics Numbers Introducing 8
Visual Data Analysis Computer presenting data in a convenient and efficient visual form for human visual analysis Symbolic into the geometric Strengths of computing as a visualisation tool: [McCormick/Defanti/Brown 1987] numbers are just symbols (representing data) images have geometry (2D or 3D) translation of relationships (structure) within the data (i.e. topology) into image geometry allows the visualisation of abstract data in the image domain multi dimensional data temporal data (suitable for animation) human computer interaction As for computer analysis see AI (e.g. computer vision), here we concentrate on computer based data presentation 3D computer graphics Introducing 9 visualisation is an interactive process Data storage / processing large amounts of data (fast, random access) data transformation This course: data representation & transformation for visualisation Introducing 10 Example : 3D scan data Example: Multi Dimensional 3D visualisation of: Source: http://www.ssec.wisc.edu/~billh/vis5d.html terrain (position x,y,z, type = colour) temperature, wind speed (rear and orthogonal display) Introducing 11 Source: Vesalius Project, http://cpmcnet.columbia.edu/vesalius/ 3D 'Virtual dissection': Bones and muscles of the pelvis 3D models made from many 2D photos of slices. Introducing 12
Presentation Graphics Existing knowledge for presentation Audience Existing knowledge Observed (The public, students, senior management, etc.) data Audience (The public, students, senior management etc.) Interaction Human expert Presentation methods Presentation Human expert media Introducing 13 algorithms Presentation media Introducing 14 Example: Basic Presentation 2D Colour Photograph and corresponding depth information Example : human viewer left to correlate the data Introducing 15 3D projection of combined data data relationship = pixel location in 2D images transformation = 3D projection Representation = 3D mesh Introducing 16
Example : Vs. Presentation 3D projection of combined data colour removed, 3D shape only use of colour to show relative elevation of features The extraction of knowledge from numbers from results of computations eg. simulation from observations eg. From satellites or experiments Graphically exploring numbers in order to gain understanding and new knowledge exploration rather than simple presentation exploration = an interactive process Different from presentation graphics Facilitating the generation of new knowledge (via effective presentation) rather than presenting existing knowledge Introducing 17 Further reading: http://www.siggraph.org/education/materials/hypervis/visgoals/visgoal2.htm Introducing 18 Features of : 1 Multi dimensional Data Features of : 2 data with 3 or more dimensions (ℛN, N >= 3) Interaction user driven process user does analysis requires consideration in data representation Many established methods for 2D/1D data signal processing / graphing (1D) image processing (2D) computer does visualisation what does this data show? graphically semantically http://www.paraview.org/ Introducing 19 Unlike numerical analysis of statistics the user does the analysis Introducing 20
Features of : 3 Data transformation data is repeatedly created and modified to enhance meaning to the user 202.0000000000 202.5000000000 202.5000000000 209.0000000000... -221.0050048828-220.5050048828-221.0050048828-296.5029907227-170.1450042725-169.8619995117-169.7920074463-165.5749969482 Features of : 4 Flow of data Raw 3D points from a range scanner focus on the flow of data in the visualisation pipeline Also well suited time varying (temporal) data Projected 3D point cloud e.g. weather, financial prices, air flow use of animation in visualisation visualisation of flow Triangulated 3D surface + associated depth colouring http://www.paraview.org/ Introducing 21 Introducing 22 What's in this course? Data representation Object oriented implementation in VTK. 3D graphics are our tool in visualisation new technology (in terms of available power to the everyday user) utilised in engineering, architecture for visualisation humans alone are poor at 3D visualisation and spatial reasoning using visualisation as a tool Surface, Volume & Image based techniques especially multi dimensional data Basic principles of visual understanding Process oriented structure of visualisation systems Focus on 3D Graphics 3 key areas of visualisation Methods of acquiring 3D data Introducing 23 Computer Graphics, and associated techniques, offer the most potential from computer science in terms of aiding in the visualisation problem Practical problems of visualising and acquiring 3D data e.g. imagine a new building from the architectural plans alone unique selling point of computer based visualisation Introducing 24
& Computer Vision is data orientated Process Input Output Computer Vision Algorithms Computer Vision (CV): computer analysis of visual information 3 main sources of visualisation data: automated visual reasoning : computer presentation of that visual information Scientific / Medical visualising results of simulations, experiments or observations Frequently data is multi dimensional Information of abstract, usually discrete data e.g financial data, web site hits etc. Real world (Virtual Reality) 3D capture technologies are reaching maturity commonly part of the output process in the CV pipeline (especially 3D capture) improve spatial awareness of 3D objects also used for evaluation in CV (i.e. to provide insight into CV process itself) virtual environments (e.g. remote viewing) Introducing 25 Introducing 26 Example : Medical Imaging Computer imaging in medicine: Computed Tomography (CT) imaging uses pencil thin X ray beams Example : Medical Imaging Computer Aided Tomography (CAT) Magnetic Resonance Imaging (MRI) uses large magnetic fields with pulsed radio waves Chest CT section Introducing 27 Numbers from scanners X ray attenuation (CT) Nuclear spin magnetization (MRI) Large amounts of data Difficult to interpret Magnetic Resonance Image showing a vertical cross section through a human head. Collect data as slice planes from patient fairly abstract physics based concepts (for a computer scientist or medical practitioner!) 2D images from CT data slices Introducing 28
Why do we need visualisation? Humans are good at inferring 3D shape from conventional 2D images (i.e. pictures) 3D surfaces from CT slices based on evolution of human visual system Many planar slices (2D) can be combined (in topological order) to form a 3D volume of data But... poor at performing simple 3D reasoning tasks: e.g. the shape formed by slicing a simple 3D solid many tasks in the modern world require advanced 3D reasoning e.g. design (e.g. engineering / architecture) e.g. mobility (e.g. sport / driving) abilities in these areas vary greatly Introducing 29 i.e. stack of 2D images Volume can be processed and rendered to reveal complete anatomical structures Introducing 30 Course Syllabus Data Representation Data geometry and topology Flow visualisation Data dimensionality Vector visualisation Surface and volume representations Tensor visualisation Systems architecture for The Pipeline Fundamental algorithms Information visualisation Advanced algorithms Networks and trees, documents Active research topics 2 & 3D Contouring, Colour mapping of real objects & environments Volume rendering Acquisition of 3D data 18 Lectures lecture notes on line (http://www.inf.ed.ac.uk/teaching/courses/vis/) background reading (mainly on line, also textbook) 2 Assessed Practicals 2 programming tasks Toolkit VTK prior weekly practicals introducing VTK Assessment 1.75 hour examination (70%) 2 practical assignments (15% each) Introducing 31 Course Outline (variation between UG4 and M.Sc. requirements) Introducing 32
Textbooks Non essential VTK The Toolkit majority of VTK information on line No single book covers all of the course (although in terse form) Maybe of use: Schroeder/Martin/Lorensen The Visualization Toolkit : An Object Orientated Approach to 3D Graphics 2 or 3 Edition. 3 Edition ISBN 1 930934 12 2 nd rd VTK is a C++ library (toolkit) that implements: visualisation data structures visualisation algorithms common data interfaces (import / internal / export) visualisation pipeline visual output in OpenGL rendering rd Available in Library (JCMB). Limited availability in UK. Amazon marketplace ~20+ Schroeder/Martin/Lorensen The Visualization Toolkit User's Guide Most recent edition ISBN 1 930934 13 0 Available in Library (JCMB). Limited availability in UK. http://public.kitware.com/vtk/buy books.php / http://www.amazon.com (expensive shipping) Introducing 33 Additional features interfaces in TCL, python & Java (N.B. Java via JNI) open source and platform independent Introducing 34 VTK Practicals / Software All practicals in VTK Summary assessed assignments (use TCL, Java or C++) TCL advised and will be presented next lecture Java / C++ will have limited support The visualisation process Vs. presentation Features of the visualisation problem weekly non assessed practicals ~20 minutes per week based in TCL Software installed on DICE available for home use: http://www.kitware.com/vtk pre build linux RPMs (from DICE) see course homepage Introducing 35 data orientated problem offered solution: interactive, 3D computer graphics Why visualise? VTK introduction Additional reading on line Next lecture: computer graphics as our visualisation tool VTK Introducing 36