Volume Illumination and Segmentation

Transcription

1 Volume Illumination and Segmentation Computer Animation and Visualisation Lecture 13 Institute for Perception, Action & Behaviour School of Informatics

2 Overview Volume illumination Segmentation

3 Volume Illumination Why do we want to illuminate volumes? illumination helps us to better understand 3D structure displays visual cues to surface orientation highlight significant gradients within volume

4 Light Propagation in Volumes Lighting in volume In ray-casting (the method in the last lecture) only transmission and emission considered can also: reflect light scatter light into different directions

5 Illumination of Volumes For every voxel ray intersects, need to consider: Light absorbed. Light emitted. Light scattered out of the ray. Light scattered into the ray.

6 Illumination of Volumes For every voxel ray intersects, need to consider: Light absorbed. Light emitted. Light scattered out of the ray. Light scattered into the ray.

7 Example : multiple scattering Light scattered multiple times to produce simulation of a cloud

8 Example : sub-surface scattering Very large statue Medium sized Small statue. Varying how light is scattered inside a surface affects perception hence useful in visualisation why? - prior visual experience, perception of distance?

9 Simple Tricks: Subsurface Scattering The more the light travels in a material, the more it gets scattered and absorbed Reduce the intensity of the light which has travelled longer

10 Volume Illumination -? Scattering is too costly so we usually do not take them into account when doing volume rendering But we still can add slight shadows to the volume by illuminating them

11 What are we illuminating? embedded (iso-) surface sharp gradients in the scalar value

12 Shading an Embedded isosurface Classify volume with a step function Use regular specular / diffuse surface shading Remember for lighting equations require illumination direction camera model (position) surface orientation need to calculate and store surface normal

13 Estimating the surface normal from the depth Use distance map to the iso-surface value Determine the threshold value Determine the surface voxels based on the threshold Compute the normal vectors based on centred difference method For example, if we sample the centre of the voxels, We can extract the normal vectors of the region where the scalar values are changing significantly, i.e. boundary of tissues

14 Result : illuminated iso-surface MIP technique Shaded embedded iso-surface. Surface normals recovered from depth map of surface

15 Illuminating Opacity (Scalar) Gradient Illuminate scalar gradient instead of iso-surface requirement : estimate and store gradient at every voxel Composite Shaded opacity gradient (shades changes in opacity)

16 Illumination : storing normal vectors Visualisation is interactive compute normal vectors for surface/gradient once store normal perform interactive shading calculations Storage : 2563 data set of 1-byte scalars ~16Mb normal vector (stored as floating point(4-byte)) ~ 200Mb! Solution : quantise direction & magnitude as small number of bits

17 Illumination : storing normal vectors Quantize vector direction into one of N directions on a sub-divided sphere Subdivide an octahedron into a sphere. Number the vertices. Encode the direction according to the nearest vertex that the vector passes through. For infinite light sources, only need to calculate the shading values once and store these in a table.

18 Overview Volume illumination Segmentation

19 Medical Segmentation Segmenting the data into different tissues The data can be either volume / image

20 Segmentation of CT imaging CT value corresponds to density easier to segment Sometimes, you might want to segment tissues of the same CT values Segmenting muscles

21 How to segment MRI data? A Dog s thorax Lungs Trachea Liver Gall Bladder How to extract an organ here? MRI values are not directly proportional to density

22 Image Processing on MRI Data From image processing / computer vision segment image slices noise removal (smoothing) edge detection (changes in colour) Region growing thresholding all scalar values in between upper and lower limits Liver outlined in red Image processing not strictly visualisation additional tool in visualisation pipeline see Advanced Vision course

23 Region Growing start from initial image patch grow outwards including similar regions stop when boundary or distinct change in value reaching

24 Example : MRI segmented knee Courtesy : Brigham & Women's Hospital Original MRI slice (top, left), Segmentation (top, right), Overlay (bottom).

25 Segmentation of dog organs Liver outlined in red Segmentation of organ in each image slice (2D) 3D model of Liver. visualisation of segmented 2D image stack as 3D volume

26 Triangulated 3D model of dog liver Vertices in one contour need to be matched with those in the next slice to produce triangulated mesh. Problem - different number of vertices in each curve

27 Problem 1 : Grow hole High slice Middle slice

28 Problem 2 : split contours Middle slice Low slice Known as the branching problem.

29 Possible Ambiguities OR OR [Geiger '93] Contour connection method needs to handle these cases Solution: NUAGES [Geiger '93] fill convex hull of contour with tetrahedra discard those outside the contours or with only one edge on each level 3D contour connection = remaining exterior tetrahedra faces

30 NUAGES : Triangulate Contour Triangulate contour vertices using Delaunay Triangulation no inside / outside constraint forms planar convex hull of each contour [Geiger '93]

31 Delaunay triangulation Input : vertices Output : triangles composed by the vertices The triangulation DT(P) of P such that no point in P is inside the circumcircle of any triangle in DT(P) maximizes the minimum angle of all the angles of the triangles in the triangulation Gives nice set of triangles for finite element analysis Connecting the centres of the circumcircles produce the Voronoi diagram

32 NUAGES : Form Tetrahedra Form tetrahedra by joining closest vertex on opposite contour [Geiger '93] tetrahedra t1 and t2 formed with closest opposite vertex enclosed within area of triangle The vertex on the opposite side that is closest to the circumcenter is selected tetrahedra t12 formed by two edges that cross Vertices are added if necessary

33 NUAGES : Form Tetrahedra Form tetrahedra by joining closest vertex on opposite contour [Geiger '93] t1 : A triangle on Plane 1 and point on t2 t2 : A triangle on Plane 2 and point on t1 t12 : connecting two edges each on t1 and t2

34 NUAGES : Remove Tetrahedra [Geiger '93] Remove : Tetrahedra with an edge outside of the contour. Remove : Non-solid ones Non-solid : tetrahedra not connected to t1 nor t2

35 Problems The closest point on the other side might be too far away We insert points by projecting the Voronoi skeleton of the top contour to the bottom

36 Voronoi diagram We are given a set of points P. Each point p has a Voronoi cell V(p). V(P) consists of all points closer to p than any other points in P Created by connecting the center of circumcircles of the triangles produced by Delaunay triangulation

37 Creating skeletons by the Voronoi diagrams By connecting the edges of the Voronoi diagram of the points sampled over a closed polygon, we can produce its skeleton The skeleton inside the polygon is called internal Voronoi skeleton (IVS), and that outside is called external Voronoi skeleton (EVS)

38 Whether we insert points We check if the IVS of one side intersects with the EVS of the other side If they do, the EVS is projected to the other side and points are sampled over it

39 NUAGES : Results Upper Contour Connected : top down view Lower Contour Connected : side view Use exterior triangles of remaining tetrahedra to form outside boundary of the shape [Geiger '93]

40 NUAGES : Results Multiple medical image slices joined by NUAGES

41 Dog Liver : Final Model Completed model despite presence of contour holes/ branches (NUAGES)

42 Fully automatic segmentation is difficult although you apply all great computer vision techniques Segmentation is a low level procedure Based on colour information But users have high level knowledge

43 Volume Catcher We need to give some knowledge to the system A good user interface can improve the efficiency of segmentation Sketch-based interface, Owada et al. I3D

44 Specifying the Region of Interest The user specifies which area he/she is interested in The system finds out the color information inside and outside the volume data The color information is used for region growing / segmentation

45 Contour-based Interface for Refining Volume Segmentation An interface to refine the contours directly in 3D

46 Summary Subsurface Scattering Volume Illumination Segmentation Readings Jensen et al. A Practical Model for Subsurface Light Transport, SIGGRAPH Marc Levoy, Display of Surfaces from Volume Data, IEEE Computer Graphics and Applications, Vol. 8, No. 3, May, 1988, pp S Owada, et al. Volume Catcher, I3D 2005 Takashi Ijiri and Hideo Yokota: Contour-based Interface for Refining Volume Segmentation Computer Graphics Forum, 29(2007), 7, (Pacific Graphics2010)