Multi-dimensional Image Analysis

Transcription

1 Multi-dimensional Image Analysis Lucas J. van Vliet

2 Image Analysis Paradigm scene Image formation sensor pre-processing Image enhancement Image restoration Texture filtering segmentation Measurements: Point: edge location, isophote curvatures Global: size and shape descriptors Local: texture attributes anisotropy, orientation, scale analysis classification CBP course: Multi-Dimensional Image Analysis 2

3 Multi-dimensional analysis Goal: Use sampling-error free measurements for analog properties in digitized images Point measurements for object identification Exact boundary location (Principal) curvatures Global measurement for object description Integrated object intensity (gray-volume) Size in 2D: area and perimeter Size in 3D: volume, surface area, length Shape: Bending energy, Euler numbers Local texture analysis Anisotropy, orientation, scale Use gray-scale rather than binary operations to obtain high accuracy and precision CBP course: Multi-Dimensional Image Analysis 3

4 Better geometric measurements State-of-the-art cameras offer: Large images of 1000 x 1000 pixels or more bit photometric information A binary image is disturbed by aliasing Thresholding corrupts data irreversible Faithful representation Binary representation CBP course: Multi-Dimensional Image Analysis 4

5 Location of curved edges Zero-crossing of second derivatives applied to blurred curved edges are biased. Laplace: D xx + D yy = D cc + D gg outwards in grad direction: D gg inwards PLUS 2D gg + D cc on edge! Results on an ellipse: object with slowly varying curvature. PLUS y x g c g PLUS c c g g = gradient c = contour Laplace SDGD CBP course: Multi-Dimensional Image Analysis 5

6 Isophote curvature in 2D An isophote is a curve of constant gray-value (level curve) Curvature is the change of contour direction per unit length Curvature (κ=1/r) with R the radius of the osculating circle object contour κ = D D g cc g =(D x,d y ) c =(-D y,d x ) Usage: corner detection, dominant points, bending energy CBP course: Multi-Dimensional Image Analysis 6

7 Isophote curvature in 3D 2D isophote surface patch in 3D image space Principal curvatures κ 1 and κ 2 along c 1 and c 2 : c1 c2 g κ 1 κ2 0 > > κ2 < κ1 < 0 κ1> 0, κ2 = 0 κ > 0, κ < 0 κ1 = κ2 = κ 1 D D cc = g κ 2 = D D cc g Usage for local shape: elliptic (convex, concave), cylinder,saddle, flat CBP course: Multi-Dimensional Image Analysis 7

8 Sampling-error free measures A sampling-error free measurement is a digital measurement performed on a sampled image that is exactly equal to its analog counterpart sampling operation in digital space operation in analog space reconstruction Digital track = Analog track The sum of all samples is measured without thresholding and does not introduce a sampling error ( b) = b( i j ) = ( π ) B( ) sum, 2 0,0 iff : f sampling x y x y > 1 2 f Nyquist 2 CBP course: Multi-Dimensional Image Analysis 8

9 Sum() as measure for Sum() is a sampling-error free measure Recipe for object measurements in gray-scale images: 1. Transform the input image with the object into an output image whose sum() is directly proportional to the feature to be measured 2. The transformation must consist of sampling-error free operations 3. Proper sampling is required to avoid aliasing 4. Bias correction terms can be extracted from the mathematical framework! (not empirically) CBP course: Multi-Dimensional Image Analysis 9

10 2D area & 3D volume Clip image a to produce a flat object on a flat background Use soft-clipping to avoid aliasing output after: linear input slope thresholding hard-clipping erf-clipping b = clip erf ( a) A V 2D 3D = b = sum( b) CBP course: Multi-Dimensional Image Analysis 10

11 2D perimeter, 3D surface area Transform the flat, bandlimited object into a contour a Analytical dilation / erosion by Taylor series around a smooth edge (with height = 1) over a distance δ =½ 2 dilation : b 1 ( r + δ) b( r ) + δ b + δ b 2 erosion : b 1 ( r δ) b( r ) δ b + δ b contour δ= 1 2 : b g g g 2 2 gg gg b b g CBP course: Multi-Dimensional Image Analysis 11

12 Shape: 2D bending energy 2D bending energy proportional to the bending energy of a deformed circular rod (Young 74). Differential geometry Image analysis 2 2 E κ ( s) ds E κ b = = be be g The simply-connected, closed contour with the minimum bending energy is the circle (2π/R, not scale invariant). b b g κ κ 2 CBP course: Multi-Dimensional Image Analysis 12

13 Shape: 3D bending energy Elastic rods (SCC space curves) κ 1 cross-section of rod κ 2 trajectory of rod Deflected thin plates (SCC surfaces) principal curvatures κ 1 and κ 2 Erod = κ 2 ( s) ds 2 Differential geometry 2 2 M ( 1 2 ) E = 8πp+ κ + κ ds plate E rod = 2 κ2 bg ( 2 2 Image analysis E ) plate κ1 κ2 = + b g circle has minimal bending energy Torque forces are neglected Poisson s ratio p [0,½]. (let p = 0) sphere has minimal energy (8π). Dimensionless and therefore scaling invariant. CBP course: Multi-Dimensional Image Analysis 13

14 Length in 3D Resume: An erf-clipped object of unit intensity, b volume: V = Σ b surface area: A = Σ b g length: L =?? image b contains a cylinder (length L,radius R) Σ b = πr 2 L = volume Σ b g = 2πRL = area Σ b gg = 2πL = length Length of spaghetti L = b 1 3D 2π gg b b cc In the plane perpendicular to the string: g,c Σ b gg + b cc = 0 all g c = 0 Σ b gg = Σ b cc = 2π (see Euler) b gg CBP course: Multi-Dimensional Image Analysis 14

15 Shape: Euler numbers Euler numbers characterize the topology 2D: number of objects number of holes 3D: number of objects number of handles (donut & coffee cup have Euler number = 0) b Differential geometry Image analysis Hopf: L κ 1 ( s) ds = 2π N = κ b 2D 2π g b g Gauss- Bonnet: M κκ ds = 4π N = κκ b D 4π 1 2 g κ CBP course: Multi-Dimensional Image Analysis 15

16 Curvilinear structures 10 km 10 cm 10 cm 1 cm z = 10 µm Anisotropy Orientation Scale Curvature CBP course: Multi-Dimensional Image Analysis 16

17 Domains vs Scale Texture attributes are domain properties CBP course: Multi-Dimensional Image Analysis 17

18 Orientation and Scale Rotational invariant chirp image Orientation map by Gradient Structure Tensor Scale map by Gaussian Scale Space CBP course: Multi-Dimensional Image Analysis 18

19 Gradient Structure Tensor How to combine vectors? G 2 fx fxf t y = g g = 2 f f f x y y f y λ 1 f x λ 2 ϕ CBP course: Multi-Dimensional Image Analysis 19

20 GST: anisotropy, orientation Closed-form solutions for: ϕ 1 2ff xy = tan f 2 2 x fy ϕ λ λ ( ) ( ) 2 2 x y x y 4 x y = f + f + f f + f f ( ) ( ) 2 2 x y x y 4 x y = f + f f f + f f λ 1 λ 2 anisotropy: A λ λ = λ + λ A CBP course: Multi-Dimensional Image Analysis 20

21 PVC particle roughness Contour information (shape features) failed to rank batches according to quality Lobes show up as lines rather than points Measure the local anisotropy (ellipses) CBP course: Multi-Dimensional Image Analysis 21

22 Roughness = Integrated anisotropy Three scales: gradient, window, particle R λ2 = 1 = 0.44 λ 1 R λ2 = 1 = 0.61 λ 1 CBP course: Multi-Dimensional Image Analysis 22

23 Roughness distributions (Cumulative) distributions for N=30 particles can be used as roughness measure Roughness measure correlates well with product quality particle roughness histograms cumulative particle roughness histograms % 10 75% 8 6 count smooth middle 50% smooth middle rough roughness rough smooth middle rough roughness % 0% CBP course: Multi-Dimensional Image Analysis 23

24 Orientation-driven analysis Dominant orientation of Gradient Structure Tensor Strongest peak in orientation space Dominant orientation of GST yields the gradient 2 -weighted orientation map. Strongest orientation in ϕ-space allows very abrupt changes in orientation map. CBP course: Multi-Dimensional Image Analysis 24

25 Orientation space Decompose the image into narrow orientation bands Apply a nonlinear operator in orientation space: Labeling yields segmentation Peak selection for detection selectivity Φ Φ 1 Φ CBP course: Multi-Dimensional Image Analysis 25

26 Circle in image-space A circle in 2D image space becomes a double-helix in orientation space. Note that orientation-axis is periodic with π CBP course: Multi-Dimensional Image Analysis 26

27 Overlapping circles Two overlapping circles in 2D image space compose a single object. Since the circles cross at different orientations, they become separated in orientation space. CBP course: Multi-Dimensional Image Analysis 27

28 Orientation selectivity N=8 orientation N=16 Green = first peak, Red = second peak, Blue = remainder N=8 N=16 N=32 CBP course: Multi-Dimensional Image Analysis 28

29 Multi-Scale Series of images filtered of decreasing scale: Scale-space Sample the scales logarithmically using filters of size = base scale yields n scales per octave base 2,2,2,...,2 1 n Input image Scale space scale 0 scale 1 scale 2 scale 3 scale 4 scale 5 Scale difference scale derivative var (scale 1) var (scale 2) var (scale 3) var (scale 4) var (scale 5) { } Local variance between scales n and n-1. Scale space CBP course: Multi-Dimensional Image Analysis 29

30 Chirp example Scale derivative Spatial variance Scale-space Scale-space CBP course: Multi-Dimensional Image Analysis 30

31 Gaussian scale space Chirp of varying contrast Low-pass filters of increasing scale: color code (fine to course) Normalization: sum per pixel is constant CBP course: Multi-Dimensional Image Analysis 31

32 Scale analysis Scale information reveals geological structures in seismic data Absolute Normalized CBP course: Multi-Dimensional Image Analysis 32

33 Morphological scale space Measure the hole-size distribution The image acts like a sieve Subtract closings of increasing scale The hole-size is the scale that closes the gap 256 x Scale CBP course: Multi-Dimensional Image Analysis 33

34 Labeling of holes by size CBP course: Multi-Dimensional Image Analysis 34

35 Pore-size distribution Milk (blue) + substate (red) + enzyme (green) Milk Milk + substrate Milk + substrate + enzime C N=32 Average pore-size distributions images CBP course: Multi-Dimensional Image Analysis 35

36 Literature pdf files available at: L.J. van Vliet and P.W. Verbeek, Better geometric measurements based on photometric information, Proc. IEEE Instrumentation and Measurement Technology Conf. IMTC94 (Hamamatsu, Japan, May 10-12), 1994, G.M.P. van Kempen et al., The application of a local dimensionality estimator to the analysis of 3D microscopic network structures, in: SCIA'99, Proc. 11th Scandinavian Conference on Image Analysis (Kangerlussuaq, Greenland, June 7-11), 1999, M. van Ginkel et al., Improved Orientation Selectivity for Orientation Estimation, in: SCIA'97, Proc. 10 th Scandinavian Conference on Image Analysis (Lappeenranta, Finland, June 9-11), 1997, CBP course: Multi-Dimensional Image Analysis 36