Multimedia Retrieval Ch 5 Image Processing. Anne Ylinen

Transcription

1 Multimedia Retrieval Ch 5 Image Processing Anne Ylinen

2 Agenda Types of image processing Application areas Image analysis Image features

3 Types of Image Processing Image Acquisition Camera Scanners X-ray imagers Computer tomography (CT) Magnetic resonance scanners (MR) Ultra sound devices (US)

4 Types of Image Processing Image Restoration Geometric distortions Noise Unsharpness

5 Types of Image Processing Image Reconstruction using models different viewpoint another imaging device

6 Types of Image Processing Image Enhancement Contrast enhancement amplitude scaling contrast modification Histogram normalization nonadaptive histogram modification adaptive histogram modification Edge enhancement linear edge crispening statistical differencing

7 Types of Image Processing Image Registration Rigid registration Non-rigid registration Used in medical applications, cartography, face recognition, etc.

8 Types of Image Processing Image Compression, Storage and Transmission Lossless image can be exactly reconstructed Lossy approximate reconstruction

9 Types of Image Processing Image Analysis Image analysis aims to generate a description of the image or of objects present in the image.

10 Application Areas Medical Imaging MR, CT, US Geo Information Systems, Satellite, Aerial photography and Cartography Biometry Face and fingerprint recognition, handpalm recognition, tracking people feature-based and holistic approaches Optical Character Recognition Industrial Vision Multimedia and Image Databases

11 Image Analysis extract information from an image detection classification parameter estimation structural analysis

12 Image Analysis observed data feature extraction observed features comparison model features match criterion model selection selected model database of models

13 Image Analysis Image analysis task the selection of the features the representation of the models the matching criterion the selection strategy

14 Image Features Image -dimensional signal represented by a matrix F of pixels of N rows and M columns A pixel value f(n,m) ) is an intensity or a vector of 3 RGB components mathematical operations are possible e.g. derivative and Fourier transformation

15 Image Features Pixel Features Neighborhood and Image filtering each pixel an individual feature neighboring pixels grouped together used to obtain higher level features

16 Image Features Scale space and derivatives scale at which objects are seen in an image depends on the distance between object and camera scale space theory for handling image structures at different scale derivatives important for edge detection, point feature detection, and so on

17 Image Features Texture small elementary pattern repeated periodically or quasi- periodically geometric or radiometric pattern important clues for segmenting the image typified by the distance over which the patter is repeated the direction in which the pattern is repeated the properties of the elementary pattern co-occurrence occurrence matrices

18 Image Features Point Features Interest points corner points and spots video tracking, stereo matching, object recognition Harris corner detector

19 Image Features Harris corner detector image I(x,y) ) and sifted image I(x+u, y+v) Gaussian window function w(x,y) ( u, v) E(u,v) ) should change fast for small sifts of (u,v( u,v) E au + bv + cuv E( u, v) = w( x, x, y y) [ I( x + u, y + v) I( x, y) ]

20 Image Features Image Features 1 1 1, det ) ( det M eigenvalues of,λ λ ), ( ], [ ), ( λ λ λ λ + = = = = tracem M tracem k M R I I I I I I y x w M where v u M v u v u E y x y y x y x x

21 Image Features R depends only on eigenvalues of M λ Edge R < 0 Corner R is large for a corner R > 0 R is negative with large magnitude for an edge R is small for a flat region Flat R small Edge R < 0 sourse( iantfeatures.pptes.ppt ) λ 1

22 Image Features Line elements line segments have a width in the image equal to the scale of the image, Gaussian like profile across the line calculate the second derivative in the direction orthogonal to the gradient vector more stable result is obtained by approximating the neighborhood of each candidate line element by quadratic surface: (n,m)) is the position of the candidate line element f ( n k, m l) f ( n, m) + ak + bl + ckl

23 Image Features using Taylor expansion f ( n k, m l) f ( n, m) + where H = f f xx xy f f xy yy [ k l] k H l λ 1,λ are eigenvalues of H for true line element, one eigenvalue should be large and the other small

24 Image Features Edge elements stepwise transition in intensities neighboring edge elements linked to gether form an edge segment gradient is large at the position of an edge Gradient-based methods Laplacian-based methods Canny s method

25 Image Features Canny s method 1. Smooth the image with Gaussian filter g(x,y)= )=g c (x,y)*f(x,y f(x,y) where g c ( x, y) 1 x + y exp σ π σ = where σ represents the width of the Gaussian distribution. Compute the second derivative in the gradient direction g gx gxx = n 3. Find zero crossings of the second derivative + g g x x g y + g g xy y + g y g yy

26 Image Features Pros: One pixel wide edges Edges are grouped together (often good for segmentation) Robust against noise! Cons: Complicated to understand and implement Slow

27 References Blanken et al, Multimedia Retrieval, 007, Springer Pratt, W: Digital Image Processing, 001, John Wiley & Sons INC Bovik,, A: Handbook of Image & Video Processing, 000, Academic Press Castelman,, K: Digital Image Processing, 1996, Prentice Hall Harris, C: A Combined Corner and Edge Detector, 1988,