5. Feature Extraction from Images

Transcription

1 5. Feature Extraction from Images

2 Aim of this Chapter: Learn the Basic Feature Extraction Methods for Images Main features: Color Texture Edges

3 Wie funktioniert ein Mustererkennungssystem Test Data x i Training Data Feature Extraction Feature Extraction Classifier Model Training Algorithm. n 3

4 5.. Histograms and Color Features 4

5 Color Histogram Calculate percentage of color present in image Deficiency: loss of regional information 5

6 Measurements at Pixels An image I is a set of pixels At each pixel: Measure some m-dimensional property Example: Each pixel of an RGB image is a 3-dimensional vector Formally: f I : R RI M RI m

7 Create a finite Partition of M Create finite partition of M: M K B k k B k are subsets of M B k are called bins and k is the label of the bin

8 Example Example: Let M be the grey levels of an image M [0: 55] Label of bin Gray levels B k

9 From Bins to Histograms Indicator function: b k ( x) 0 if f I ( x) B otherwise k x is element of the image

10 From Bins to Histograms A histogram is a vector ( h,... h K ) In words: with h k x R b k ( f I ( x)) dx dx for each bin: count which fraction of pixels falls into that bin. x R

11 Histogram Distances Motivation: measures the similarity of - images - speech - music Issue: how to capture perceptual similarity

12 Histogram Distances L distance (Manhattan distance) d K ( H, L) h k l k k L distance (euklidian distance) d K ( H, L) h k l k k L distance (maximum distance) d ( H, L) max( hk lk k )

13 Exercise 3

14 Example for potential problem with histogram distance

15 Example for potential problem with histogram distance 5

16 Distances of the three checkerboard Distance type d(a,b) images d(a,c) L ~0.67 L ~0.8 ~0.47 L ~0.33 ~0.33 None of the distances captures perceptual similarity 6

17 Realistic example for problem with distances 7

18 Realistic example for problem with distances Creating distance measures that capture the human notion of similarity is difficult. 8

19 Potential problem with histogram distance There are alternative distance measures Details beyond the scope of this lecture If you seem to have such a problem: look into the literature 9

20 Issue: loss of regional information Partition the image One histogram per region

21 5.3. Texture Features

22 What s in the image?

23 What is texture? Texture has no precise definition. Texture is a tactile or visual characteristic of a surface. Texture primitives (or texture elements, texels) are building blocks of a texture. Texel: A small geometric pattern that is repeated frequently on some surface resulting in a texture.

24 Use of Texture Analysis Segment an image into regions with the same texture, i.e. as a complement to gray level or color Recognize or classify objects based on their texture Find edges in an image, i.e. where the texture changes shape from texture object detection, compression, synthesis 4

25 Difficulties of Texture Analysis Which scale to use?

26 Texture Analysis Generic research area of machine vision Topic of research for over three decades Aim: to find a unique way of representing the underlying characteristics of textures and represent them in some simpler but unique form, so then they can be used to accurately and robustly classify and segment objects. 6

27 Types of Texture Strong Texture spatial interactions between primitives are somewhat regular frequency of occurrence of primitive pairs in some spatial relationship used for description Weak Texture small spatial interactions between primitives frequencies of primitive types appearing in some neighborhood used for description Two basic texture description approaches: syntactic statistical 7

28 Syntactic texture description Not used as widely as statistical approach Analogy between texture spatial relationships and structure of a formal language. Grammar representation - primitives are terminal symbols, relationships are represented as transformation rules. 8

29 Syntactic Approaches Shape Grammars G = <T,N,R,S> T : Primitive shapes R : Rules showing how elements can be composed S : Start symbol N : Non terminals 9

30 Example of Shape Grammar T = { } N = {. } Need a few additional rules for completion. 30

31 Comments Syntactic approaches are often not practical Difficult to model Difficult to compute Natural textures are complex!!! 3

32 First Order Statistics Mean K k k p k (hardly a useful feature) Variance k K ( k ) p k Why? Skewness K 3 3 ( k ) 3 k p k with Kurtosis 4 4 k K ( k ) 4 p k 3 p k h k K k h k

33 Example for first Order Statistics Brickwall texture Mean 79.3 Variance 4.73 Skewness.37 Kurtosis 5.93 Granite texture Mean Variance Skewness Kurtosis 3.5

34 Autocorrelation Function ff ( i, j) x y f ( x, y) f ( x i, y j) What is the frequency of repetition of structures: coarse/fine texture How strongly are they correlated: Similarity of the texels 34

35 Autocorrelation: Example 35

36 Autocorrelation: Example 36

37 Gabor Filter Family of filters Product of Gaussian with traveling waves 37

38 Gabor Filter Base filter x y G00( x, y) exp cos( x) x y x y Bank of Gabor Filters G mn with x' y' and m ( x, y) a G00( x', n a m cos sin n / K n n y') sin cos n n x y

39 Position of Frequencies for a Set of Gabor Filters Maple script 39

40 Gabor Filter For each filter of the set, a filtered image is calculated w mn ( x, y) a, b G mn ( x a, y b) I( a, b) Features for image processing: Mean and variance of the filtered images w mn (x,y) 40

41 Feature Extraction A look at gabor filter extraction for a 0 degree filter From: 4

42 Gabor filters, a D example from

43 5.. Edge Information 43

44 Characterizing Edges Images are discrete functions indicating the light intensity of a scene What happens at an edge?

45 Characterizing Edges (cont d) Let s look at one line for now

46 Detecting Edges Edges correspond to large changes in the image How do we detect such changes? 46

47 Gradient Definition I( x, y) The gradient is a vector with magnitude in the u and v directions equal to the respective partial derivatives How do we compute the partial derivative of a discrete function? I x xˆ I y yˆ 47

48 Taylor Series f ( x h) f ( x) hf '( x) h f ''( x)... or f ( x h) f ( x) hf '( x) h f ''( x)... Subtracting the second from the first we obtain f '( x) f ( x h) f h ( x h) O( h ) 48

49 Discrete Gradient Estimation Discrete functions: use first order approximation of the gradient h corresponds to the step size Images: h corresponds to the width of pixel => h h x f h x f x f ) ( ) ( ) '( ), ( ), ( ), ( ), ( ), ( ), ( y x I y x I y y x I y x I y x I x y x I 49

50 Discrete Gradient as a Linear Filter Gradient can be written as a linear filter Drop factor because it just scales the image I x I * 0 I y I * 0 50

51 Taking the discrete derivative [ 0 ] abs()

52 Basic Edge Detection Step I( x, y) INPUT IMAGE Horizontal [- 0 ] ) Edge Enhancement Vertical [- 0 ] T I ( x, y) x I ( x, y) y Issue: sensitivity to noise?

53 Basic Edge Detection Steps - I( x, y) INPUT IMAGE 4 /6 Horizontal [- 0 ] ) Edge Enhancement I ( x, y) x ) Noise Smoothing Vertical [- 0 ] T I ( x, y) y

54 Discrete Gradient Estimation Gradient is a vector we have calculated the coefficients in the x and y directions at each point in the image After convolving, we get the magnitude of the gradient from at each point (pixel) from G( x, y) I x I y In practice, we often sum the absolute values of the components for computational efficiency 54

55 Basic Edge Detection (cont d) I( x, y) INPUT IMAGE 4 /6 Horizontal [- 0 ] I ( x, y) x ) Noise Smoothing ) Edge Enhancement Vertical [- 0 ] T I ( x, y) y I( x, y) I( x, y) x I( x, y) y GRADIENT IMAGE

56 Thresholding Remove lighting effects Convert to binary image using a threshold Results from threshold values of 50 and 00

57 Basic Edge Detection Summary I( x, y) INPUT IMAGE 4 /6 Horizontal [- 0 ] ) Edge Enhancement I ( x, y) x ) Noise Smoothing Vertical [- 0 ] T I ( x, y) y 3)Threshold I( x, y) I( x, x y) I( x, y y) EDGE IMAGE GRADIENT IMAGE

58 The effects of Filtering Noise Threshold 0 Threshold 50 Unsmoothed Edges Gaussian Smoothing

59 Sobel Edge Detection Integrate smoothing and gradient calculation Sobel operators: widely spread scheme Sobel V Sobel H Convolving generates horizontal and vertical gradient images

60 Other Edge Detectors Prewitt: similar to the Sobel, but different kernel 0 P V 0 P H Canny edge detector

61 Other Edge Detectors Roberts: early edge detector kernel 0 0 R 0 R 0 Very sensitive to noise Very fast

62 Summary Color features: use histograms issue: robust distances measures Texture: first order statistics auto correlation function Gabor filter 6

63 Summary Edges correspond to abrupt changes in image intensity Edges can be detected by Smoothing out image noise Estimating the gradient of the image at every point to generate a gradient image Thresholding the gradient image