Lecture 10: Image Descriptors and Representation

Transcription

1 I2200: Digital Image processing Lecture 10: Image Descriptors and Representation Prof. YingLi Tian Nov. 15, 2017 Department of Electrical Engineering The City College of New York The City University of New York (CUNY) Thanks to G&W website, Dr. Shahram Ebadollahi for slide materials 1

2 Image Representation and Description? Objective: To represent and describe information embedded in an image in other forms that are more suitable than the image itself. Benefits: Easier to understand Require fewer memory, faster to be processed More ready to be used 2

3 Outline Image Description Shape Descriptors Region Descriptors Texture & Texture Descriptors

4 Shape Description Shape Represented by its Boundary Shape Numbers, Fourier Descriptors, Statistical Moments Shape Represented by its Interior Topological Descriptors Moment Invariants 4

5 Boundary Representation: Chain Code Why boundary? The boundary is a good representation of an object shape and also requires a few memory. Chain codes: represent an object boundary by a connected sequence of straight line segments of specified length and direction. 5

6 Boundary Representation: Chain Code Object boundary (resampling) Boundary vertices 4-directional chain code 8-directional chain code 6

7 The First Difference of a Chain Codes Problem of a chain code: a chain code sequence depends on a starting point. Solution: treat a chain code as a circular sequence and redefine the starting point so that the resulting sequence of numbers forms an integer of minimum magnitude. The first difference of a chain code: counting the number of direction change (in counterclockwise) between 2 adjacent elements of the code. 7

8 The First Difference of a Chain Codes: Example 2 Example: Example: - a chain code: The first difference = Treating a chain code as a circular sequence, we get the first difference = Chain code : The first difference (counterclockwise) The first difference is rotational invariant. 8

9 Chain Code: example (First difference of chain code) 9

10 Boundary Descriptors 1. Simple boundary descriptors: we can use - Length of the boundary - The size of smallest circle or box that can totally enclosing the object 2. Shape number 3. Fourier descriptor 4. Statistical moments

11 Shape Numbers Shape number of the boundary definition: the first difference of smallest magnitude The order n of the shape number: the number of digits in the sequence

12 Shape Numbers Shape numbers of order 4, 6 and 8

13 Example: Shape Numbers 2. Find the smallest rectangle that fits the shape 1. Original boundary Chain code: First difference: Create grid 4. Find the nearest Grid. Shape No

14 Boundary descriptor Fourier Fourier descriptor: view a coordinate (x,y) as a complex number (x = real part and y = imaginary part) then apply the Fourier transform to a sequence of boundary points. Let s(k) be a coordinate s( k) x( k) jy( k) of a boundary point k : Fourier descriptor : a( u) 1 K K 1 k 0 s( k) e 2 uk / K Reconstruction formula Boundary points s( k) 1 K K 1 k 0 a( u) e 2 uk / K

15 Fourier Descriptor Properties

16 Boundary Reconstruction using Fourier Descriptors 2868 descriptors Only 8 descriptors 16

17 Polygon Approximation Represent an object boundary by a polygon Minimum perimeter polygon consists of line segments that minimize distances between boundary pixels. 17

18 Polygon Approximation: Splitting Techniques 1. Find the line joining 0. Object boundary two extreme points 2. Find the farthest points from the line 3. Draw a polygon 18

19 Boundary Representation: Signatures Represent 2-D boundary shape using 1-D signature signal 19

20 Boundary Representation: Signatures 20

21 Boundary Segments Concept: Partitioning an object boundary by using vertices of a convex hull. Partitioned boundary Convex hull (gray color) Object boundary 21

22 Skeletons Obtained from thinning or skeletonizing processes 22

23 Region Descriptors - Simple 23

24 Region Descriptors - Example White pixels represent light of the cities % of white pixels Region no. compared to the total white pixels % % 3 4.9% % Infrared image of America at night 24

25 Topological Region Descriptors Topological properties: Properties of image preserved under rubber-sheet distortions 25

26 Topological Descriptors: example Original image: Infrared image Of Washington D.C. area The largest connected area (8479 Pixels) (Hudson river) After intensity Thresholding (1591 connected components with 39 holes) Euler no. = 1552 After thinning 26

27 Boundary Description: Statistical Moments Definition: the n th moment where ( r ) n m K 1 i 0 ( r K 1 i 0 i m ) r g ( i r i ) n g ( r i ) Example of moment: The first moment = mean The second moment = variance Boundary segment 1D graph 1. Convert a boundary segment into 1D graph 2. View a 1D graph as a PDF function 3. Compute the n th order moment of the graph 27

28 Geometric Moment Invariants 28

29 Central Moments 29

30 Moment Invariants (translation, scale, mirroring, rotation) 30

31 Moment Invariants (translation, scale, mirroring, rotation) 31

32 Affine Transform & Affine Moment Invariants 32

33 Affine Transform & Affine Moment Invariants 33

34 Elliptical Shape Descriptors 34

35 Texture - Definition Purpose: to describe pattern of the region. 35

36 Texture Quantification Methods 36

37 Statistical Texture Analysis 1st order statistics 37

38 Texture Examples: optical microscope images: B C A Superconductor (smooth texture) Cholesterol (coarse texture) Microprocessor (regular texture) 38

39 Texture Example: The 2 nd moment = variance measure smoothness The 3 rd moment measure skewness The 4 th moment measure uniformity (flatness) A B C 39

40 Statistical Texture Analysis: 1st order statistics 40

41 Statistical Texture Analysis: 2nd order statistics: Co-occurrence 41

42 Statistical Texture Analysis: 2nd order statistics: Co-occurrence 42

43 Statistical Texture Analysis: 2nd order statistics: Co-occurrence 43

44 Statistical Texture Analysis (Example) 44

45 Statistical Texture Analysis (Example) 45

46 Statistical Texture Analysis 2nd order statistics: Difference Statistics 46

47 Statistical Texture Analysis 2nd order statistics: Autocorrelation 47

48 48

49 49

50 Fourier Approach for Texture Descriptor Concept: convert 2D spectrum into 1D graphs Original image FFT2D +FFTSHIFT Fourier coefficient image Divide into areas by angles Divide into areas by radius Sum all pixels in each area S( ) R 0 r 1 S r ( ) Sum all pixels in each area S( r) S ( r) 0 50

51 Fourier Approach for Texture Descriptor Original image 2D Spectrum (Fourier Tr.) S(r) S( ) Another image Another S( ) 51

52 Which descriptors? Image Feature Evaluation 52

53 Announcement Reading G&W Chapter 11 Next lecture: Object Recognition (Chapter 12) 53