Autoregressive and Random Field Texture Models

Size: px

Start display at page:

Download "Autoregressive and Random Field Texture Models"

Allison Morris
5 years ago
Views:

1 1 Autoregressive and Random Field Texture Models Wei-Ta Chu 2008/11/6

2 Random Field 2 Think of a textured image as a 2D array of random numbers. The pixel intensity at each location is a random variable. One can model the image as a function f(r,w), where r is the position vector representing the pixel location, and w is a random parameter. Once we select a specific texture w, f(r,w) is an image. f(r,w) is called a random field

3 Random Field Model 3 A typical random field model is characterized by a set of neighbors. Given an array of observations of pixel-intensity values {y(s)}, it s natural to expect that the pixel values are locally correlated. Markov model

4 4 Simultaneous Autoregressive Model (SAR) A special case of Markov random field

5 Multiresolution SAR (MRSAR) 5 It s not trivial to determine the appropriate size of the neighborhood. The MRSAR model tries to account for the variability of texture primitives by defining the SAR model at different resolutions. SAR SAR SAR Original image Image pyramid

6 6 Spectral Texture Features Wei-Ta Chu 2008/11/6

7 Introduction 7 Any function that is periodically repeats can be expressed as the sum of sines and/or cosines of different frequencies, each multiplied by a different coefficient Fourier series. Even functions that are not periodic can be expressed as the integral of sines and/or cosines multiplied by a weighting function. The formulation is the Fourier transform.

8 Definition of the Fourier Transform 8 Forward Continuous-Time Fourier Transform Inverse Continuous-Time Fourier Transform The forward transform is an analysis integral because it extracts spectrum information The inverse transform is a synthesis integral because it is used to create the time-domain signal from its spectral information.

9 Definition of the Fourier Transform 9 Time domain and frequency domain It is common to say that we take the Fourier transform of x(t), meaning that we determine so that we can use the frequency-domain representation of the signal. We often say that we take the inverse Fourier transform to go from the frequency-domain to the time-domain.

10 Example: Forward Fourier Transform 10 Consider the one-sided exponential signal Take the Fourier transform of x(t) Time-Domain Frequency-Domain

11 Rectangular Pulse Signals 11 Consider the rectangular pulse The Fourier transform is Time-Domain Frequency-Domain

12 Rectangular Pulse Signals 12 The Fourier transform of the rectangular pulse signal is called a sinc function. The formal definition of a sinc function is Time-Domain Frequency-Domain

13 Discrete Fourier Transform 13 One-dimensional DFT for u= 0, 1, 2,, M-1 for x= 0, 1, 2,, M-1 In order to compute F(u), we start by substituting u = 0 in the exponential term and then summing for all values of x. We then substitute u= 1 Like f(x), the transform is a discrete quantity, and it has the same number of components as f(x).

14 Discrete Fourier Transform 14 Euler s formula: Each term of the Fourier transform (the value of F(u)) is composed of the sum of all values of the function f(x). The domain (values of u) over which the values of F(u) range is called the frequency domain, because u determines the frequency of the components of the transform. Each of the M terms of F(u) is called a frequency component of the transform.

15 Discrete Fourier Transform 15 Express F(u) in polar coordinates: Magnitude or spectrum Phase angle or phase spectrum

16 Discrete Fourier Transform 16 Two-dimensional DFT

17 Images in Frequency Domain 17 Gonzalez and Woods, Chapter 4 of Digital Image Processing, Prentice-Hall, 2001.

18 Images and Their FT 18

19 Frequency Domain Features 19 Fourier domain energy distribution Angular features (directionality) u v Radial features (coarseness) u Uniform division may not be the best v

20 Gabor Texture 20 Fourier coefficients depend on the entire image (Global) we lose spatial information Objective: local spatial frequency analysis Gabor kernels: looks like Fourier basis multiplied by a Gaussian Gabor filters come in pairs: symmetric and anti-symmetric We need to apply a number of Gabor filters at different scales, orientations, and spatial frequencies Symmetric kernel Anti-symmetric kernel

21 Gabor Texture 21 Image I(x,y) convoluted with Gabor filters h mn (totally M x N) Using first and 2nd moments for each scale and orientations Features: e.g., 4 scales, 6 orientations 48 dimensions odd even

22 Gabor Texture 22 scale orientation Arranging the mean energy in a 2D form structured: localized pattern oriented (or directional): column pattern granular: row pattern random: random pattern

23 Homogeneous Texture Descriptor 23 Frequency plane partition is uniform along the angular direction (30º), non-uniform along the radial direction (on an octave scale) B.S. Manjunathand W.Y. Ma, Texture features for browsing and retrieval of image data, IEEE Trans. on PAMI, vol. 18, no. 8, 1996, pp

Gabor Function 24 On the top of the feature channel, the following 2D Gabor function (modulated Gaussian) is applied to each individual channels.

24 Gabor Function 24 On the top of the feature channel, the following 2D Gabor function (modulated Gaussian) is applied to each individual channels. Equivalent to weighting the Fourier transform coefficients of the image with a Gaussian centered at the frequency channels as defined above Each channel filters a specific type of texture

25 Homogeneous Texture Descriptor 25 Partition the frequency domain into 30 channels (modeled by a 2D Gabor function) Computing the energy and energy deviation for each channel Computing the mean and standard deviation of frequency coefficients HTD = {f DC, f SD, e 1,e 2,,e 30,d 1,d 2,,d 30 } f DC and f SD are the mean and standard deviation of the image e i and d i are the mean energy and energy deviation of the corresponding ith channel

26 Distance Measure 26 B.S. Manjunathand W.Y. Ma, Texture features for browsing and retrieval of image data, IEEE Trans. on PAMI, vol. 18, no. 8, 1996, pp Resources: On-line demo:

subsampling At each level, it decomposes a 2D signal into four subbands,

27 Wavelet Features 27 Wavelet transforms refer to the decomposition of a signal with a family of basis functions with recursive filtering and subsampling At each level, it decomposes a 2D signal into four subbands, which are often referred to as LL, LH, HL, HH (L=low, H=high) LL2 LH2 HL2 HH2 HL1 LH1 HH1

28 Wavelet Features 28 Using the mean and standard deviation of the energy distribution in each subband at each level. PWT (Pyramid-structured wavelet transform) Recursively decompose the LL band Results in 30-dimensional feature vector (3x3x2+2=30) TWT (Tree-structured wavelet transform) Some information appears in the middle frequency channels decomposition is not restricted to the LL band Results in 40x2 = 80 dimensional feature vector Original image PWT TWT T. Chang and C.C.J. Kuo, Texture analysis and clasification with tree-structure wavelet transform, IEEE Trans. On Image Processing, vol. 2, no. 4, 1993, pp

29 29 Edge Histogram Descriptor Wei-Ta Chu 2008/11/6 Park, et al. Eficient use of local edge histogram descriptor, Proc. of ACM International Workshop on Standards, Interoperability and Practices, pp , 2000.

Introduction 30 Spatial distribution of edges Edge histogram descriptor (EHD) Dividing the image into 4x4 subimages, and generate the edge histogram based on the edges

30 Introduction 30 Spatial distribution of edges Edge histogram descriptor (EHD) Dividing the image into 4x4 subimages, and generate the edge histogram based on the edges in the subimages. Edges are categorized into five types: vertical, horizontal, 45º diagonal, 135º diagonal, and nondirectional edges. A total of 5x16=80 histogram bins

31 31 Local Edge Histogram

32 32 Global, Semi-global, and Local Histograms Global-edge histogram Accumulate five types of edge distributions for all subimages Semiglobal-edge histogram

33 Image Matching 33 Combining the local, the semiglobal, and global histogram together. Total of 150 bins 80 bins (local) + 5 bins (global) + 65 bins (13x5, semiglobal) The L 1 distance measure D(A,B) can be: This feature is one of the MPEG-7 texture descriptors.

34 Performance Comparison 34 Retrieval performance of different texture features for the Corel photo databases. L 1 distance is used to computing the dissimilarity between images. For the MRSAR, Mahalanobis distance is used. #relevant images MRSAR (M) Gabor TWT PWT MRSAR Manjunath and Ma, Chapter 12 of Image Database: Search and Retrieval of Digital Imagery, edited by V. Castelli and L.D. Bergman, John Wiley & Sons, #top matches considered Tamura (improved) Coarseness histogram Directionality Edge histogram Tamura (traditional)

35 Performance Comparison 35 Retrieval performance of different texture features for the Brodatz texture image set. Percentage of retrieving all correct patterns Gabor MRSAR (M) TWT PWT Tamura (improved) MRSAR Tamura (traditional) Coarseness histogram Directionality Edge histogram #top matches considered

36 36 Shape for CBIR Wei-Ta Chu 2008/11/6

37 Shape Features 37 MPEG-7 provides contour-based shape and regionbased shape tools. region-based similarity contour-based similarity Bober, MPEG-7 visual shape descriptors, IEEE Trans. On CSVT, vol. 11, no. 6, pp , 2001.

38 Region-Based Shape Descriptor 38 The region-based SD expressed pixel distribution within a 2D object or region. It can describe complex objects consisting of multiple disconnected regions. 2D Angular Radial Transformation (ART) Gives a compact and efficient way of describing multiple disjoint regions Robust to segmentation noise

39 Angular Radical Transform (ART) 39 For each image, a set of ART coefficients F nm is extracted: The MPEG-7 Visual Part of the XM 4.0, ISO/IEC MPEG99/W3068, Dec W.-Y. Kim and Y.-S. Kim, A New Region-Based Shape Descriptor, ISO/IEC MPEG99/M5472, Maui, Hawaii, Dec

40 Contour-Based Shape Descriptor 40 The contour SD is based on the Curvature Scale-Space (CSS) representation of the contour. Distinguish between shapes that have similar region-based shape (b) Support search for shapes that are semantically similar, even significant intraclass variability (c) Robust to significant nonrigid deformations (d) and to perspective transformation (e)

41 Curvature Scale-Space (CSS) 41 When comparing shapes, humans tend to decompose shape contours into concave and convex sections. Features: How prominent they are, their length relative to the contour length, and their position and order on the contour CSS representation decomposes the contour into convex and concave sections by determining the reflection points (points at which curvature is zero)

42 Curvature Scale-Space (CSS) 42 CSS image shows how the inflection points change when filtering is applied to the contour X-axis corresponds to the position on the contour (clockwise, starting from any arbitrary point) Y-axis corresponds to the values of a shape smooth parameter (when y- values increase, amount of smoothing increases) Any black point in the CSS image signifies that at the corresponding position and at the corresponding scale, there is an inflection point.

43 Curvature Scale-Space (CSS) 43 The smoothing is performed iteratively and for each level, the zero crossings of the curvature function are computed. The CSS image is obtained by plotting all zero-crossing points on a plane Mokhtarian and Mackworth, A theory of multiscale, curvature-based shape representation for planar curves, IEEE Trans. on PAMI, vol. 14, no. 8, pp , 1992.

44 Shape Descriptor 44 Based on CSS images, the descriptor consists of Eccentricity ( 偏移量 ) and circularity ( 環狀 ) values of the original and filtered contour Number of peaks The magnitude (height) of the largest peak The x and y positions on the remaining peaks Chapter 15 of Introduction to MPEG-7: Multimedia Content Description Interface. Edited by Manjunath, et al., John Wiley & Sons, 2002.

45 45 Example: The QBIC System

46 Example: The QBIC System 46 Color Color histogram Texture Coarseness, contrast, directionality Shape Area, circularity, eccentricity, major-axis direction Fusion of multiple types of features often gives better performance.

47 References 47 Tamura, et al. "Textural feature corresponding to visual perception,"ieee Trans. on Systems, Man, and Cybernetics, vol. SMC-8, no. 6, pp , Park, et al. Eficient use of local edge histogram descriptor, Proc. of ACM International Workshop on Standards, Interoperability and Practices, pp , Manjunath and Ma, Chapter 12 of Image Database: Search and Retrieval of Digital Imagery, edited by V. Castelli and L.D. Bergman, John Wiley & Sons, Bober, MPEG-7 visual shape descriptors, IEEE Trans. on CSVT, vol. 11, no. 6, pp , 2001.

48 Next Week 48 Multidimensional Indexing Techniques Homework 2 will be announced

Spectral Texture Features

1 Spectral Texture Features Wei-Ta Chu 2010/10/21 Gabor Texture 2 The Gabor representation has been shown to be optimal in the sense of minimizing the joint twodimensional uncertainty in space and frequency.