A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coecients

Transcription

1 A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coecients Javier Portilla and Eero P. Simoncelli Center for Neural Science, and Courant Institute of Mathematical Sciences, New York University, New York, NY 10003, USA gaurav aggarwal, umd 1

2 Texture Characterization - Traditional models Julesz hypothesis Nth-order empirical densities of image pixels Markov Random Fields Statistical interactions within local neighborhood Linear kernels Multiple orientations and scales. gaurav aggarwal, umd 2

3 Contributions of this paper : What's new? Universal(?) parametric model for visual texture Overcomplete multi-scale complex wavelet representation Markov statistical descriptors : pairs of wavelet coecients at adjacent spatial locations, orientations and scales Novel method for sampling from this model Iterative projection onto sets Revisit Julesz conjecture gaurav aggarwal, umd 3

4 How do we represent a texture? A random eld X(n, m) on a nite lattice (n, m) A set of constraint functions {φ k (X), k = 1... N c }, such that E(φ k (X)) = E(φ k (Y )), k Universal? samples of X and Y are perceptually equivalent What about the reverse? gaurav aggarwal, umd 4

5 Testing a Representation/Model Julesz Conjecture Perceptual equivalence? Individual images versus Statistics of RFs? Requires ergodicity. Proposed synthesis-by-analysis approach Practical ergodicity is enough P X ( φ(x(n, m)) E(φ(X)) < ɛ) p gaurav aggarwal, umd 5

6 The Complete Framework requires a real two-dimensional homogeneous random eld candidate constraint functions a method for estimating statistical paramaters an algorithm for sampling a RF satistfying the statistical constraints a method for measuring percptual similarity of two texture images. gaurav aggarwal, umd 6

7 Random elds from Statistical Constraints Density with maximum entropy Optimal - no other constraints imposed Dicult to compute Alternative Sampling from "Julesz Ensemble" T φ, c = { x : φ k ( x) = c k, k} Equivalent to maximal entropy distribution as lattice size grows to innity gaurav aggarwal, umd 7

8 Sampling via Projection Assuming X 0 is a homogeneous RF, and p φ, c : R L T φ, c we can get X t = p φ, c (X 0 ) Choice of X 0 That maximizes the entropy of X t - equally dicult High-entropy distribution for X 0 - Gaussian white noise gaurav aggarwal, umd 8

9 Projection onto Constraint Surfaces Dicult to construct a single p φ, c Alternative : an iterative solution Set of functions p k : R L T k where T k = { x : φ k ( x) = c k } gaurav aggarwal, umd 9

10 How do we project? Gradient Projection where λ k is chosen such that x = x + λ k φk ( x) φ k ( x ) = c k gaurav aggarwal, umd 10

11 Texture Model Constraint functions?? On pixel values / some other basis Biological motivation - localized oriented bandpass linear lters Steerable pyramid gaurav aggarwal, umd 11

12 Statistical Constraints - Perceptual Criteria Following approach used to incrementally augment the set of constraint functions Initialization - some basic parameters Gather synthesis failure New statistical constraint Verify that the new constraint works! Verify that the old constraints are still necessary gaurav aggarwal, umd 12

13 Constraints used : Marginal Statistics Normalized moments, range of the lowpass images computed at each level gaurav aggarwal, umd 13

14 Constraints used : Coecient Correlation Local auto-correlation of the lowpass images computed at each level gaurav aggarwal, umd 14

15 Constraints used : Magnitude Correlation Correlation of complex magnitude of pairs of coeceints at adjacent positions, orientations and scales gaurav aggarwal, umd 15

16 Normalized magnitude responses : gaurav aggarwal, umd 16

17 Necessary? gaurav aggarwal, umd 17

18 Constraints used : Cross-Scale Phase Statistics Cross-correlation of the real part of the coecients with both the real and imaginary part Edges/lines dilemma? gaurav aggarwal, umd 18

19 Necessary? gaurav aggarwal, umd 19

20 How good is the synthesis? On classic counterexamples of Julesz conjecture : gaurav aggarwal, umd 20

21 Failures: gaurav aggarwal, umd 21

22 On interesting "non"-textures: gaurav aggarwal, umd 22