I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University. Computer Vision: 6. Texture

Transcription

1 I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University Computer Vision: 6. Texture

2 Objective Key issue: How do we represent texture? Topics: Texture analysis Texture synthesis Shape from texture Textbook: David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, New Jersey, Some contents are from the reference lecture notes or project pages: Prof. D. Lowe, Computer Vision, UBC, CA. Prof. T. Darrell, Computer Vision and Applications, MIT. Prof. D.A. Forsyth, Computer Vision, UIUC.

3 Texture How to capture the essence of texture? Repeated/structured stochastic Both?

4 Texture analysis Compare textures and decide whether they are of the same stuff. same or different

5 Texture synthesis Given a set of textures, the goal is to synthesize novel texture samples.

6 Representing textures Observation: textures are made up of sub elements, repeated over a region with similar statistical properties. Texture representation: Find the sub elements, and represent their statistics What filters can find the sub elements? Human vision ii suggests spots and oriented dfilters at a variety of different scales What statistics? Mean of each filter response over region Other statistics can also be useful

7 Derivative of Gaussian Filters Derivatives of Gaussian filters measure magnitudes and direction of image gradients.

8 Texture representation Figure 9.6 of D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Prentice Hall.

9 Different scales Figure 9.6 of D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Prentice Hall.

10 Different scales

11 Responses of different scales Take into account multiple scales!

12 The Laplacian Pyramid Building a Laplacian pyramid: First, create a Gaussian pyramid Take thedifference betweenonegaussian pyramid level andthenext next A close approximation to the Laplacian. The coarsest level is the same as that in the Gaussian pyramid. Bandpass filters: each level represents a different band of spatial frequencies Reconstructing the original image: Reconstruct the Gaussian pyramid starting at top layer

13 The Laplacian pyramid Create pyramid Collapse pyramid Figures from: ~morgan/texturematch/paper_html/node3.html

14 A Gaussian pyramid

15 The corresponding Laplacian pyramid

16 Oriented pyramids Laplacian pyramid is orientation independent. Apply an oriented filter to determine orientations at each layer

17 Pyramid based texture analysis/synthesis

18 Gabor filters Gabor filters: Product of a Gaussian with sine or cosine Considering local spatial frequency Top row shows anti symmetric (or odd) filters, bottom row the symmetric (or even) filters.

19 The texture synthesis problem Generate new examples of a texture. t Pyramid approach: Use the same representation for analysis and synthesis This can produce good results for random textures, but fails to account for some regularities Recent approach: Use an image of the texture as the source of a probability model This draws samples directly from the actual texture, so can account for more types of structure Very simple to implement However, depends on choosing a correct distance parameter

20 Heeger and Bergen 95

21 Non parametric texture synthesis [Efros and Leung 99] Use image as a source of probability model Choose pixel values by matching neighborhood, then filling in Matching process look at pixel ldifferences count only synthesized pixels

22 Synthesizing i one pixel finite sample image SAMPLE p Generated image Since our sample image is finite, an exact neighbourhood match mightnot be present Find the best match using SSD error (weighted by a Gaussian to emphasize local structure), and take all samples within some distance from that match

23 Growing Texture Starting from the initial configuration, we grow the texture one pixel at a time The size of the neighbourhood window is a parameter that specifies how stochastic the user believes this texture to be To grow from scratch, we use a random 3x3 33patch from input image as seed

24 Randomness parameter

25 Increasing window size More synthesis results

26 The problem

27 Image quilting Efros and Freeman, Image quilting for texture synthesis and transfer,, SIGGRAPH 01

28 Image quilting

29 Shape from texture Texture distorts distortion is a cue to orientation changes in distortion are a cue to curvature But how do we use this cue? impose a regularity condition

30 Shape from texture (cont.) [W. C. Lin and Y. X. Liu 07] [A. Lobay and D.A. Forsyth 06]