I Chen Lin, Assistant Professor Dept. of CS, National Chiao Tung University Computer Vision: 6. Texture
Objective Key issue: How do we represent texture? Topics: Texture analysis Texture synthesis Shape from texture Textbook: David A. Forsyth and Jean Ponce, Computer Vision: A Modern Approach, Prentice Hall, New Jersey, 2003. Some contents are from the reference lecture notes or project pages: Prof. D. Lowe, Computer Vision, UBC, CA. Prof. T. Darrell, Computer Vision and Applications, MIT. Prof. D.A. Forsyth, Computer Vision, UIUC. http://graphics.cs.cmu.edu/people/efros/research/efrosleung.html
Texture How to capture the essence of texture? Repeated/structured stochastic Both? http://graphics.cs.cmu.edu/people/efros/research/efrosleung.html
Texture analysis Compare textures and decide whether they are of the same stuff. same or different http://graphics.cs.cmu.edu/people/efros/research/efrosleung.html
Texture synthesis Given a set of textures, the goal is to synthesize novel texture samples.
Representing textures Observation: textures are made up of sub elements, repeated over a region with similar statistical properties. Texture representation: Find the sub elements, and represent their statistics What filters can find the sub elements? Human vision ii suggests spots and oriented dfilters at a variety of different scales What statistics? Mean of each filter response over region Other statistics can also be useful
Derivative of Gaussian Filters Derivatives of Gaussian filters measure magnitudes and direction of image gradients.
Texture representation Figure 9.6 of D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Prentice Hall.
Different scales Figure 9.6 of D.A. Forsyth and J. Ponce, Computer Vision: A Modern Approach, Prentice Hall.
Different scales
Responses of different scales Take into account multiple scales!
The Laplacian Pyramid Building a Laplacian pyramid: First, create a Gaussian pyramid Take thedifference betweenonegaussian pyramid level andthenext next A close approximation to the Laplacian. The coarsest level is the same as that in the Gaussian pyramid. Bandpass filters: each level represents a different band of spatial frequencies Reconstructing the original image: Reconstruct the Gaussian pyramid starting at top layer
The Laplacian pyramid Create pyramid Collapse pyramid Figures from: http://sepwww.stanford.edu/ ~morgan/texturematch/paper_html/node3.html
A Gaussian pyramid
The corresponding Laplacian pyramid
Oriented pyramids Laplacian pyramid is orientation independent. Apply an oriented filter to determine orientations at each layer
Pyramid based texture analysis/synthesis
Gabor filters Gabor filters: Product of a Gaussian with sine or cosine Considering local spatial frequency Top row shows anti symmetric (or odd) filters, bottom row the symmetric (or even) filters.
The texture synthesis problem Generate new examples of a texture. t Pyramid approach: Use the same representation for analysis and synthesis This can produce good results for random textures, but fails to account for some regularities Recent approach: Use an image of the texture as the source of a probability model This draws samples directly from the actual texture, so can account for more types of structure Very simple to implement However, depends on choosing a correct distance parameter
Heeger and Bergen 95
Non parametric texture synthesis [Efros and Leung 99] Use image as a source of probability model Choose pixel values by matching neighborhood, then filling in Matching process look at pixel ldifferences count only synthesized pixels
Synthesizing i one pixel finite sample image SAMPLE p Generated image Since our sample image is finite, an exact neighbourhood match mightnot be present Find the best match using SSD error (weighted by a Gaussian to emphasize local structure), and take all samples within some distance from that match
Growing Texture Starting from the initial configuration, we grow the texture one pixel at a time The size of the neighbourhood window is a parameter that specifies how stochastic the user believes this texture to be To grow from scratch, we use a random 3x3 33patch from input image as seed
Randomness parameter
Increasing window size More synthesis results
The problem
Image quilting Efros and Freeman, Image quilting for texture synthesis and transfer,, SIGGRAPH 01
Image quilting
Shape from texture Texture distorts distortion is a cue to orientation changes in distortion are a cue to curvature But how do we use this cue? impose a regularity condition
Shape from texture (cont.) [W. C. Lin and Y. X. Liu 07] [A. Lobay and D.A. Forsyth 06]