Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science.

Size: px

Start display at page:

Download "Colorado School of Mines. Computer Vision. Professor William Hoff Dept of Electrical Engineering &Computer Science."

Kristina Anthony
5 years ago
Views:

1 Professor William Hoff Dept of Electrical Engineering &Computer Science 1

2 Statistical Models for Shape and Appearance Note some material for these slides came from Algorithms and Applications, by Szeliski, Springer 2011 Computer vision: models, learning and inference, by Prince, Cambridge University,

3 Statistical Models for Shape and Appearance We often have knowledge about the possible ways that shape and appearance can vary When matching to a new image, we can use this to constrain the shape and possibly give us better results 3

4 Principal Components One way to model the variation in your data, is to use principal components Principal components: Assume we have a population of measurements vectors The covariance matrix is Find the eigenvectors and eigenvalues of The are called principal components We can express any of the original vectors as a linear combination of PCs: where,,,,,,, 4

5 Generative Model This is a generative model (as opposed to a discriminative model) Given coefficients h (which represent the parameters of the model in the world), we can generate examples of measurements x: where is normally distributed noise with covariance 5

6 Approaches We ll look at 3 approaches that use PC based statistical models: 1. Appearance based (use images) Also called eigenimages 2. Shape based (use contour points or surface points) Also called active shape models or point distribution models 3. Shape and appearance based (use both) Also called active appearance models 6

7 Appearance Based (eigenimages) We have a collection of images, I 1..I K We subtract off the mean of the collection of images We transform each MxN image I i into a column vector x i = [I i (1,1), I i (1,2), I i (1,N), I i (2,1), I i (M,N)] T x 1 x 2 x 3 x K Find the covariance matrix C x The eigenvectors (principal components) represent basis images (or eigenimages ) I 1 I 2 I 3 I K 7

8 Application Face images Input images (Yale database) from: 8

9 Eigenfaces 10 PC s capture 36% of the variance 25 PC s capture 56% of the variance 9

additional PC for reconstruction Each new image from left to

10 Reconstruction with a small number of Principal Components Each new image from left to right corresponds to using 1 additional PC for reconstruction Each new image from left to right corresponds to using 8 additional PC for reconstruction 10

11 Recognition To recognize a new image x, project it onto the linear subspace spanned by the eigenimages (i.e., find the coefficients ): DFFS: test if x is a face DIFS: measure how close x is to a training image face 11

12 Shape based (Active Shape Models) Represent a shape with a set of points For example, evenly spaced points along a contour Let x be the vector consisting of the set of points We compute the covariance of a set of training vectors, and get the eigenvectors (principal components) 12

13 Examples of contours Resistors 13

14 Hand 14

15 Recognition We assume that the coefficients of the PCs (the shape parameters) are normally distributed To recognize a new shape x, we project it onto the linear subspace formed by the PCs, and compute the Mahalanobis distances to each category center 15

16 Alignment of shapes The shapes must be registered and points sampled in the same places 16

Registration to image To register a point distribution model with an image, each control point searches in a direction normal to the contour to find

17 Registration to image To register a point distribution model with an image, each control point searches in a direction normal to the contour to find the most likely corresponding image edge point Then search over (a) the shape parameters and (b) the parameters of a transform to align the shapes 17

18 Examples 18

19 19

3D Shape Models Example: Creating bone models from 2D silhouettes

Thesis title: Bi Planar Image Registration and Modeling of Bones A.

Mahfouz, 3D Shape from Silhouette Points in Registered 2D Images

20 3D Shape Models Example: Creating bone models from 2D silhouettes using a statistical atlas Josh King (MS 2007),. Thesis title: Bi Planar Image Registration and Modeling of Bones A. Szymczak, W. Hoff and M. Mahfouz, 3D Shape from Silhouette Points in Registered 2D Images Using Conjugate Gradient Method, Medical Imaging 2010: Image Processing. Proceedings of the SPIE, Volume 7623, pp ,

Shape and Appearance Based You get better results if you model both shape and appearance This is called an Active Appearance Model (AAM) We use principal

21 Shape and Appearance Based You get better results if you model both shape and appearance This is called an Active Appearance Model (AAM) We use principal components for shape (encoded by a set of control points) and appearance (ie, the image texture) The texture is normalized to a canonical shape before being analyzed 21

22 Active Appearance Model Both shape and texture are represented as deviations from a mean shape and mean texture Note the same appearance parameters h simultaneously control both the shape and texture deformations these are correlated! 22

23 Example face modeling 23

24 24

Warping images To align the image texture according to shape displacements, use a piecewise affine transformation Triangulate image points using

25 Warping images To align the image texture according to shape displacements, use a piecewise affine transformation Triangulate image points using Delaunay triangulation. Image in each triangle is warped by an affine transformation. Computer vision: models, learning and inference Simon J.D. Prince 25

26 Registering a new image To register the face model to a new image, search over (a) the appearance parameters and (b) the warping parameters of a transform to align the shapes Computer vision: models, learning and inference Simon J.D. Prince 26

27 Example AAMs have been used directly for recognition, but their main use is to align faces into a canonical pose so that other methods of face recognition can be used 27

Image Coding with Active Appearance Models

Image Coding with Active Appearance Models Simon Baker, Iain Matthews, and Jeff Schneider CMU-RI-TR-03-13 The Robotics Institute Carnegie Mellon University Abstract Image coding is the task of representing