Determining the 2d transformation that brings one image into alignment (registers it) with another. And

Transcription

1 Last two lectures: Representing an image as a weighted combination of other images. Toda: A different kind of coordinate sstem change. Solving the biggest problem in using eigenfaces? Toda Recognition b alignment: modeling the geometr of vision Homogeneous coordinates -- what s so special about? Determining the d transformation that brings one image into alignment (registers it) with another. Determining the d transformation that brings one image into alignment (registers it) with another. And Or. Determining the d transformation that brings one image into alignment (registers it) with another. Determining the d transformation that brings one image into alignment (registers it) with another.

2 Two parts Forward Geometric Transformation T(;θ) How one image should be warped to make another. How to interpolate data to warp one image. Moved image, I m Fied image, I f Transformed moving image, I m. It is mapped into the coordinate sstem of I f Transformation Models - A Start Translation and Independent Scaling Translation and scaling in d and d Similarit transformations in d and d Affine transformations in d and d We ve alread seen this in d: Transformations between piel and phsical coordinates Translation and scaling in the retina application In matri form: Where In homogeneous form: Similarit Transformations Components: Rotation (net slide) Translation dependent scaling Invariants Angles between vectors remain fied All lengths scale proportionall, so the ratio of lengths is preserved Eample applications: Multimodalit, intra-subject (same person) brain registration Registration of range images (no scaling) Rotations in d Assume (for now) that forward transformation is onl a rotation Image origins are in upper left corner; angles are measured clockwise with respect to ais I m I f

3 Similarit = Rotation, Translation and scale Affine Transformations As a result: X is (,) t Or: Geometr: Rotation, scaling (each dimension separatel), translation and shearing Invariants: Parallel lines remain parallel Ratio of lengths of segments along a line remains fied. Eample application: Mosaic construction for images of earth surface (flat surface, relativel non-oblique cameras) Affine Transformations in d Unified framework (ignore the ). In d, multipl location b arbitrar, but invertible matri: Translation + scale General affine A is an b matri. This matri product is A + t, (plus an etra row that sas =. Question, what would A look like for the similarit transform? What are homogeneous coordinates? Represent d points using numbers. Camera coordinates == Homogeneous coordinates Normal homogeneous (,) (w, w, w) Homogenous normal (a,b,c) (a/c, b/c) Wh important for vision? D coordinates are the homogeneous coordinate of their d image! canonical coordinates at the center of projection z f = in the image Origin: Blue point:

4 Camera coordinates == Homogeneous coordinates Camera coordinates == Homogeneous coordinates A camera identifies all points along a ra with a single image point. z Origin: in the image in space z f = Origin: in the image in space or s f = Blue point: Coordinates are equivalent up to a scale factor. Blue point: 6 9 s s s Eample coordinates Homogeneous or Projective Vanishing Points Coordinates are equivalent up to a scale factor. Points at infinit Image = 5 = 4 = What would reasonable coordinates be for this point? Where would be located? So if m camera was facing the left filing cabinets, this point at infinit would reall be at infinit. Points at infinit Vanishing Points Translation + scale Back to math Unified framework. General affine Projective. What if ou allow ANY matri?

5 Projective Transformations General linear transformation Lengths, angle, and parallel lines are NOT preserved Coincidence, tangenc, and complicated ratios of lengths are the invariants Application: Mosaics of image taken b camera rotating about its optical center Mosaic images of planar surface Algebra of Projective Transformations Add non-zero bottom row to the homogeneous matri: Transformed point location: Or write out the elements of A b to make it easier So what? The above gave different was of transforming the coordinates of one image into the coordinates of another. Graphics question: Given one image, and the transformation, make the second image. Process: Take piel (,) of image, and put its color at piel T(,) in new image. Problems? Backward Transformation Because of the above, some techniques estimate the parameters of the backwards transformation, T-: From the fied image to the moving image Intensit-based algorithms generall estimate the backwards transformation And he, if our transformation is just a matri, ou don t have to work hard to find the backwards transformation. It s just the matri inverse. Mapping Piel Values - Going Backwards If ou project backward from a real piel, ou also won t land eactl on a piel??!!? (but there is a good wa of making up a value) Moving image, I m Fied image, I f Need to go backward to get piel value. This will generall not land on a single piel location. Instead ou need to do interpolation.

6 Use linear interpolation along the top and bottom horizontal lines to determine za and zb: za = ( f) z(i, j) + f z(i+, j) zb = ( f) z(i, j+)+ f z(i+, j+) Use linear interpolation along the vertical line between za and zb to determine z(,): z(,) = ( g) za + g zb Or, Summar of coordinate sstem transforms Images and image coordinate sstems Transformation models Forward and backward mapping Transformations: Similarit Affine Projective Non-linear and deformable (squish) z(,) = ( f) ( g) z(i, j) + f ( g) z(i+, j) + ( f) g z(i, j+) + f g z(i+, j+) Interpolation and Image Warping Eample warps. What kind? Ok ok, enough graphics Graphics question: Given one image, and the transformation, make the second image Vision Question: Given two images, what is the transformation between them? Solving for transformation Solving for transformation Find the T that maps points in H to points in I. The aren t, above, but suppose H and I are related b just a translation. Then what parameters are we tring to solve? How man equations do we need? Solving for a matri of unknowns What if the are affine? What if the are general projective?

7 Solving for transformation Net project Details online tomorrow. In the general case ou have a b matri A such that A = in homogenous coordinates. (is it reall = )? How can ou solve for A? A) Write a program which takes two images, ou click or otherwise input corresponding points, it finds their transformation and makes a mosaic. B) Choice. Either etend A to: ) find corresponding points automaticall. ) Work to combine 7 or 8 images into one mosaic. C) Choice Correct non-geometric problems Something else. Image How Registration man points do ou need? Lecture Intensities Application: Image Alignment