CV: Matching in 2D. Matching 2D images to 2D images; Matching 2D images to 2D maps or 2D models; Matching 2D maps to 2D maps MSU CSE 803

Transcription

1 CV: Matching in 2D Matching 2D images to 2D images; Matching 2D images to 2D maps or 2D models; Matching 2D maps to 2D maps

2 2D Matching n n Problem 1) Need to match images to maps or models 2) need to match images to images Applications 1) land use inventory matches images to maps 2) object recognition matches images to models 3) comparing X-rays before and after surgery

3 Methods for study n Recognition by alignment n Pose clustering n Geometric hashing n Local focus feature n Relational matching n Interpretation tree n Discrete relaxation

4 Tools and methods n Algebra of affine transformations scaling, rotation, translation, shear n Least-squares fitting n Nonlinear warping n General algorithms graph-matching, pose clustering, discrete relaxation, interpretation tree

5 Alignment or registration DEF: Image registration is the process by which points of two images from similar viewpoints of essentially the same scene are geometrically transformed so that corresponding features of the two images have the same coordinates

6 Components of transformations Scaling, rotation, translation, shear

7 scaling transformation

8 Rotation transformation

9 Pure rotation

10 Orthogonal tranformations n n n n n n DEF: set of vectors is orthogonal if all pairs are perpendicular DEF: set of vectors is orthonormal if it is orthogonal and all vectors have unit length Orthogonal transformations preserve angles Orthonormal transformations preserve angles and distances Rotations and translations are orthonormal DEF: a rigid transformation is combined rotation and translation

11 Translation requires homogeneous coordinates

12 Rotation, scaling, translation

13 Model of shear

14 General affine transformation

15 Solving for an RST using control points

16 Extracting a subimage by subsampling

17 Subsampling transformation

18 subsampling At MSU, even the pigs are smart.

19 recognition by alignment n Automatically match some salient points n Derive a transformation based on the matching points n Verify or refute the match using other feature points n If verified, then registration is done, else try another set of matching points

20 Recognition by alignment

21 Feature points and distances

22 Image features pts and distances

23 Point matches reflect distances

24 Once matching bases fixed n n n can find any other feature point in terms of the matching transformation can go back into image to explore for the holes that were missed (C and D) can determine grip points for a pick and place robot ( transform R and Q into the image coordinates)

25 Compute transformation Once we have matching control points (H2, A) and (H3, B) we can compute a rigid transform

26 Get rotation easily, then translation

27 Generalized Hough transform Cluster evidence in transform parameter space

28 Best affine transformation from overdetermined matches

29 Best affine transformaiton Use as many matching pairs ((x,y)(u,v)) as possible

30 result for previous town match

31 y v x u

32 11 matching control points x, y, u, v below = T [ u, v, 1] = T [x, y, 1] t

33 Least Squares in MATLAB n BigPic = n n n n n n n n n n n [ a11 a21 a12 a22 = a13 a23 ] n LilPic = n n n n n n n n n n n

34 Least squares in MATLAB 2 n >> ERROR = LilPic - BigPic * AFFINE n >> AFFINE = BigPic \ LilPic n AFFINE = n n n The solution is such that the 11D vector at the right has the smallest L2 norm n ERROR = n n n n n n n n n n n Worst is 1.8 pixels

35 Least squares in MATLAB 3 n X = n >> T = X \ Y n n n n n >> Y n Y = n n n n Solution from 4 points has smaller error on those points n T = n n n n n E = >> E = Y - X*T n n n n

36 Least squares in MATLAB 4 When the affine transformation obtained from 4 matching points is applied to all 11 points, the error is much worse than when the transformation was obtained from those 11 points. n >> E2 = LilPic - BigPic * T n E2 = n n n n n n n n n n n