Image Matching. AKA: Image registration, the correspondence problem, Tracking,

Transcription

1 Image Matching AKA: Image registration, the correspondence problem, Tracking,

2 What Corresponds to What? Daisy? Daisy From:

3 Relevant for Analysis of Image Pairs (or more)

4 Also Relevant for Recognition Find Example of Model Where is Waldo?

5 Or More Relevant

6 Strategy!? Many exist, the one in focus here is; For each image pair 1. Extract Features, here points. 2. Calculate Feature Descriptors. 3. Match features by pairing similar descriptors. Aggregate solution to multiple images.

7 Strategy Illustrated Windows Candidate Matches Feature Image n Image n+1

8 Where Did the Underlying Feature Go??

9 Feature Extraction What are good features to track? The ones that solves the problem! Usual nice properties: Uniqueness, i.e. it is clear where the feature is. Repeatability, i.e. we can detect it in other similar images. Distinguishable, i.e. we can find the feature again in another image.

10 What to Track I

11 Typical Features: Corners Blobs Edges Ridges What to Track II Very Successful, Focus here NB: Aperture problem.

12 Corner Detector Corner: High curvature in both directions. Stabile, or well located. Best all round solution: Harris Corner detector. Harris and Stephens A combined corner and edge detector Schmid, Mohr and Bauckhage Evaluation of interest point detectors 2000.

13 Harris Corner Detector Taylor Expansion Gaussian Window

14 Harris Corner Detector II Consider the Two Eigen Values of C(x,y) 1 large, one small Both large Both small

15 Harris Corner Detector III

16 Harris Corner Detector IV Calculating R

17 Harris Corner Detector V Flowchart dx ^2 Smooth a Image dy + ^2 Smooth Smooth c b 2 ab c k a + ( b) R Threshold & Non-max suppression 2 Scale Dependent Corners

18 Non Maximum Suppression

19 Harris Corner Detector VI An Example

20 Blob Detection - SIFT Detects places where the second order derivative are high - in both directions. Recently proved highly successful for matching, as SIFT features. See Lowe Distinctive image features from scale invariant keypoints, In the SIFT work features are searched for in an image pyramid aka. scale space. Stabile, or well located.

21 SIFT Features II

22 SIFT Features III DOG Example Calculate DOG through Scale Space.

23 SIFT Features IV Algorithm Outline: 1. Calculate Gaussian Scale Space. 2. Calculate DOG by subtracting adjacent scale images. 3. Threshold results and do non maximum suppression.

24 SIFT Features V Example Go play with the demos of And/or

25 Similarity Measure

26 Similarity Measures II What are good similarity measures? The ones that solve the problem Again Usual nice properties: Discriminates between respective features. Robust to wards noise. Invariant to certain transformations BUT NOT TO MANY.

27 Match by Correlation typical similarity measure Windows Candidate Matches Feature Image n Image n+1

28 Calculating Correlation of Patches Affine invariant; Cool but problematic

29 Why Correlation? MatLab Demo, correx_1. Maybe use pre-processing. The patch around the feature is the descriptor.

30 SIFT Feature Descriptors Outline: Align a grid to image gradient. With grid dimensions set by the feature scale.

31 SIFT Feature Descriptors Outline: Align a grid to image gradient. With grid dimensions set by the feature scale. Compute image gradient magnitude for each bin.

32 SIFT Feature Descriptors Outline: Align a grid to image gradient. With grid dimensions set by the feature scale. Compute image gradient magnitude for each bin. Compute directional histograms for each meta bin.

33 SIFT Feature Descriptors Outline: Align a grid to image gradient. With grid dimensions set by the feature scale. Compute image gradient magnitude for each bin. Compute directional histograms for each meta bin. These histograms are the descriptors, which can be matched by proximity, e.g. using a KD-tree.

34 SIFT Descriptor Process

35 Best(A i )=B j and Best(B j )=A i. A? Matching Descriptors B NB: Only One match per Feature For the sets A i and B j define: Best(A i )=min j d(a i,b j ), Best(B j )=min i d(a i,b j ). Then a match of a specific A i and B j is made if

36 Constraints in Search Space Point in Space Image of Point Focal Point Focal Point Image Plane 1 Image Plane 2

37 Constrained Tracking Illustration Image 1 Image 2

38

39

40

41 Two View Stereo