Image Matching Fundamentals. Presented by: Dr. Hamid Ebadi

Transcription

1 Image Matching Fundamentals Presented by: Dr. Hamid Ebadi

2 Historical Remarks

3 Terminology, Working Definitions Conjugate Entity Matching Entity Similarity Measure Matching Method Matching Strategy

4 Relationship between Matching Methods and Matching Entities Matching Method Similarity Measure Matching Entities Area-based Correlation, Least-squares Gray levels Feature-based Cost function Edges, Regions symbolic Cost function Symbolic description

5 Problem Statement 1- Select a matching entity in one image 2- Find its conjugate entity in the other image 3- Compute the 3d location of the matched entity in object space 4- Assess the quality of the match

6 Fundamental Problems of Image Matching Search Space, Uniqueness of Matching Entity Combinatorial Eplosion Ambiguity Approimations, Constraints and Assumption Geometric Distortion of Matching Entities Noise Changing illumination Reflection properties Central projection Relief

7 Approimations, Constraints and Assumption Pull-in range Secondary Minimum

8 Geometric Distortion of Matching Entities b a

9 Geometric Distortion Due to Orientation Parameters a) Scale Difference Between the two Images b) Difference Rotation Angles Between the two Images

10 Scale Difference Between the two Images a b

11 Difference Rotation Angles Between the two Images a b c

12 Effect of Tilted Surface on Geometric Distortion a b

13 Effect of Relief on Geometric Distortions

14 Solutions to the Fundamental Problems

15 Search Space and Approimations Epipolar Lines Vertical Line Locus Hierarchical Approach

16 Epipolar Lines

17 Epipolar Lines '' p = ' p p p p p = H D B Z p f '' p = ' p b o H H D D Z p S = " " U L

18 Epipolar Lines U D D o U U Z H H b = ' " L D D o L L Z H H b = ' " ( )( ) U D L D D o Z H Z H Z H b s = δ ( ) 2 p D o Z H Z H b s D δ

19 Steps for Implementing Matching Scheme Along Epipolar Lines Select Matching entity in one image, p Estimate the Elevation, Z p,of the matching entity and its uncertainty rangeδz Compute approimate location P Compute search interval, s Perform matching within search interval Analyze the similarity measures obtained in previous step for determining conjugate location p

20 Vertical Line Locus

21 Concept of Matching Along the Vertical Line Loci v and V

22 Combining Vertical Line Locus and Epipolar Line Method

23 Implementation procedure for VLL 1-Select the X-Y position of a point P in object space and estimate the elevation Zp and it lower and upper bounds ZL, ZR 2-Begin the matching process by projecting Xp,Yp,ZLback to both images 3-Perform a similarity measure at the image positions X L, Y L, and X L, Y L 4-Move along the vertical line in suitable steps dz=(zu- ZL)/n 5-Project the object points Xp,Yp,ZL+dz back to the images and repeat the similarity measure 6-Repeat step 4 until Zu is reached 7-Analyze the similarity measures obtained in every step and determine the maimum/minimum for the conjugate position

24 Hierarchical Approach

25 Uniqueness of Matching Entity Variance Autocorrelation Entropy

26 Aera-Based Matching Matching Window Template Search Window

27 Area-Based Matching Location of Template Size of Template Location and Size of Search Window Acceptance Criteria Quality Control

28 Area-Based Matching Correlation Cross-Correlation Factor Maimum Correlation Factor Least Squares Matching

29 Cross-Correlation Factor σ LR σ L σ R ρ = σ σ L LR σ Covariance of image patches L and R Standard deviation of image patch L Standard deviation of image patch R R

30 Cross-Correlation Factor ( ) m n y g g n i m j j i L L., 1 1 = = = ( ) n m y g g n i m j j i R R., 1 1 = = = ( ) 1., = = = n m g y g n i m j L j i L σ L ( ) 1., = = = n m g y g n i m j R j i R σ R ( ) ( ) 1.,, 1 1 = = = n m g y g g y g n i m j R j i R L j i L σ LR

31 Maimum Correlation Factor a b c

32 Steps of Area-Based Matching with Correlation

33 Steps of Area-Based Matching with Correlation Select the centre of the template in one image Determine approimate locations for the conjugate position in the other image For both the template and the correlation window, determine the minimum size which passes the uniqueness criterion, select the larger of the two values as the window size for the current matching location Compute the correlation coefficient Pr,c for all positions r,c of the correlation window within the search window Analyze the correlation factors. A minimum threshold must be reached for a valid match, Apart from the maimum, determine the distinctness as a quality measure Repeat above steps for new template location until all positions to be matched are visited Analyze the matching results on a global base for consistency and/or compatibility with a priori scene knowledge

34 Least Squares Matching To minimize the gray level differences between the template and the matching windows whereby the position and the shape of the matching window are parameters to be determined in adjustment process

35 Least Squares Matching Mathematical Model Adjustment Procedure

36 Least Squares Matching Factors affecting gray level differences between template and matching windows Illumination and reflectance differences between the two images the photographic development process and scanner in case of digitized photographs Geometric distortions

37 Mathematical Model To compensate for differences in brightness and contrast, a transformation of the matching window is introduced { m( i, j) } = r r m( i j) 1. T R, 0 +

38 Mathematical Model (, y) = m T ( i j) m G, T, ( i j) = y = T ( i, j) y

39 Mathematical Model Since the two image patches are related via the common surface patch by the central projection, central projection model should be used Matching windows and surface are small It may be approimated by a plane

40 Mathematical Model (Affine Transformation) Affine Transformation: = t + t i t y = t3 + t4. i + t5. j j Observation Equation: ( i, j) = t( i j) m( y) r,,

41 Mathematical Model (Affine Transformation) Linearization of the model: ( ) ( ) ( )...,,, = t t T t t T t t T T y m y m y m δ δ δ ( ) , t t T t t T t t T T y m y y y δ δ δ

42 Mathematical Model (Linearization process) ( y) m, T ( y) m, T y = g = g y T t t 0 T 3 = 1 = 1 T = t4 T = t 1 T = t T = t 2 5 y y ( ) 0, y m ( i j) 0 m, = c = t( i, j) m( i, j)

43 Mathematical Model (Linearization process)

44 Adjustment procedure r = A t + c t = ( ) A T 1 PA A T c 2 σ r T Pr = n. m 6

45 Adjustment Procedure (Resampling)

46 LSM procedure Select the centre of the template in one image (Rt,Ct) Determine approimate locations for the matching window (RM,CM) Determine minimum size of template and matching window which passes the uniqueness criteria, select the larger of the two values as the window size Start first iterations with matching window at location RM,CM Transform matching window and determine the gray values for the tessellation (Resampling) Repeat adjustment and resampling sequence until termination criteria is reached. Assess quality of conjugate point Repeat above steps for new position of template

47 References T. Schenk, Digital Photogrammetry, Terra Science, 1999 M. Kasser and W. Egels, Digital Photogrammetry, Taylor and Francis, 2002 H. Ebadi, Advanced Analytical Aerial Triangulation, Lecture Note, K.N.Toosi University of Technology, 1999 T.C.Tang, Digital Image Correlation, UCSEm Report, 1988