INFO - H Pattern recognition and image analysis. Vision

Transcription

1 INFO - H Pattern recognition and image analysis Vision

2 Stereovision digital elevation model obstacle avoidance 3D model scanner human machine interface (HMI)...

3 Stereovision image of the same point seen by two cameras = conjugated pair both points are on a same line : epipolar line if axes are // and both images share a same base epipolar are // to X (special case)

4 Ideal case optical axis // Z X,Y,Z f xl,xl b xr,yr

5 Ideal case optical axis // P global space (X,Y,Z) common base line, distance between origins = b, yl=yr left (l) and right (r) projection coordinate (xl,yl,zl) = (X-b/2, Y, Z) et (xr,yr,zr) = (X+b/2, Y, Z) xl = (X+b/2)f/Z xr = (X-b/2)f/Z where Z = bf/(xl- xr) and X = b(xl+ xr)/2(xl-xr) Y = by/(xl-xr)

6 Disparity disparity: d = xl - xr X,Y, Z position is given by X = (b[xr + xl]/2)/d Y = by/d Z = bf/d

7 Disparity Distance inversely proportional to disparity d = 0 iif P inf. Disparity proportional to b b>> better accuracy b>> smaler coverage

8 Pinhole camera Image plane (x,y) (x, y) = ( f X Z, f Y Z )

9 False matching case Stanford CS223B Computer Vision

10 False matching case Phantom points Stanford CS223B Computer Vision

11 Point matching Correlation Left Right Rectified images

12 Point matching Correlation Left Right scanline Rectified images

13 Point matching Correlation Left Right scanline SSD error Rectified images disparity

14 Point matching neighborhood signature comparison Left Right

15 Point matching (Normalized) Sum of Squared Differences Normalized Correlation

16 Point matching

17 Disparity map

18 Disparity map W = 3 W = 20

19 General case Difficult : optical axis not // base line not the same angle chosen to optimise scene coverage calibration needed 2 solids oriented by R + T)

20 Point matching using special points Harris, surf, fast,... add some robustness to deformation, allows wider angle between point of view use of structured light...

21 General case both camera are linked by a solid transform in general the line joining c to x is the epipolar line in c epipolar plane : c,c and x

22 Ambiguity mismatch Figure from Forsyth & Ponce

23 Ambiguity no fix characteristic points (smooth surface)

24 Epipolar lines convergent axis

25 Epipolar lines Translation // (XY)

26 Epipolar lines translation Z

27 Camera parameters

28 Camera parameters extrinsic parameters position and orientation in space instrinsic parameters link between pixel projected and space

29 Extrinsic parameters identify camera position w.r.t. global space translation T rotation R

30 Extrinsic parameters Pw coordinate in world space Pc coordinate in camera space X c Y c Z c = R = r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 P c = R(P w T ) r 11 r 12 r 13 r 21 r 22 r 23 r 31 r 32 r 33 X w T x Y w T y Z w T z X c = R T 1 (P w T ) Y c = R T 2 (P w T ) Z c = R T 3 (P w T )

31 Intrinsic parameters Geometrical and optical camera characteristics projection (focal distance f) image plane <> pixel transform optical distortions

32 Intrinsic parameters projection (focal distance f) x = f X c Z c = f RT 1 (P w T ) R T 3 (P w T ) y = f Y c Z c = f RT 2 (P w T ) R T 3 (P w T )

33 Intrinsic parameters image plane<> pixels transform sx,sy pixels size x = (x im o x )s x y = (y im o y )s y

34 Intrinsic parameters image plane<> pixels transform sx,sy pixels size x im y im 1 = 1/s x 0 o x 0 1/s y o y x y 1 image coordinates x im = fs x R T 1 (P T w ) R T 3 (P W T ) + o x y im = fs y R T 2 (P T w ) R T 3 (P W T ) + o y

35 Intrinsic parameters optical distortions x u = x d +(x d x c )(K 1 r 2 + K 2 r )+ (P 1 (r 2 + 2(x d x c ) 2 )+ 2P 2 (x d x c )(y d y c ))(1 + P 3 r ) y u = y d +(y d y c )(K 1 r 2 + K 2 r )+ (P 2 (r 2 + 2(y d y c ) 2 )+ 2P 1 (x d x c )(y d y c ))(1 + P 3 r )

36 Camera parameters intrinsic parameters M in = 1/s x 0 o x 0 1/s y o y extrinsic parameters M ex = r 11 r 12 r 13 R T 1 T r 21 r 22 r 23 R T 2 T r 31 r 32 r 33 R T 3 T

37 Multi-camera camera en translation ligne de base commune distance = cste

38 Multi-camera different base lines géométries épipolaires différentes permet l observation des dépouilles

39 Reconstruction ideal case P l P P r p l p r O l O r

40 Reconstruction real case two lines are not secant P l P P r p l p r O l O r

41 Calibration software example

42 Calibration software example