Time To Contact (TTC) Berthold K.P. Horn, Yajun Fang, Ichiro Masaki

Transcription

1 Time To Contact (TTC) Berthold K.P. Horn, Yajun Fang, Ichiro Masaki

2 Estimating Time-To-Contact (TTC)

3 Time-To-Contact (TTC) Time-to-contact is the ratio of distance to velocity. Time-to-contact estimated using fast low-level methods. Distance and velocity themselves require more than that. Camera need not be calibrated. TTC useful for collision warning.

4 Focus of Expansion (FOE) Complementary to TTC is the FOE. Focus of expansion is projection of velocity into image. Focus of expansion estimated using fast low-level methods. Methods for TTC related to, but different from those for FOE. TTC and FOE useful in control.

5 Real Video Data Test of FOE Recovery Accuracy

6 Guidelines for Solution: Avoid higher-level processing; Avoid learning methods;

7 Guidelines for Solution: Avoid higher-level processing; Avoid learning methods; Focus on direct motion vision methods; Focus on robustness; Focus on DSP implementable methods;

8 Guidelines for Solution: Avoid higher-level processing; Avoid learning methods; Focus on direct motion vision methods; Focus on robustness; Focus on DSP implementable methods; Find solut on for simple scenarios first; Built up to more and more realistic situations;

9 First Explore Mathematically Most Tractable Scenarios: Translation along optical axis towards orthogonal plane;

10 First Explore Mathematically Most Tractable Scenarios: Translation along optical axis towards orthogonal plane; Translation in arbitrary direction towards orthogonal plane; Translation along optical axis towards arbitrary plane;

11 First Explore Mathematically Most Tractable Scenarios: Translation along optical axis towards orthogonal plane; Translation in arbitrary direction towards orthogonal plane; Translation along optical axis towards arbitrary plane; Translation in arbitrary direction towards arbitrary plane;

12 Outline of Mathematical and Physical Basis: Brightness of image remains constant as object moves; d dt E(x,y,t) = 0 ue x + ve y + E t = 0 Image motion (u, v) is function of vehicle motion and object shape; Parameterize relative vehicle motion and object shape; Optimization problem to find best-fit parameters; min ( ue x + ve y + E t ) 2 dx dy

13 Brightness Change Constraint Line

14 Estimating the Derivatives (E x, E y, and E t )

15 Coordinate System Definition

16 First, some relationships of interest: If Z is distance and W = dz/dt velocity then T == Z / / dz d dt = 1 dt log e(z), Approaching planar object lying perpendicular to optical axis Let S be the (linear) size of the object and s that of its image s f = S Z that is sz = fs Then s dz dt + Z ds dt = 0 T = s / ds dt = 1 / d dt log e(s)

17 Motion Field Perspective projection (X, Y, Z) (x, y) x f = X Z and y f = Y Z Motion field (u, v) by differentiation: u f = U Z X Z W Z and v f = V Z Y Z W Z Rewrite in terms of image coordinates u = W Z (x x 0) and v = W Z (y y 0) where (x 0,y 0 ) = f(u/w,v/w) is the FOE and W/Z = 1/T.

18 Translation Relative to Planar Surface Combine (i) brightness change constraint equation, and (ii) motion field. Simplest Case: translation, optical axis and surface normal parallel. Since U = 0 and V = 0, substitute u = x(w/z) and v = y(w/z) in brightness change constraint equation ue x + ve y + E t = 0: W Z (xe x + ye y ) + E t = 0 Direct relationship between T = (Z/W ) and derivatives of brightness! Note appearance of radial derivative G = (xe x + ye y ). CG + E t = 0 where C = W/Z = 1/T.

19 Optimization Problem Minimize (w.r.t. C): ( CG + Et ) 2 Differentiate w.r.t. unknown C and set result equal to zero: ( CG + Et ) G = 0 or so C G 2 = GE t T = 1/C = G 2/ GEt where G = xe x + ye y. Simple direct calculation for TTC! No higher level processing, feature detection, or whatever...

20 Arbitrary translational motion direction If U and V are not zero we get instead AE x + BE y + CG + E t = 0 where A = f(u/z), B = f(v/z), C = W/Z, and G = (xe x + ye y ). Again, a relationship between motion parameters and brightness derivatives. But now with three unknowns (A, B, and C = 1/T ) Minimize ( AE x + BE y + CG + E t ) 2 = 0 Leading to A E 2 x + B E x E y + C GE x = E x E t, A E x E y + B E 2 y + C GE y = E y E t, A GE x + B GE y + C G 2 = GE t. Threee linear equations in three unknowns.

21 Further Generalization Arbitrary surface orientation. Arbitrary direction of motion and arbitrary surface orientation Quadratic surface approximations. Known rotational component of motion.

22 Time To Contact (synthetic sequence)

23 Time To Contact (time lapse sequence)

24 Time To Contact (real world sequence)

25 [ w o w Fig. 7. Integral of 1/TTC (red dots) versus log e (s(t)/s(0)) (marked every ten frames). The diagonal dashed green line shows the ideal relationship.

26 Future Extensions Compensate for AGC and AEC Compensate for Progressive Scan Allow for distant stationary background; Discount contribution from ground surface;

27 Future Extensions Compensate for AGC and AEC Compensate for Progressive Scan Allow for distant stationary background; Discount contribution from ground surface; Allow for non-planar objects;

28 Future Extensions Compensate for AGC and AEC Compensate for Progressive Scan Allow for distant stationary background; Discount contribution from ground surface; Allow for non-planar objects; Allow for rotational motion using vehicle sensors; Allow for rotational motion using only video itself;

29 Future Extensions Compensate for AGC and AEC Compensate for Progressive Scan Allow for distant stationary background; Discount contribution from ground surface; Allow for non-planar objects; Allow for rotational motion using vehicle sensors; Allow for rotational motion using only video itself; Allow for multiple independently moving objects;

30 Previous Work Direct Motion Vision Horn, B.K.P. & B.G. Schunck, Determining Optical Flow, Artificial Intelligence, Vol. 16, No. 1 3, August 1981, pp Bruss, A.R. & B.K.P. Horn, Passive Navigation, Computer Vision, Graphics, and Image Processing, Vol. 21, No. 1, January 1983, pp Horn, B.K.P. & S. Negahdaripour, Direct Passive Navigation, IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-9, No. 1, January 1987, pp Horn, B.K.P. & E.J. Weldon, Jr., Direct Methods for Recovering Motion, International Journal of Computer Vision, Vol. 2, No. 1, pp , June Negahdaripour, S. & B.K.P. Horn, A Direct Method for Locating the Focus of Expansion, Computer Vision, Graphics and Image Processing, Vol. 46, No. 3, June 1989, pp Horn, B.K.P., Parallel Analog Networks for Machine Vision, in Artificial Intelligence at MIT: Expanding Frontiers, edited by Patrick H. Winston and Sarah A. Shellard, MIT Press, Vol. 2, pp , Horn, B.K.P. & J.G. Harris, Rigid Body Motion from Range Image Sequences, Computer Vision, Graphics and Image Processing, Vol. 53, No. 1, January McQuirk, I.S., B.K.P. Horn, H.-S. Lee, & J.L. Wyatt Estimating the Focus of Expansion in Analog VLSI, International Journal of Computer Vision, Vol. 28, No. 3, 1998, pp

31

32 [8] E. De Micheli, V. Torre, & S. Uras, The Accuracy of the Computation of Optical Flow and of the Recovery of Motion Parameters, IEEE Transactions on PAMI, Vol. 15, No. 5, May 1993, pp [9] T.A. Camus, Calculating Time-to-Contact Using Real-Time Quantized Optical Flow, Max-Planck-Institut fur Biologische Kybernetik, Technical Report No.14, February, [10] P. Guermeur & E. Pissaloux, A Qualitative Image Reconstruction from an Axial Image Sequence, 30th Applied Imagery Pattern Recognition Workshop, AIPR 2001, IEEE Computer Society, pp [11] S. Lakshmanan, N. Ramarathnam, & T.B.D. Yeo, A Side Collision Awareness Method, IEEE Intelligent Vehicle Symposium 2002, Vol. 2, pp , June [12] H. Hecht & G.J.P. Savelsbergh (eds.), Time-To-Contact, Elsevier, Advances in Psychophysics, 2004.