Estimation of Automotive Pitch, Yaw, and Roll using Enhanced Phase Correlation on Multiple Far-field Windows

Transcription

1 Estimation of Automotive Pitch, Yaw, and Roll using Enhanced Phase Correlation on Multiple Far-field Windows M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester Visual Sensorics and Information Processing Lab Goethe-University, Frankfurt and Computer Vision Laboratory Linko ping University, Sweden July 22, 2015 M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 1

2 Motion estimation From far-field sub-images at two instants t 1 and t, the module determines: Global 2D translational motion Rate changes in the camera angle (roll/pitch/yaw) Rotation matrix between t 1 and t The motion is analysed for each pair of corresponding windows at instants t 1 and t M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 2

3 System Overview M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 3

4 Structure test The motion is only measured on those cells with enough structure information. To find those, we compute the windowed average and the variance: ˆm k = 1 x i w T (x i ) ˆσ 2 k = 1 x i w T (x i ) M w T (x i ) s(x i ) (1) i=1 M w T (x i ) (s(x i ) ˆm k ) 2 (2) i=1 Further computations are performed only on cells with sufficient structure. ˆσ k 2 too weak structure < > sufficient structure T var (3) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 4

5 Determining the translation motion We use a regularized version of Phase-only-Correlation. P(f) = S(f) G (f) S(f) G (f) + λ (4) Where lamba performs a soft thresholding of the FFT, leaving almost intact the votes from amplitudes larger than lambda and giving small weight to the rest. The displacement d between cells is determined as the location of the maximum of the inverse Fourier transform of P(f): p(x) = F 1 {P(f)} δ(x d) (5) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 5

6 Thresholding the back-projected spectrum Due to non-pure translation motion, the back-projected spectrum might not concentrate in one peak. We compute the variance of the noise σ 2 n = 1 M p(x) 2 F (6) and then we select a subset P of pixel coordinates belonging to the pixels of the peaks that passed the test p(x)! > 5 σ n p(x)! > 0.6 max(p(x)) (7) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 6

7 Checking the compactness of the peaks To find the peak compactness, the image intensity is normalized p(x)/ p(x), x P h(x) = x P (8) 0, else We first determine the centroids µ = (µ 1, µ 2 ) of h(x) µ = h(x) x (9) x and then the compactness K is computed as the sum of the 2nd order moments in both directions K = h(x) x µ 2 2 (10) x We test the compactness K and only the compact cells are used afterwards compact K < > T comp (11) not compact M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 7

8 Estimating the roll angle We define the matrix A with the centres of the windows and matrix B with position after applying the displacement d Both sets of points are aligned to the origin of coordinates by A = A 1 k k i=1 c i B = B 1 k k c i (12) i=1 M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 8

9 Estimating the roll angle We find the rotation matrix R that minimizes the element-wise difference of the points in A and the rotated points in B : min A R B F (13) From the sought rotation matrix R: ( ) cos(φ) sin(φ) R = sin(φ) cos(φ) (14) We obtain the roll angle Φ according to: Φ = arctan (R 2,1, R 1,1 ) (15) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 9

10 Estimating the roll angle Additionally we determine the average translation d between the point sets A and B, for latter estimating the pitch and yaw angles: d = 1 k k (c i c i ) = 1 k i=1 k d i (16) i=1 The transformation T = (R, d) maps the B points into A. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 10

11 Testing the validity of the mapping We validate if the mapping of the point sets is correct: 1 k k T c i c i 2 2 i=1 valid < > not valid T dist (17) If the distance is too large, we iteratively repeat the previous process using a leave-one-out strategy and we take the motion vector with the smallest distance M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 11

12 Finding pitch and yaw angles α f f -p 1 H 1 H 2 -p 2 x f α H -p 1 -p 2 x The pitch θ and yaw Ψ are computed as: ( cx p ) ( 2x cx p ) 1x θ = arctan arctan f x ( cy p ) ( 2y cy p ) 1y Ψ = arctan arctan f y f y f x (18) (19) M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 12

13 Motion results Estimation of camera angles for Kitti sequence 3 rel. yaw rate [deg/frame] rel. pitch rate [deg/frame] rel. roll rate [deg/frame] frame number frame number frame number gt estim. gt estim. gt estim. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 13

14 Motion results Estimation of camera angles for Kitti sequence 7 rel. roll rate [deg/frame] rel. pitch rate [deg/frame] rel. yaw rate [deg/frame] frame number frame number frame number gt estim. gt estim. gt estim. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 14

15 Motion results Table with results for the KITTI odometry sequences Roll Pi/Ya Pitch Yaw Seq Valid Est. E Corr Valid Est. E Corr Est. E Corr M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 15

16 Conclusions What we achieved: A reliable method for determining 2D global motion vectors and rotation angles for a pair of consecutive frames of an automotive sequence. An algorithm which can run on real-time (20-25fps) and assist on the prediction of tracked keypoints. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 16

17 Future work Boost the accuracy of the 2D motion estimates by using sub-pixel measurements. Choose among multiple hypotheses from phase correlation. Measure the motion in more windows, and use depth estimates from SfM of previous frame. Use more complex consistency checks to reduce outliers still further. M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 17

18 Contact M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester M. Barnada, C. Conrad, H. Bradler, M. Ochs and R. Mester 18