Combining 3D Shape, Color, and Motion for Robust Anytime Tracking David Held, Jesse Levinson, Sebas5an Thrun, and Silvio Savarese

Transcription

1 Goal: Fast and Robust Velocity Es5ma5on Combining 3D Shape, Color, and Motion for Robust Anytime Tracking David Held, Jesse Levinson, Sebas5an Thrun, and Silvio Savarese Our Approach: Alignment Probability P 1 P 2 p(x t z 1...z t )=ηp(z t x t,z t 1 )p(x t z 1...z t 1 ) Annealed Dynamic Histograms P 3 P 4 Spatial Distance Color Distance (if available) Probability of Occlusion

2 Goal: Fast and Robust Velocity Es5ma5on Combining 3D Shape, Color, and Motion for Robust Anytime Tracking David Held, Jesse Levinson, Sebas5an Thrun, and Silvio Savarese Baseline: Centroid Kalman Filter t+1 t p(x t z 1...z t )=ηp(z t x t,z t 1 )p(x t z 1...z t 1 ) Annealed Dynamic Histograms Baseline: ICP Local Search Poor Local Optimum!

3 Goal: Fast and Robust Velocity Es5ma5on Combining 3D Shape, Color, and Motion for Robust Anytime Tracking David Held, Jesse Levinson, Sebas5an Thrun, and Silvio Savarese Our Approach: Alignment Probability P 1 P 2 p(x t z 1...z t )=ηp(z t x t,z t 1 )p(x t z 1...z t 1 ) Annealed Dynamic Histograms P 3 P 4 Spatial Distance Color Distance (if available) Probability of Occlusion

4 Mo#va#on Quickly and robustly estimate the speed of nearby objects

5 Camera Images System Laser Data

6 Camera Images System Laser Data Previous Work (Teichman, et al)

7 Camera Images System Velocity Estimation Laser Data This Work Previous Work (Teichman, et al)

8 Velocity Es#ma#on t

9 Velocity Es#ma#on t+1 t

10 Velocity Es#ma#on t+1 t

11 Velocity Es#ma#on t+1 t

12 Velocity Es#ma#on t+1 t

13 ICP Baseline

14 ICP Baseline

15 ICP Baseline

16 ICP Baseline

17 ICP Baseline Local Search Poor Local Optimum!

18 Tracking Probability

19 Velocity Es#ma#on t

20 Velocity Es#ma#on t+1 t

21 Velocity Es#ma#on t+1 t

22 Velocity Es#ma#on t+1 t

23 Velocity Es#ma#on t+1 t

24 Velocity Es#ma#on t+1 t

25 Velocity Es#ma#on t+1 t X t

26 Velocity Es#ma#on t+1 t X t

27 Tracking Probability Measurement Model Motion Model

28 Tracking Probability Measurement Model Motion Model Constant velocity Kalman filter

29 Tracking Probability Measurement Model Motion Model

30 Tracking Probability Measurement Model Motion Model

31 Tracking Probability Measurement Model Motion Model

32 Tracking Probability Measurement Model Motion Model

33 Tracking Probability Measurement Model Motion Model

34 Tracking Probability Measurement Model Motion Model

35 Tracking Probability Measurement Model Motion Model

36 Tracking Probability Measurement Model Motion Model

37 Tracking Probability Measurement Model Motion Model

38 Tracking Probability Measurement Model Motion Model

39 Tracking Probability Measurement Model Motion Model

40 Tracking Probability Measurement Model Motion Model k

41 Tracking Probability Measurement Model Motion Model k Sensor noise Sensor resolution

42

43 Color Probability Probability Delta Color Value

44 Including Color

45 Including Color ( ) η z i z exp( 1 t 2 (z i z j ) T Σ 1 (z i z j )) + k

46 Including Color ( ) η z i z exp( 1 t 2 (z i z j ) T Σ 1 (z i z j )) + k

47 Including Color

48 Including Color

49 Including Color Probability Delta Color Value

50 Including Color Probability Delta Color Value

51 Including Color Probability Delta Color Value

52 Probabilis#c Framework 3D Shape Color Motion History Tracking

53 Tracking Probability P 1 P 2 P 3 P 4

54 Tracking Probability v y????? v x

55 Tracking Probability v y v x

56 Dynamic Decomposi#on v y v x

57 Dynamic Decomposi#on v y v x

58 Dynamic Decomposi#on v y v x

59 Dynamic Decomposi#on v y Derived from minimizing KL-divergence between approximate distribution and true posterior v x

60 Inflate the measurement model Annealing

61 Inflate the measurement model Annealing

62 Inflate the measurement model Annealing

63 Algorithm 1. For each hypothesis A. Compute the probability of the alignment Measurement Model Motion Model

64 Algorithm 1. For each hypothesis A. Compute the probability of the alignment Measurement Model B. Finely sample high probability regions Motion Model

65 Algorithm 1. For each hypothesis A. Compute the probability of the alignment Measurement Model B. Finely sample high probability regions Motion Model C. Go to step 1 to compute the probability of new hypotheses

66 Annealing More time More accurate

67 Any#me Tracker RMS error (m/s) Mean runtime (ms)

68 Any#me Tracker 0.6 RMS error (m/s) Choose runtime based on: Total runtime requirements Importance of tracked object Mean runtime (ms)

69 Comparisons 1.2 RMS error (m/s) Kalman Filter Mean runtime (ms) Mean

70 Comparisons 1.2 RMS error (m/s) Kalman Filter ICP Kalman ICP with Centroid Init Kalman ICP with Kalman Init Mean runtime (ms) Mean

71 Comparisons 1.2 RMS error (m/s) Kalman Filter ICP Kalman ICP with Centroid Init Kalman ICP with Kalman Init Annealed Dynamic Histograms Mean runtime (ms)

72 Kalman Filter

73 Kalman Filter ADH Tracker (Ours)

74 Models

75 Quan#ta#ve Evalua#on Crispness Kalman Filter Kalman ICP ADH Tracker (Ours) 0 People Bikes Cars

76 Sampling Strategies RMS error (m/s) ADH Tracker (Ours) Dense sampling Dense sampling with motion prediction Top cell sampling Mean runtime (ms)

77 Advantages over Radar

78 Conclusions 3D Shape Color Motion History Tracking Robust to Occlusions, Viewpoint Changes

79 Conclusions 3D Shape Color Motion History Tracking Robust to Occlusions, Viewpoint Changes Runs in Real-time Robust to Initialization Errors

80

81 Color Probability Probability Delta Color Value

82 Error vs Number of Points 1.5 Kalman Filter ADH Tracker (Ours) RMS error (m/s) Number of points

83 RMS error (m/s) Error vs Distance Kalman Filter ADH Tracker (Ours) Distance to tracked vehicle (m)

84 Error vs Number of Frames 2 RMS error (m/s) Number of frames tracked

85 Error vs Number of Frames 2 ADH Tracker ADH Tracker without motion model RMS error (m/s) Number of frames tracked

86 RMS error (m/s) No color No motion model No 3D shape