Learning for Active 3D Mapping

Transcription

1 Learning for Active 3D Mapping Karel Zimmermann, Tomáš Petříček, Vojtěch Šalanský, Tomáš Svoboda ICCV 2017 Vision for Robotics and Autonomous Systems Center for Machine Perception Department for cybernetics Faculty of Electrical Engineering

2 Motivation Motivation: New Solid State Lidars will allow independent steering of depth-measuring rays S3 principle Emitted laser beams Transmitted through Optical Phased Array Controlling optical properties of OPA elements, allows to steer laser beams in desired directions Reflected laser beams are captured by SPAD array Images of S3 Lidar redistributed with permission of Quanergy Systems (

3 Problem definition 1. Learn to reconstruct dense 3D voxel map from sparse depth measurements

4 Problem definition 1. Learn to reconstruct dense 3D voxel map from sparse depth measurements 2. Optimize reactive control of depth-measuring rays along an expected vehicle trajectory Images of S3 Lidar redistributed with permission of Quanergy Systems (

5 Overview of active 3D mapping Learning of 3D mapping network. M(x ) x y(x, ) M(x )

6 Overview of active 3D mapping Learning of 3D mapping network. M (x ) Planning of depth measuring rays... J( ) y(x, ) x M (x ) J( )

7 Overview of active 3D mapping Learning of 3D mapping network. M (x ) Planning of depth measuring rays... J( ) SSL provides following sparse measurement x( ) y(x( ), ) x( ) M (x ) J( )

8 Learning & Planning minimize common objective X arg min L(, ) voxels y(x( ), ) x( ) M (x ) J( )

9 Learning & Planning minimize common objective X y arg min L(, ) voxels y(x( ), ) x( ) M (x ) J( )

10 Learning & Planning minimize common objective X arg min L(y(x( ), ), y ) subject to J` ( ) K voxels y(x( ), ) x( ) M (x ) J( )

11 Learning as minimization over X arg min L(y(x( ), ), y ) subject to J` ( ) K voxels Result of planning is not differentiable y(x( ), ) x( ) M (x ) J( )

12 0 Locally approximate objective around X 0 arg min L(y(x( ), ), y ) subject to J` ( 0 ) K voxels 0 y(x( ), ) x( 0 ) M (x ) J( 0 )

13 1 = Minimize approximated objective to get 1 arg min X voxels L(y(x( 0 ), ), y ) 0 SGD 1

14 Minimize approximated objective to get 1 arg min X voxels L(y(x( 1 ), ), y ) 0 SGD 1

15 2 = Minimize approximated objective to get 1 arg min 1 SGD X voxels 0 2 L(y(x( 1 ), ), y )

16 Iteratively optimize approximated objective t+1 = arg min X voxels L(y(x( t ), ), y ) t t+1 Fix point of this mapping would assure: local optimality of the objective statistical consistency of the learning In practise, we iterate until validation error stops decreasing

17 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated y(x( ), ) x( ) M (x ) J

18 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated from current map y voxel i

19 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated from current map y voxel i 1 (y) (y) occupied unoccupied

20 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated from current map y i = H current loss in voxel i voxel i

21 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated from current map y i = H current loss in voxel i voxel i rays J Y j2j p ij prob. that voxel i is not visible by any ray j 2 J Total expected loss: X voxels i Y j2j p ij

22 Planning of depth measuring rays J No ground truth y available online Objective for planning is approximated from current map y i = H current loss in voxel i voxel i rays J Y j2j p ij prob. that voxel i is not visible by any ray j 2 J Total expected loss: X voxels i Y j2j p ij

23 Planning of depth measuring rays J Planning of J = {J 1...J L } over horizon L (i.e. for following positions ` =1...L): X Y arg min i p ij subject to J` apple K J voxels j2j Convex approximations Naive greedy algorithm Prioritized greedy algorithm Upper bound on the approximation ratio of prioritized greedy proposed optimal UB 30% optimal coverage 50%

24 Experiment: Structure of 3D mapping network 2 R 20M

25 Experiment: Setting Steerable SSL is not yet avaiable Simulation of SSL on Kitti dataset.

26 apping Figure 4. Recall to false-positive rate on test data for network h0 (Random) and h4 (Coupled). False positives which can be attributed to discretization error (in 1-voxel distance to occupied voxels) do not count. Experiment: Qualitative evaluation Note: Detailed quantitative evaluation in poster and paper iment, we used 17 and 3 sequences from category for training and validation, respecquences from the City category for testing. lternating procedure of learning and pland above (see Sec. 3 5), we learned mapping 1,..., ht using batch size 1 and momentum ng rate always started at = Trainetwork h0 took iterations and twice learning rate, to 10 4 after iteraafter iterations. Training the succest took iterations (approximately one entially decreasing learning rate to ved that the net ht achieve best results al4 planning-training iterations. n from the ROC curves in Fig. 4, the perforplanning-training iterations overcomes netlanning. When we have evaluate network g for new input, ROC curve was almost infrom the one generated by h0 with random Sparse measurements trees cars Reconstructed map network xl contains around 2.5% of known f the voxels are estimated by the CNN. ROC ction are computed using global confidence nd truth map y. An examples from reconown in Fig. 5. Czech Technical University in Prague Figure 5. Two examples of global map reconstruction. The black son to a Recurrent Faculty Image-based Ar- Engineering, of Electrical of car. Cybernetics line denotes Department trajectory of the Top row: The measurement Ground truth

27 Experiment: Summary & Questions [1] Zimmermann, Petricek, Salansky, Svoboda, Learning for Active 3D Mapping, ICCV,