Memory Management Method for 3D Scanner Using GPGPU

Transcription

1 GPGPU 3D 1 2 KinectFusion GPGPU 3D., GPU., GPGPU Octree. GPU,,. Memory Management Method for 3D Scanner Using GPGPU TATSUYA MATSUMOTO 1 SATORU FUJITA 2 This paper proposes an efficient memory management system for real time 3D scanner using GPGPU and Kinect. 3D scanners proposed in KinectFusion has limitations on density and range from video memory size for GPU. In order to loosen the limitation, octree is said to work well in terms of data storing and processing. In addition to the original octree approach, we propose a new method which improves efficiency using main memory and sorting octree data and evaluate it from viewpoints of memory transfer size and processing load. 1. 3D 3D Newcombe KinectFusion[1][2] Structured Light Kinect GPGPU 3D Kinect KinectFusion Kinect GPU GPGPU Octree [3][4] [5][6] [6] Zeng Octree KinectFusion KinectFusion[1] Kinect GPGPU 3D 1 Graduate School of Computer and Information Sciences, Hosei University 2 Faculty of Computer and Information Sciences, Hosei University 1

2 Truncated Signed Distance Field 1 Kinect 2 KinectFusion 3D 1 2 Kinect Kinect 1 Kinect 6 ICP Iterative Closest Point ICP KinectFusion ICP 1 ICP KinectFusion GPU ICP Kinect Kinect TSDF Truncated Signed Distance Field Weight TSDF Signed Distance Field ray-marching TSDF ray-marching ray, TSDF Octree KinectFusion Zeng KinectFusion Octree Octree 8 8 X Y Z 2 2

3 4 Octree 5 Octree KinectFusion SwapLayer 4 3 TSDF GPGPU raycast TSDF ray-marching bravely-raymarching KinectFusion 10% TSDF Zeng Octree KinectFusion Octree KinectFusion OS 3.2 Octree KinectFusion Octree KinectFusion 4 Top Layer Branch Layer Middle Layer Data Layer Top Layer raycast Branch Layer 5 IdChild IdChild -1 xyzkey D xyzkey 1 Middle Layer IdChild xyzkey Data Layer xyzkey tsdf weight Octree KinectFusion Scan [8] Octree KinectFusion Data Layer TSDF 0 Middle Layer IdChild Scan

4 01: count = number of nodes at L 02: splitflag[1..count] = 0 03: swapflag[1..count] = 0 04: 05: // Split SwapIn 06: for all node O(i) at Level L in parallel do 07: if O(i) is in the view frustum then 08: sdf calculate sdf from O(i) and current depth image plane 09: if (O(i) has no child or has swap) and NeedSplit(O(i), sdf) then 10: splitflag[i] = 1 11: if O(i) has swap then 12: swapflag[i] = 1 13: endif 14: end if 15: end if 16: end for 17: 18: // 19: (shift, count) Scan(splitFlag, +) 20: 21: // Split SwapIn 22: for all node O(i) at level L in parallel do 6 Octree 23: if splitflag[i] == 1 then 24: swap_id -(O(i).idChild + SwapLayerIdBias) 25: O(i).idChild count + (shift[i] 8) 26: key O(i).xyzkey 8 27: for k from 0 to 7 do 28: addr O(i).idChild + k 29: if HasSwap(O(i)) then 30: oct[l + 1][addr] = SwapLayerO[swap_id + k] 31: else 32: oct[l + 1][addr].xyzkey key k 33: endif 34: end for 35: end if 36: end for 37: 38: // SwapLayer 39: (SwapLayer, swaplayercount) Compact(SwapLayer, swapflag) 40: 41: // 42: O.count = O.count + count * 8 43: SwapLayer.count = swaplayercount IdChild Compact [8] Octree KinectFusion Swap Layer Data Layer 5 [7] GPU Data Layer Swap Layer IdChild Data Layer IdChild Swap Layer Index 2 Swap Layer IdChild Swap Layer IdChild 2 Swap Layer ID

5 01: count O.count 02: swapcount SwapLayer.count 03: mergeflag[1..count] 0 04: swapoutflag[1..count] 0 05: // 06: for all nodes O(i) at level L - 1 in parallel do 07: if O(i).idChild >= 0 then 08: mergeflag[o(i).idchild / 8] NeedRemove(O(i)) or NeedSwapOut(O(i)) 09: swapoutflag[o(i).idchild / 8] NeedSwapOut(O(i)) and Layer below L is Data Layer 10: end if 11: end for 12: 13: // Scan 14: (shift, count) Scan(mergeFlag, +) 15:(swapoutScanOut,swapoutScanOutCount) Scan(swapoutFlag, +) 16: 17: // ID 18: for all node O(i) at level l - 1 in parallel do 19: idc O(i).idChild 20: if idc >= 0 and mergeflag[idc/8] == 0 then 21: O(i).idChild idc - shift[idc/8] 8 22: else 23: if swapoutflag[idc / 8] == 1 then 24: // SwapOut 25: swap_layer_idc swapoutscanout[idc / 8] * 8 + SwapCount 26: O(i).idChild -(swap_layer_idc + SwapLayerIdBias) 27: for child of O(i) nodes C(j) do 28: SwapLayerO[j] = C(j) 29: endfor 30: else 31: if mergeflag[idc / 8] == 1 then 32: O(i).idChild -1 33: else 34: O(i).idChild idc 35: endif 36: end if 37: end for 38: 39: // 40: Compact(oct[l], mergeflag, shift) 41: 42: // 43: O.count O.count - count 8 44: SwapLayerO.count swapcount + swapoutscanoutcount 8 7 Octree 01: // 1/8() 02: count number of nodes at level L / 8 03: 04: // 05: for i from 1 to count in parallel do 06: idxs[i] i 07: xyzkeys[i] O(i * 8).xyzkey 08: end for 09: 10: // xyzkey 11: (sorted_idxs, sorted_xyzkeys) Sort(xyzkeys, idxs) 12: 13: // 14: for i from 1 to count in parallel do 15: src_idx sorted_idxs[sorted_xyzkeys]; 16: new_o(i) O(sorted_idxs) 17: end for 18: 19: // 20: Swap(new_O, O) 8 1 CPU Intel Core i7-3930k 3.2GHz DDR3-SDRAM 8GB NVIDIA GeForce GTX 690 GPU : 2GB Kinect for Windows CUDA SDK 5.0 Kinect SDK 1.7 cudpp 2.0 Data Layer Middle Layer Branch Layer 3.4 Branch Layer 3 Data Layer xyzkey xyzkey KinectFusion Kinect SDK Newcombe Zeng GPGPU Octree ICP 1024 raycast Zeng Octree raycast bravely-raycast 5

6 Kinect Point Clouds kinfu[9] KinectFusion KinectFusion [3][4] Scan Compact Sort Octree KinectFusion cudpp[10] 4.2 Chair Room Human Kinect Kinect SDK Kinect Studio Kinect Far 0.5m 4.0m 8 Chair 4.3 Kinect Studio TSDF Data Layer 9 Chair xyzkey 200 xyzkey raycast 10 Room 6

7 2 [ ] Chair Room Human Room ICP Update Raycast 1% 67% 2% 22% 0% 1% 68% 2% 20% 2% Human 13 Human Chair 0 Chair Human Human raycast ICP 2 3 xyzkey 7

8 raycast KinectFusion Heredia [3] Whelan Kinitinous[4] Kintinous TSDF CPU 2 Octree Heredia Kintinous TSDF 6. Newcombe KinectFusion Zeng KinectFusion Octree GPGPU xyzkey Kinect xyzkey 1024 xyzkey 32bit bit 3 33bit 32bit xyzkey 64bit SSD HDD Kinect 1) R. A. Newcombe, S. Izadi, O. Hilliges, D. Molyneaux, D. Kim, A. J. Davison, P. Kohli, J. Shotton, S. Hodges, A. Fitzgibbo Kinectfusion: Real-time dense surface mapping and tracking, Procedings of IEEE / ACM International Symposium on Mixed and Augmented Reality, pp , ) S. Izadi, D. Kim, O. Hilliges, D. Molyneaux, R. A. Newcombe, P. Kohli, J. Shotton, S. Hodges, D. Freeman, A. Davison, A. Fitzgibbon KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera, ACM Symposium on User Interface Software and Technology, ) M. Zeng, F. Zhao, J. Zheng, X. Liu Octree-based Fusion for Realtime 3D Reconstruction, Graphical Models, Volume 75, Issue 3, pp , ) M. Zeng, F. Zhao, J. Zheng, X. Liu A memory-efficient kinectfusion using octree, Proceedings of Computational Visual Media 2012, pp , ) F. Heredia, R. Favier KinectFusion extensions to large scale environments, The Point Cloud Library, ) T. Whelan, J. McDonald, M. Kaess, M. Fallon, H. Johannsson, J. J. Leonard Kintinuous: Spatially extended kinectfusion, RSS Workshop on RGB-D: Advanced Reasoning with Depth Cameras, ) CUDA C Programming Guide: NVIDIA Corporation, ) M. Harris, S. Sengupta, John D. Owens GPU Gems 3, chapter 39, pp , ) The Point Clound Libray: kinfuls, The Point Cloud Library, _l_s.html ) M. Harris, J. D. Owens, S. Sengupta, Y. Zhang, A. Davidson cudpp - CUDA Data Parallel Primitives Library Google Project Hostinghttps://code.google.com/p/cudpp/