Realtime Object Detection and Segmentation for HD Mapping

Transcription

1 Realtime Object Detection and Segmentation for HD Mapping William Raveane Lead AI Engineer Bahram Yoosefizonooz Technical Director NavInfo Europe Advanced Research Lab Presented at GTC Europe 2018 AI in HD Mapping Panel Session E8469

2 NavInfo Europe Introducing NavInfo Europe NavInfo Europe incorporates the European Advanced Research Lab and Corporate Development activities of NavInfo in Europe. We work on cutting edge technologies in the context of NavInfo vision to become the Digital Brain of Intelligent Driving. From our research roots on applying artificial intelligence (deep learning) on computer images, we have extended our research towards other topics such as data security, communication (V2X, V2V, 5G), smart cities, and the Internet of Things. ~30 Headquartered in Eindhoven, the Netherlands, NavInfo Europe establishes partnerships with innovative European companies and knowledge institutes to support NavInfo in establishing its vision. NavInfo Europe is a 100% daughter member of the NavInfo group.

3 Feature Extraction

4 Computer Vision for HD Mapping Feature Extraction Computer Vision and Deep Learning play a vital role in automating feature extraction from daily data collected on field vehicles drives

5 Field Drive Data Collection Use Case Real Time Feature Extraction Camera Real Time Feature Extraction LiDAR Real Time Feature Extraction Sensor Fusion (Semi) Automated Map Updating

6 Gigabit Research Vehicle Drive PX2 Implementation Computer Vision: Detection and Segmentation Tegra A Pascal A Denver Denver Monocular Camera USB A57 A57 A57 A57 PCI Ex Pascal Dedicated GPU Pascal Integrated GPU Point Cloud and Sensor Fusion: Feature Extraction and Localization Tegra B Pascal B LiDAR Denver Denver GPS USB A57 A57 A57 A57 PCI Ex Pascal Dedicated GPU IMU Pascal Integrated GPU

7 Research Vehicle Drive PX2 Implementation Object Detection Segmentation Tegra A Pascal A Denver Denver Monocular Camera USB A57 A57 A57 A57 PCI Ex Pascal Dedicated GPU Pascal Integrated GPU Tegra X2 igpu Running the Object Detection Stack Object Detection Network Object Tracker Pascal MXM dgpu Running the Segmentation Stack Top View Projection Semantic Segmentation Network Requires 1 TFLOPS 30 fps Requires 2.5 TFLOPS 30 fps

8 Object Detection

9 NODE Object Detection System A vision based system that can detect and classify objects of interest such as traffic signs and traffic lights Detection based on Single Shot Detector (SSD) architecture

10 NODE Sample Detections

11 NODE Sample Detections

12 NODE Sample Detections

13 NODE Object Detection System Real Time NODE Performance Over 220 traffic sign classes supported Up to 32 fps on FP16 on NVIDIA Drive PX2 igpu Inference at Full HD resolution (1920x1080) Features Supported Traffic Signs Gantry Signboards Traffic Lights Digital Traffic Signs Optimizations TensorRT Implementation Custom CUDA Kernels for SSD

14 Demo Realtime Object Detection Watch online at

15 Semantic Segmentation

16 Road Marking Feature Extraction Vision based system to segment and extract road features of interest on a pixel level and to aid in object matching and sensor fusion Camera Input Top View Transformation Segmentation Network

17 Real Time Road Marking Scene Segmentation Based on the ContextNet architecture Currently supports 17+ Road Marking Classes Features Supported Surface Level: Lane Markings Speed Limits Text Arrows Road Objects: Gantry sign boards Guard Rails Curbs Performance: Inference at 110 fps on Titan Xp Inference at 32 fps on Drive PX2 Inference at 11 fps on Jetson TX2 (Inference at 1024x384 image size)

18 Accuracy Lane Lines + Features Input Image (Top View Transformed) Ground Truth Prediction

19 Accuracy Lane Lines + Features Input Image (Top View Transformed) Ground Truth Prediction

20 Accuracy Lane Lines + Features Input Image (Top View Transformed) Ground Truth Prediction

21 Accuracy Lane Lines + Features Input Image (Top View Transformed) Ground Truth Prediction

22 Demo Real Time Segmentation Watch online at

23 Realtime Object Detection and Segmentation for HD Mapping William Raveane Lead AI Engineer Bahram Yoosefizonooz Technical Director NavInfo Europe Advanced Research Lab Presented at GTC Europe 2018 AI in HD Mapping Panel Session E8469