LEHMANN + PARTNER GmbH. Automatic detection of defects based on machine learning algorithm Dr.-Ing. Dirk Ebersbach Prof. Dr.-Ing.

Transcription

1 LEHMANN + PARTNER GmbH Automatic detection of defects based on machine learning algorithm Dr.-Ing. Dirk Ebersbach Prof. Dr.-Ing. Andreas Großmann

2 What does ASINVOS mean?: Assistive and interactive machine learning based monitoring system for pavement surfaces analysis funded by 23/10/2017 2

3 Project motivation / main goal : Development of a semi-autonomous and interactive learning mobile mapping system based on deep learning (GAP) Detection of deviations from a previously trained normal state Interactive interpretation of the detected anomalies by the operator Confirm / Modify / Discardaus With each evaluation by the operator the system learns Application: Pavement 23/10/2017 3

4 Steps: Step 1: Requirements Analysis, Conceptual Design (news, development trends, learning process) Step 2: Interactively Navigable Data Stream (Interface, data structure, data integration ) Step 3: Development Graphical User Interface (Development Graphical User Interface, GUI) Step 4: Development Software Module Learning - normal state (process implementation, verification, computing optimization, ) Project Duration: /10/2017 4

5 Pavement Data Collection: Mobile Mapping System S.T.I.E.R Applanix Positioning System Inertial Navigation System Fraunhofer Pavement Profile Scanner Different Camera Systems 23/10/2017 5

6 Data Acquisition: High-resolution surface cameras (left & right camera) Pictures of about km federal roads are available = about 16 Mio. images 23/10/2017 6

7 Data Acquisition: Profilescanner 3D-Point Cloud 2.5D Pavement Surface Model 23/10/2017 7

8 Previous work and results - Conception image interface and realization - Included integration of 3D image data - Preparation and Integration of image data - the number of images is sufficient for the first training (more than 1000 images) - Preparation and Integration of 3D-point cloud - Only a small number (less than 1000 images) - Preparation and Integration of Graphical User Interface - Preparation and Integration Communication Protocol - Preparation and Integration of Operator-GUI (Interface to Detector-Server) 23/10/2017 8

9 Damage Classes: According to the German Standard (Road Monitoring and Assessment RMA) CRACK POTHO INPAT APPAT OPJOI BLEED Object Classes (no damages): expansion, gully, road marking, water hydrant, drill core sampling site, anchor,. 23/10/2017 9

10 Deep Learning Approaches What is a Convolutional Neural Network (CNN)? source: Wikipedia

11 Deep Learning Approaches Convolutional Neural Network used Normalzustand Schaden 11 Layers 4 Mio. free parameter Regularization technique: Dropout (avoids co-adaption) 23/10/

12 First results Examples POTHOLE PATCH CRACK

13 Datasets Content 3 federal roads (A, B, C) 1969 HD-images Normal state + 5 damage classes Patches of size 64x64 (8x8cm) Division Training (Road A): 1418 images = 4.9 Mio. patches Validation (Road B): 51 images = 200k patches Test (Road C): 500 images = 1.2 Mio. patches

14 Visual results Low threshold level high threshold level

15 Results from: International Joint Conference on Neural Networks + results from Boosting, SVM und AutoEncoder F 1 -Score 0,3121 0,4882 CrackIT 0,6373 Auto- Encoder 64x64 0,6642 RCD (CNN) 64x64 0,7184 RCD (CNN) 99x99 0,7246 ASINVOS (CNN) 64x64 Merkmale + Boosting Machine Learning Bildverarbeit. Deep Learning

16 - Future Application Integration in LP standard RMA tool

17 - Future Application Integration in LP standard RMA tool Project the damage according to the German RMA RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS RISS

18 Conclusions Millions of high-resolution road surface images are available in the framework of RMA and PMS Need: An automatic distress detection system Summarized: Only deep learning approaches were able to achieve satisfiying results The best generalization results were achieved using dropout only Project ASINVOS: First step to automate the labor process of analyzing millions of road surface images

19 Thank you