Neue Verfahren der Bildverarbeitung auch zur Erfassung von Schäden in Abwasserkanälen? Fraunhofer HHI 13.07.2017 1
Fraunhofer-Gesellschaft Fraunhofer is Europe s largest organization for applied research. 69 institutes and research units in Germany Research cooperation around the world 24 500 staff members Budget: more than 2.1 billion Research highlights: MP3 audio codec, H.264, H.265 video codec, LTE mobile communications standard Fraunhofer HHI 13.07.2017 2
Fraunhofer Institute for Telecommunications, Heinrich Hertz Institute, HHI 1928 Founding of Heinrich Hertz Institute in Berlin Since 1960s Research on fiber optical transmission Since 1980s Research on digital video coding Since 2003 Member of Fraunhofer-Gesellschaft Since 2009 Branch site in Goslar, Germany Fraunhofer HHI 13.07.2017 3
Facts and Figures Last updated: 2015 Fraunhofer HHI 13.07.2017 4
Departments Photonic Network and Systems Photonic Components Fiber Optical Sensor Systems Wireless Communications and Networks Vision and Imaging Technologies Video Coding and Analytics Photo: fotolia.com/seanpavonephoto Edit: Fraunhofer HHI Fraunhofer HHI 13.07.2017 5
Image Processing Pipeline Processing & Analysis scene acquisition geometry motion / deformation semantic data inspection rendering & display Fraunhofer HHI 13.07.2017 6
2D Scene Capturing Projection of 3D scene into 2D image plane Loss of 1 dimension Still images or video Higher and higher resolution Fraunhofer HHI 13.07.2017 7
Image Reprojection perspective camera fisheye camera Camera calibration determines parameters of projection Image can be reprojected onto any other surface But viewing position cannot be changed Fraunhofer HHI 13.07.2017 8
Image Reprojection perspective camera fisheye camera Fraunhofer HHI 13.07.2017 9
Spectral Sensitivity NIR imaging RGB image of tumor tumor @ 500nm Regular camera and humans: RGB colors Multi-spectral imaging can provide material properties Information invisible to humans thermal imaging Fraunhofer HHI 13.07.2017 10
3D Scene Capturing disparity d=x l -x r x l x r z~ 3D geometry can be estimated from multiple views (photogrammetry) Ill-posed problem Requires determination of corresponding points between views Epipolar geometry constrains search space triangulation Fraunhofer HHI 13.07.2017 11
Depth Estimation from Stereo Images captured with high resolution (10-20 MP) Dense reconstruction from 2 or more views Arbitrary objects Reconstruction of entire head or persons (incl. hair, eyes, etc.) Fraunhofer HHI 13.07.2017 12
High Detail 3D Reconstruction [Blumenthal, Computers & Graphics 2014] Fraunhofer HHI 13.07.2017 13
Textured 3D Model from 2 Views Fraunhofer HHI 13.07.2017 14
3D Models in Security Applications augmented reality visualization security gates Fraunhofer HHI 13.07.2017 15
High Quality 3D Reconstruction Fraunhofer HHI 13.07.2017 16
Aerial Surveillance, Object Detection Fraunhofer HHI 13.07.2017 17
Analysis of Vegetation and Landscapes Fraunhofer HHI 13.07.2017 18
Inspection of Laser Welds Fraunhofer HHI 13.07.2017 19
3D Recording of Footprints Fraunhofer HHI 13.07.2017 20
Support of Large Range of Object Sizes Fraunhofer HHI 13.07.2017 21
Tracking and Motion Estimation camera motion object motion Object / camera motion can be determined from point correspondences 3D motion requires additional priors (e.g. static scene, deformation model) Dense / sparse motion models Fraunhofer HHI 13.07.2017 22
3D Motion Tracking of Robot 3D tracking of motion path Corresponding points with epipolar constraint Assumption of near cylindrical pipe shape Fraunhofer HHI 13.07.2017 23
Enhanced Surface Textures Projection onto 3D cylinder surface Correction of lighting artifacts Compared to simple unwrap higher resolution less stitching artifacts unwrapping with 3D motion Panoramo Fraunhofer HHI 13.07.2017 24
Deformable Surface Tracking Fraunhofer HHI 13.07.2017 25
Deformable Surface Tracking Fraunhofer HHI 13.07.2017 26
3D Surface Tracking for Navigated Surgery Fraunhofer HHI 13.07.2017 27
Image Analysis with Machine Learning [ImageNet] Deep neural networks highly successful for many image analysis tasks image classification (e.g. detection of defects in pipes) scene labeling / segmentation image synthesis Fraunhofer HHI 13.07.2017 28
Image Classification (labeled) Training data Classifier Testing Training Fish Fraunhofer HHI 13.07.2017 29
Convolutional Neural Network Fraunhofer HHI 13.07.2017 30
Explanation of Black Box Black box classifier Explanation of deep neural network decisions Visualization via relevance feedback Fraunhofer HHI 13.07.2017 31
Detection of Morphing Attacks Original Morph Detection of morphing attacks in ABC Gates Anomaly detection in passport images Sematic image forensics Machine learning Fraunhofer HHI 13.07.2017 32
Creation of Face Morphs Morphing pipeline with exchangeable components Feature detection Warping Blending Automatic alignment of face images based of feature points/feature lines Automatic adaptive blending that prevents visible cuts Field warping Alpha blending (α=.5) Piecewise affine Our adaptive Original image source: pics.stir.ac.uk Fraunhofer HHI 13.07.2017 33
Detection of Face Morphs Morphing attack detection using Deep Neural Networks Creation of training and test datasets >1500 original images Image pre-processing (filtering, noise) to increase variations in dataset Comparison of different network architectures Fraunhofer HHI 13.07.2017 34
VR / AR for Interaction and Assistance Fraunhofer HHI 13.07.2017 35
Dynamic 3D Scene Capturing Fraunhofer HHI 13.07.2017 36
TimeLab Fraunhofer HHI 13.07.2017 37
Omnidirectional Video High resolution 360 video No stitching artifacts 2D and 3D Real-time end to end streaming Fraunhofer HHI 13.07.2017 38
Sample Productions Fraunhofer HHI 13.07.2017 39
Summary Image processing and analysis is a powerful tool for many applications Significant progress in imaging hardware High-resolution, ubiquitous and cheap cameras VR / AR glasses Increased computing power (mobile devices, graphics boards) Machine learning enables novel AI technologies New applications in water resource management? Fraunhofer HHI 13.07.2017 40
Fraunhofer-Institut für Nachrichtentechnik, Heinrich-Hertz-Institut, HHI WE PUT SCIENCE INTO ACTION. Contact: peter.eisert@hhi.fraunhofer.de +49 30 31002 614 Einsteinufer 37 10587 Berlin Fraunhofer HHI 13.07.2017 41