CONTENT ENGINEERING & VISION LABORATORY. Régis Vinciguerra

Transcription

1 CONTENT ENGINEERING & VISION LABORATORY Régis Vinciguerra

2 ALTERNATIVE ENERGIES AND ATOMIC ENERGY COMMISSION Military Applications Division (DAM) Nuclear Energy Division (DEN) Technological Research Division (DRT) Materials Sciences Division Life Sciences Division Mission DAM : France s national security independence Mission DEN : France s energy independence Mission DRT : French business economic competitiveness 2

3 CEA TECH: A WIDE OFFER OF KEY ENABLING TECHNOLOGIES Laboratory of Electronics and Information Technologies Laboratory of Integrated Systems and Technologies Laboratoty of Innovation for new Technologies for Energy and Nanomaterial Solar 3

4 Laboratoire Vision & Ingénierie des Contenus Computer Vision & Natural Language Processing Scene Understanding 3D Perception & Mobility Localisation Augmented Reality Detection/tracking and activity analysis Multimedia Semantic Analysis Text / Image +80 researchers & engineers on projects + 15 PhD students +50 publications / year +30 active patents

5 3D Perception & Mobility Real-time Localisation and Environment perception Augmented Reality Indoor/outdoor geo-localisation Industrial control

6 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

7 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

8 AUGMENTED REALITY APPLICATIONS Marketing Real Time Visual Registration Technologies Navigation Training Industrial maintenance

9 AUGMENTED REALITY APPLICATIONS: 2 main kinds Augmented Objects Augmented Environnements

10 AUGMENTED REALITY & DISPLAYS AR Glasses Concept Lab prototype Transparent screens HUD - Towards full windscreen

11 DIOTA: Industrial Partner For Augmented Reality Applications Projective AR AR software solution AR on tablet PAGE 11

12 RESEARCH ACTIVITIES Augmented Reality 3D Localization & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

13 GUIDANCE Indoor - Indoor Guidance - Mobile Robots - Outdoor - Autonomous vehicles - Pedestrian Guidance - PAGE 13

14 LOCALIZATION BY MULTI-SENSOR FUSION SENSORS Camera: mono, stereo, RGB-D GPS IMU Odometry Lidar

15 CAR GUIDANCE

16 CONNECTRAM Augmented Reality in TRAMWAY Concept Realization Prototype

17 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

18 3D RECONSTRUCTION - Low-cost or wearable systems for 3D Scene Modeling Stereovision Multicaméras Sensor fusion PAGE 18

19 Market Principle Product 3D RECONSTRUCTION A consumer hardware & software product 1 or n camera/sensors Carried or Remote Dedicated to the Virtual Reality market 4+ bracelets/anklets PAGE 19

20 MODÉLISATION DU VIVANT Automatic Physiological Parameters measures

21 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

22 3D CONTROL & MANUFACTURING Defects Detection Measured features : holes, curved edges Automatic CAD Registration Bin Picking PAGE 22

23 3D CONTROL & MANUFACTURING 3D Multispectral Real-Time Vision sensor for Robotics & Industries Fast & Accurate 3D Vision solution: Up to 4 3D-frames per second Depth resolution up to cm Robustness with reflecting objects Multispectral data Useful for material identification applications, True Color Numerisation ( independent from ambient light) PAGE 23

24 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

25 MATERIAL IDENTIFICATION Multispectral cameras better material segmentation in images Brick Concrete Metal (aluminium) Wood PVC Rust Plaster 3D Rust Detection PAGE 25

26 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

27 SCENE ILLUMINATION ANALYSIS Shadows Detection Robustness to Light Changes Realism of Scene Augmentations Specularities Segmentation PAGE 27

28 SCENE ILLUMINATION ANALYSIS Input video Retexturing PAGE 28

29 RESEARCH ACTIVITIES Augmented Reality 3D Localisation & Mapping 3D Reconstruction 3D Control & Manufacturing Material Segmentation Scene Illumination Analysis Diminished Reality

30 DIMINISHED REALITY Toward a full concept Reality Diminished Reality Augmented Reality PAGE 30

31 Scene Understanding Videoprotection & Video Assistance Infrastructure Security Driving assistance Smart Home / Smart building /Smart City

32 IMAGE ANALYSIS DETECTION, TRACKING, RECOGNITION Generic algorithms Various sensors (Visible, IR, IR/Visible, Fish-eye, PTZ ) Various Contexts (indoor, outdoor, broadcast) Fixed or Embedded Urban Transportation Infrastructure Automotive Building Manufacturing Smart Home

33 DEEP LEARNING MULTI-CLASS AND PRECISION Challenge KITTI : Automatic 3D Analysis of Vehicles (1) Vehicle detection (2) Type recognition (3) Fine pose estimation (4) 3D location (5) Visibility of parts CEA APPROACH : DEEP MANTA 1st on detection and orientation estimation Close to Real time (3 Hz on VGA) Accuracy +30% vs Sota, similar to stereo approach

34 FACE RECOGNITION REAL TIME FACE DETECTION FACE VERIFICATION: Examples of match pairs

35 Deep Learning for Face Recognition (2016) Face detection & alignment Deep Multi-Task Multi-Scale Cascaded Neural Network Feature extraction Deep Metric Learning from Tiplet Network Video query Authorized access! Face Recognition Verification: is that face of a given people? Retrieval: similar faces ranked according to learned metric face features (1792-dim) projected in 3-D space Image query ranking of faces This is!

36 SECURITY: VIDEOPROTECTION Real-time Analytics Camera Network & 3D Display Operated in VisionLab Behaviour Analysis Perf Evaluation

37 SAFETY: PEDESTRIAN DETECTION BLAXTAIR : Embedded Vision System to enhance safety in critical industrial contexts Thesis Collaborative Projects Arcure creation Joint Lab CEA/Arcure 1st prototype BLAXTAIR Vehicle Rotation Large Zone Close FOV Variable Posture Non-planar ground Obstacle segmentation TECHNOLOGY POSITIONNING Robust 2D/3D analysis : TP= 99 %, FA= 1/jour Learning on specific image database : positives > 1 million negatives Real time: 15 Hz (2 cœurs ARM) 5 patents

38 SAFETY : VISION SYSTEM FOR PEOPLE DETECTION Obstacle Detection Lidar/ IR bareer People Classification Blaxtair / IR Video SICK

39 AMBIANT LIVING Eating Setting the table Vulnerable people Activity Recognition Applications Safety Security Energy Comfort IHM Sensor fusion Camera IR detector Home sensing Platform MobileMii TV Control

40 Multimedia Semantic Analysis of Text, Image, Video Extraction Synthesis Retrieval

41 EXTRACTION D INFORMATION : IMAGES

42 EXTRACTION D INFORMATION : TEXTES Cherchons Lemme Racine chercher cherch

43 VISION AND CONTENT ENGINEERING LAB CEA List/LVIC 70 researchers 15 PhD students 3D Perception Augmented Reality 3D Localization Scene Understanding Surveillance systems Assistance systems Multimedia fusion Information retrieval Synthesis

44 JOB OPPORTUNITIES We are recruiting! Our lab recruts non permanent researchers & engineers almost thoughout whole the year Main research domains: Computer Vision Deep Learning Sensor Fusion (mainly Vision/IMU) (Common minimal skill: C++) Main Contact: Other contacts: 3D Perception & Mobility: Scene Understanding: Multimedia fusion: