SEMI-AUTOMATED ANNOTATION

Transcription

1 VIDEO ANONYMIZATION VIA SEMI-AUTOMATED ANNOTATION SIS77 Automated vehicle data sharing enabled by Feature Extraction and Anonymization Marcos Nieto - Vicomtech 1

2 OUTLINE Vicomtech Privacy Masking Semi-Automated Annotation Concept Deep Learning Architecture Technologies Open issues 2

3 Applied Research Centre, founded in 2001, specialising in Computer Graphics, Visual Computing and Multimedia Technologies Strong focus on technology transfer +125 Staff (30% Ph.D.) PhD., engineers, computer scientists,. ) International Team Ph.D. From (among others) Imperial College London (UK), Cambridge University (UK), Manchester University UMIST (UK), ETH Zurich (Switzerland),, Technical University of Darmstadt (Germany), Ohio State University (USA), University of Navarre (Spain), Basque Country University (Spain), Deusto University (Spain) Paseo Mikeletegi 57 San Sebastian, Spain H2020 coordinators Partners: Valeo, TomTom, Intempora, IBM, Intel, TASS (Siemens), DCU, XL, Honda Research Institute, TUe, ULIM, CEA, ERTICO, AKIANI, IFSTTAR, XL Groups, etc. 3

4 PRIVACY MASKING Concept To protect regions of images which contain information that can be used to identify individuals Context Modern test cars are recording massive amounts of video footage on streets, roads, etc. Images of faces, license plates, bodies, telephone numbers are being recorded. Problem How to protect this content whitout stopping progress? How to be compliant with the GDPR? Approaches Semi-automated processing platforms can be used to anonymize videos Video encryptation standards need to be used to ensure original video is kept/recovered. 4

5 PRIVACY MASKING ROI configuration (XML) Extract ROIs Person / License plate Annotation Encrypt Anonymized video GStreamer multiplexer Encryption keys 5 5

6 SEMI-AUTOMATED ANNOTATION Manual labour is usually more reliable, but way slower and expensive Automated computing can be much faster, but incomplete and/or unreliable Semi-automatic annotation can be the solution Manual? Automated? Semi-automatic Human-machine interaction Users validate / correct / verify automatic annotations Online / Iterative learning mechanisms Expert annotators and scientists Efficient annotation tools (GUI) Multi-sensor information fusion High Performance Computing resources 6

7 DEEP LEARNING DL can help can be executed in batch against millions of images simultaneously DL is not perfect no 100% will ever be reached: human intervention is always needed DL can learn from feedback human feedback can be used to re-train DL, to asymptotically approach to 100% DL is not free need to be run using High Performance computation platforms (e.g. GPU-enabled cloud) 7

8 Vehicle Cloud Tools and algorithms (computer vision, deep learning) Update Algorithms / Learning Process SEARCH ENGINE Search Autonomous Driving Function Continuous training and Validation Data/Scenario Requirements: Training Validation External DATA Other Open Data DATA Annotated Data Groundtruth Data Synthetic Scenario & Data ADAS Functions Design & Training Anonymised Video + Labels Anonymised Video + Labels Synthetic Data ADAS Functions Validation FOT Data Automatic Labelling: Semantic, Bbox, 3D Point Cloud Ground Truth Generation Interactive labelling Data Management System Synthetic Scenario generation Simulated Scenario Scenario Exploiting: Dynamic + Static parameter variation Retrain, refine, update algorithms from new models trained in the Cloud Perception Sensors Real Scene Recordings Pre Annotated Data Real Scene Recordings Raw Data AI based Algorithms for online data Mining Edge Computing In-vehicle pre-annotation Scenario selection Specific manoeuvre recording Embedded Processing: Autonomous Driving Function Perception Module ADAS Functionality 8

9 OPEN ISSUES Are anonymized videos an asset to store permanently? OR videos are anonymized only during their usage (for ground truth generation)? If original videos are to be kept untouched stored anonymized videos duplicate (multiply) storage resources Anonymization Separate files ( masking ) Orig. video Masked video Single file ( encrypting ) Is there a legal framework to be applied during annotation? Are there privileged users that can see the videos and others that cannot? Annotation process implies human intervention -> private data is exposed to persons during annotation Automatic labelling (e.g. face detection) is not perfect human validation is needed Masked videos affect performance of computer vision algorithms Encrypted videos can be de-encrypted and not affect performance Multi-stream Encrypted video 9

10 Multi-sensor annotation In-vehicle processing Cloud Deep Learning 10

11 THANKS! Visit us at Vicomtech C