Vision-Based Human-Computer-Interaction

Transcription

1 Vision-Based Human-Computer-Interaction Ulrich Leiner,, Interactive Media Human Factors, Einsteinufer 37, Berlin, Germany

2 Outline Ulrich Leiner, Heinrich-Hertz-Institute, heading the department Interactive Media Human Factors Today s topics: Eye-tracking and autostereoscopic 3D-displays Gesture recognition and touchless control Query by example: Similarity search

3 Robust Eye Tracking Eye pattern recognition and tracking Fast 2 step approach Head finding trained pattern matching, Eye tracking by adaptive block matching Real time (up 120Hz) Robust against changing illumination conditions Robust against skin color, glasses, hair cut

4 Viewer Dependent Content It happens, when/if nobody is looking at An awarded campaign of Amnesty International Future digital signage may react to Number of viewers Age Gender Pay per view

5 3D Cinema is Now On November 6, 2009, Disney released A Christmas Carol, its latest 3D production in 3683 cinemas Digital Cinemas are spreading quickly DLP projection technology based on digital micromirror devices is the technological enabler for 3D in the cinema How to migrate this success into TV, home and office applications?

6 3D is Easy To enable stereoscopic viewing, left and right eye have to receive their specific image If properly fitting, the brain will combine them to a 3D-image of a scene Left image in green, right image in blue columns Lenticular lens array (or parallax barrier) as beam splitter

7 3D is Easy But Many solutions are available, all with strengths and shortcomings Stereo Displays Glassesbased Autostereoscopic Several Views Holographic Approach > 2 Views 2 Views + Eye tracking

8 Adding Eye Tracking (Position 1) Position 1 The sweet spot for this solution is rather limited Adding cameras to track the user s eye position enables a new solution The lens array is continuously shifted to compensate head movement The sweet spot is increased virtually without limits Position

9 Shifting the Lens Plate

10 Fast Feedback Loop Viewer changes head position Find new eye position Static Image Control loop has to be updated in less than 100ms Lens array shift is calculated Lens plate is moved mechanically

11 Exhibit 1: Free2C Display No glasses High resolution (up to 1920x1200 equals 1300x800 stereo) Broad sweet spot through tracking Very low cross talk Combined with touchless interaction Motion parallax simulation can be added

12 Motion Parallax Simulation Re-use head tracking to update virtual camera position Motion parallax can be added to a CG scene Works for 2D and 3D screens 3D + motion parallax comes close to holographic approaches Medical applications Tele-presence Gaming

13 Exhibit 2: Free2C digital Free2C is a great solution, but moving the lens plate causes high construction efforts Instead of moving the lens plate, subpixels of the screen content are moved digitally Position 1 Position 2 The sweet spot can be moved, if the subpixels are shifted properly

14 Do we Really Need it? Users are highly trained to use monocular depth cues when viewing 3D models on a 2D screen The more complex 3D data is, the higher the benefit of stereoscopic displays Simulation Advertisement Medical Geo-Exploration Dataand Visualization

15 Touchless In 2009 the Bavarian State Library acquired two renowned family book of the Fugger family from 1560 Fuggerorum et Fuggerarum Imagines A world s first: Gesture controled presentation system for ancient manuscripts

16 Camera Based Hand Tracking Hand tracking hardware: 2 infrared cameras with pulsed LED-illumination Three main steps describe the method Hardware calibration and preprocessing of image data elimination of noise and disturbances Data stream interpretation, Definition of semantic patterns mapping of image data and patterns Stop Turn Back Yes Develop application or connect to an application API

17 Benefits Touchless gesture interaction offers several significant benefits over touch screen, mouse or mechanical switches Sterile and hygienic usage Vandalism proof Energy saving: No need for touch foils, which reduce brightness Not limited to arm length for screen size and distance 3D-gesture data offer more versatile input data Gesture languages are virtually unlimited Joy of Use it is simply more fun

18 Application Areas Medical Systems POI/POS Digital Signage Entertainment

19 Query by Example Images are indexed by their own visual content, such as color, texture, contours etc. Database Query image Match No Match

20 Learning by Examples Selection of example images Matching images Learning of relevant image features

21 I m looking forward to answer your questions and discuss your comments Exhibits are shown next door Way more info at