Eye tracking by image processing for helping disabled people. Alireza Rahimpour

Transcription

1 An Introduction to: Eye tracking by image processing for helping disabled people Alireza Rahimpour Fall

2 Eye tracking system: Nowadays eye gaze tracking has wide range of applications in human computer interaction. One of these applications is using trajectory of eye gaze instead of foot or hand for disabled people to execute some commands. Other Applications: ü Automatic monitoring of drivers for accident avoidance ü Commercial uses ü Psychological uses 2

3 Different types of Eye trackers 1-using EOG signal: Systems based on electrooculography use electrodes placed on the face to detect the movements of the eyes Intrusive 2-using active infrared illumination v Difficult calibration v need high resolution and expensive cameras v Interference : External sources of IR : Sun 3

4 Other types: 4

5 GOAL Design an Eye tracking system as human computer interface with Avoiding use of: ü specialized hardware ü infrared light sources ü expensive cameras And ü Increasing the accuracy and speed of the system to be usable in realtime applications 5

6 Eye tracking system Recently, various methods have been proposed, some of these methods can successfully track the eye gaze. However, they always require specific circumstances, training or are not capable of real-time performance. Track eye gaze in real-time by using a simple and low cost webcam mounted on ordinary laptops. Be able to : control the motions of mouse cursor and click on an onscreen keyboard in real time 6

7 Implementation Several methods designed and implemented for facial and eye feature extraction. One of them is: Template matching Template of Feature The region of image with maximum similarity with the template, refers to position of our feature in image. 7

8 Template matching Similarity between image and Template?? Minimizing the error: Parametric templates ü Correlation function 8 ), ( ), ( ), ( = = + + B j B i t j v i u I j i T v SAD u

9 Correlation function as a measure of similarity For template matching the template, t slides over f (image) and C is calculated for each coordinate. After calculation, the point which exhibits maximum C is referred to as the match point. u, v = 9

10 Template specifications Size size of the template is not the only issue, but more importantly, tracking performance depends on the complexity of the template. Complexity: have enough brightness variations, e.g., texture or lines, to be recognized as distinct features 10

11 Templates Templates with min size which contain enough details are chosen. Face template: 75x106 pix Eye template: 25x15 Pix Pupil template: Eye corners: 11

12 Enhancing the accuracy and speed Calculation of normalized correlation coefficients for each pixel is very time consuming. Accuracy and speed have inverse relationship with search windows size: 12

13 Reducing the search windows size ü Reduce the search area in each step by cropping ü Using structure of face for determine the search region for each feature. 36X45 75X106 25X15 320X240 13

14 For increasing the accuracy: Accuracy the weighted correlation coefficient is used. Considering several frames of image, the Maximum index in correlation coefficient matrix can be determined. For next frames we consider more weight for the winner index( and a few neighbors) 14

15 Tracking system The system consists of two main modules: 1) the face detector and tracker 2) the eye analysis module The output from the eye module can be the input to a simple computer control interface by simulating a key press. Also: Different modes are defined correspond to time of closed eyes 15

16 EYE GAZE ESTIMATION Horizontal direction 1-using sclera region changes 2-using center of pupil Vertical Direction 1-using eye corners 2-using upper eyelids 16

17 Eye gaze in horizontal direction using sclera region Dividing the monitor in horizontal direction to 16 point and define alpha as follow: alpha 1 α = A A( L) ( L) + A( R) position

18 Sclera Region extraction Cropped eye image Contrast enhancement Convert to binary image using a threshold Morphological operations (opening closing) Making contour and calculate the number of pixels in them And the coordinates. 18

19 Eye gaze detection horizontal direction 2-Using pupil center coordinate with respect to eye corners Vertical direction: 1-eyelids: 2-eye corners 19

20 Experiment and results ü Frame size 320 x 240 pix 30 frame per sec ü Distance of user to monitor(17 ) : 60 cm ü Data set : real time video taken from regular webcam on laptops. ü No specific illumination condition ü Tested on two on screen keyboard: ü 7x3 and 5x5 keys. 20

21 Result( face tracker module) Robustness to : different users and background changes and tilt of head Fast moving of user in front of camera 21

22 DEMO Results: Eye gaze detection (with method 2) Using 7x3 on screen keyboard: 22

23 Results: Detection rate Face illuminati on Keyboard Vertical Pre. Horizontal precision Method 76.6% sensitive 5x5 4 deg. 6.8 deg. Method 1 90% Non sensitive 7x3 2.8deg 11.3deg. Method2 Method1: using sclera region for horizontal detection and eye corners for vertical Method2: pupil center for horizontal and eye corners for vertical detection Eye tracking system: ü Real time ü Not expensive ü More accuracy compare to similar systems ü No need to training and overwhelming calibration 23

24 Any Question? 24

25 25

26