ELECTROOCULOGRAPHY HUMAN COMPUTER INTERFACE ECE-492/3 Senior Design Project Fall 2013

Transcription

1 ELECTROOCULOGRAPHY HUMAN COMPUTER INTERFACE ECE-492/3 Senior Design Project Fall 2013 Electrical and Computer Engineering Department Volgenau School of Engineering George Mason University Fairfax, VA Team members: Faculty Supervisor: Mark Jeiran, Ammar Palla and Xue Yin Dr. Wilsaan Joiner Abstract: The Electrooculography Human Computer Interface (EOG HCI), or as we like to call it, The Eye Mouse, is a low-cost device which simulates four directional movements and clicking of a computer mouse through manipulation of horizontal and vertical eye movements and blinking. As eye movements are the most frequent movements in humans and sufficient control of eyes is still maintained even among people with motor disabilities in the upper body [1], we developed an eyes-controlled computer cursor. This has been achieved through measuring physiological signals produced by the potential between the cornea and retina in the eyes. The device includes a pair of 3D printed goggles with two pairs of electrodes mounted on the frame, a filter/amplification circuit, and a microcontroller unit connected to the computer. Each pair of electrodes is responsible for capturing signals generated in horizontal or vertical directions during eye movement. 1. Introduction According to the United States Census of Bureau, the number of people requiring assistance for ordinary tasks has grown to 12 million within the U.S. in 2012 [2]. While computer technology can help these people independently handle a wide range of activities such as reading, communicating, and searching information, they cannot provide computer inputs with a standard keyboard or mouse due to motor disabilities in the hand and arm. The demand to provide equal access to computers motivated our project. The goal was to find a convenient and inexpensive alternative for standard computer operations. The solution we proposed to accomplish this goal was The Eye Mouse (EOG HCI). Requirements for such a device included: (1) EOG-based control, (2) cost under $100, (3) portability and ease of use, (4) USB powered, and (5) performance within 50% or greater of a typical computer mouse. 2. Approach To ensure compliance with the project requirements, overall design (Figure 1) was broken down into four major parts. Part A includes a data collection apparatus. This is an easily adjustable pair of 3D printed PVC goggles containing an array of electrodes. Two pairs of reusable silver/silver-chloride electrodes are used to sense EOG 1

2 signals with low noise. Part B contains the circuit board. The EOG signal is amplified, filtered and forwarded to an analog-to-digital converter (ADC) in a microcontroller. Part C includes the microcontroller, ADC, and it connects to the target computer via USB. The device acts as a generic USB mouse, thus is usable across various platforms and operating systems. Part D includes a benchmark software that is freely accessible offline, or on the web. System functional decomposition is shown below. User Monitor Mouse movement Virtual mouse driver Movement data MATLAB data processing MATLAB Data Acquisition Toolbox Computer Bio-potential signals from eyes USB Cable Digital data from MCU Acquire signals from probes Acquire horizontal signal Acquire vertical signal Prepare signal for USB interface ADC A (Horizontal) ADC B (Vertical) ATxmega128A1U Unfiltered signal INA118P Unamplified signal UA741 Op- Amps (x3) Amplified analog signal Figure 1: System components, electrode placement around eyes, and functional decomposition The combination of all processes was divided into states, from the signal detection to the movement of the mouse cursor on the display. These states are shown below in Figure 2. In order to distinguish eye movements in two dimensions, bio-potential signals are processed separately for horizontal and vertical movements. 2

3 No movement Start signal detection Idle Move up Perform Click Process data Move down Move right Move left Figure 2: System state diagram 3. System hardware Primary functionality of the circuitry is to receive the EOG signal from electrodes and feed it to the microcontroller board. Along the signal s route the input signal is amplified, filtered, and modified to a shape and range appropriate for our application. Figure 3 illustrates the final circuit design and a 3D model of the populated multilayer PCB that was designed. The amplified signal enters an 8 th order Butterworth filter with a Sallen Key configuration. The filter characteristics include: (i) cutoff frequency 30Hz, (ii) stopband frequency 60Hz, and (iii) attenuation 45dB. Figure 4 shows the filter frequency response. The output stage is a differentiator RC circuit. The Atmel SAM4S Xplained Evaluation Board (32-bit ARM-based ATSAM4SA16C with a 12-bit ADC and 11 conversion channels) was used to digitize the signal from the circuit and evaluate signal patterns for movement. Once the evaluation was complete, the microcontroller sent the appropriate USB cursor movement signals to the personal computer. 4. System software The software for the microcontroller was written in C with help of Atmel Framework Libraries for cross-platform capability. The microcontroller runs in an infinite loop and generates software interrupts to start analog-to-digital conversions (ADC). Each ADC records data from two separate ADC channels and stores them in separate buffers that can hold 512 samples each. These buffers are circular, so the same buffers are used when the data exceeds 512 samples. In addition to this data, a difference of the present and past value is also recorded with each 3

4 conversion. This data is stored in two other buffers that hold up to seven differences for each channel. The final operations of the microcontroller involve executing an algorithm to recognize different patterns in the signal, and then moving the mouse cursor accordingly. Figure 3: Electronic circuit and a 3D model of designed PCB Figure 4: Frequency sweep output of the designed circuit 4

5 In order to perform the cursor movement, two algorithms were used. One algorithm detects the horizontal signal, and the other is used for the vertical signal. The general correlation of the eyes to the movement of the mouse cursor can be seen in Figure 5. The algorithm detects whether the signal is above the threshold of the center eye position value and increments a counter. Each ADC result that exceeds a vertical or horizontal threshold will increment the corresponding counter. After the signal falls back within the thresholds, the counters are checked. If the counters are greater than a certain calibrated value, then the mouse cursor vector is set in the corresponding direction. Blinks are detected in the same manner with the exception that blinks include a signal that goes above and below two thresholds so two counters must be incremented. Eye Movement Action Cursor Action Look Center Cursor Stationary Look Left Look Right Look Up Look Down Cursor Moves to Left Cursor Moves to Right Cursor Moves Downwards Cursor Moves Upwards Figure 5: Eye to cursor correlation 5. Benchmark Software The application used to test the functionality of the cursor movement was developed in HTML, CSS, and JavaScript. This application is designed to work in most modern browsers that support JavaScript. The application is based on Fitts s Law and uses a series of 22 unique trials that are each repeated three times in succession. As the test proceeds, the application also plots the data and allows the user to generate a table to reproduce the linear regression line. After a test is complete, the application outputs the linear regression co-efficient above the progress bar along with a pass or fails status. Figure 6: Testing application plotting functionality 5

6 5. Experimental Evaluation and Validation As EOG signals of the four directional movements are all square shaped, the same algorithm for the detection of the rising and falling edges of the signals was developed and tested in Matlab. The algorithm combines differentiation and threshold methods to detect the beginning of the movement. In Figure 7, the first window is a band-pass filtered rightward EOG signal. The difference signal derived by subtracting the present value from the previous one is shown in the second row. The threshold value was set at a quarter of the maximum value in the corresponding direction. Any signal greater than the positive threshold or smaller than the negative threshold was kept. Otherwise, the value was set to zero. As indicated in the third window, the algorithm is capable of detecting the starting and ending points of movement. Further tests indicated that the sensitivity of the algorithm varied depending on the threshold values and the regularity of the difference signals below the threshold. As the difference signal was not affected by signal drift and the onset of movement is shown by steep pulses, this difference signal directly obtained from the circuit was used as the input for eye movement detection. To produce a joystick-like movement, we tested the possibility of mapping speed with the amplified horizontal signal amplitude. As shown in the second window of Figure 8, high value in the vertical axis represented high speed. Consecutive pulses in the same direction suggest continuous eye movement in the same direction. With this information, velocity could potentially be mapped. While most pulses in the horizontal direction generated different speed levels in the simulated signal, this algorithm was not accurate enough to map speed at each instant of time. The codes were further modified and tested directly in MCU. Figure 7: Sample experimental results for eye movement in the rightward direction 6

7 Failed detection Failed detection Failed detection Figure 8: Speed mapping of horizontal movement in Matlab The EOG HCI, or The Eye Mouse, was tested separately for horizontal and vertical movement. Summary of the results are shown in Table 1. The horizontal movement test was performed using a testing application we designed based on Fitts s Law, and the vertical movement was tested for consistency against other conventional mice using repeated trials. Table 1: Summary of test results Movement Result Comparison to Conventional Mice Horizontal linear correlation co-efficient 41.1% less performance Vertical ms average movement time 77.5% slower movement References [1] Bridgeman B., "Conscious vs Unconscious Processes," Theory&Psychology, vol. 2, no. 1, pp , [2] "United States Census of Bureau," 25 July [Online: ] 7