A COMPLETE SOFTWARE SOLUTION FOR A HANDICAPPED USER TO ACCESS A COMPUTER SYSTEM USING HEADMOUSE TECHNOLOGY & SPEECH RECOGNITION

Transcription

1 International Journal of Emerging Technology and Innovative Engineering Volume I, Issue 6, June 2015 (ISSN: ) A COMPLETE SOFTWARE SOLUTION FOR A HANDICAPPED USER TO ACCESS A COMPUTER SYSTEM USING HEADMOUSE TECHNOLOGY & SPEECH RECOGNITION Pavan Shah Sinhgad Institute Of Technology & Science pavanshah77@gmail.com ABSTRACT According to a recent survey, there are 650 million handicapped people in the world. Moreover, there are 39 million visually challenged people and 245 million with low sight. These people cannot access the computers due to their disabilities. There is a need of some technology which allows these people to get access of the mouse and keyboard without using hands and eyes. This paper gives us the brief introduction of Helping Hands Technology which is intended to help handicapped users in getting access of the basic technologies. This technology uses the head mouse technology by which the real time head movements of the computer user is translated into a directly proportional cursor movement of the computer mouse. This technology also provides an interface for the user to handle the click event of the computer. A handicapped user can access the mouse of the computer without using hands. The other module of this system is the speech recognition technology, which allows the computer user to give commands to the system by using the keywords specified in the natural language. A blind user can give speech commands to the computer allowing him to access different applications of the computer. This system also provides a screen reader to the user which can read different documents, internet files, s of the user. 1. INTRODUCTION The Helping Hands Technology is divided into 3 main sections - 1] Head Mouse Technology - The Head mouse translates natural movements of a user's head into directly proportional movements of the computer mouse pointer, so as the user moves their head the mouse pointer on the screen also moves. The Head mouse has a wireless optical sensor which tracks a tiny disposable target that is worn by the user in a convenient location on their forehead, glasses, hat, etc. It works just like a computer mouse, with the mouse pointer being moved by the motion of the user's head. The Head Mouse will track the user's head with the user located in any comfortable viewing position relative to the computer display. Resolution of the Head mouse is precise to allow a user to control the mouse pointer down to the minimum, pixel perfect, resolution of the computer display. This precision allows a user to perform such tasks as drawing, gaming, graphics work. 2] Speech Recognition Technology- Voice recognition is an alternative to typing on a keyboard. Put simply, you talk to the computer and your words appear on the screen. It can provide a fast method of writing onto a computer and can help people with a variety of disabilities. It is 59

2 useful for people with physical disabilities who often find typing difficult, painful or impossible. Voice recognition software can also help those with spelling difficulties, including users with dyslexic, because recognized words are always correctly spelled. 3] Screen Reader Facility- Screen Reader is the software program which can read the documents, internet files and s for a blind user. It reads the complete text present on the computer screen in a synthetic voice.screen Reader can be used for reading the contents which are available on screen as well as we can even read the mails which are available on the internet These three core concepts of the image processing domain are combined in Helping Hands Technology to achieve the maximum accuracy and reliability. 2. SYSTEM ARCHITECTURE 3. MAJOR ALGORITHMS Following four algorimths are mainly user in the implementation of Helping Hands Technology 1] Gray scaling Algorithm 2] Blurring Algorithm 3] Thresholding Algorithm 4] Blob Detection Algorithm 3.1 Gray scaling Algorithm We convert coloured images to gray-scale images because gray-scale images are easier to process and increase the accuracy of head tracking. RGB MAX = (R MAX + G MAX + B MAX) /3...1 R MAX =Maximum value of red color G MAX =Maximum value of green color B MAX =Maximum value of blue color RGB MIN =(R MIN +G MIN +B MIN) /3...2 R MIN =Minimum value of red color G MIN =Minimum value of green color B MIN =Minimum value of blue color V=RGB MAX 3 If (V=0) H=S=0 Else S=255*(Max- Min)/V...4 If(S=0) H=0 Else If(S==R) H = (G-B) / (RGB MAX RGB MIN ) If(S==G) H = (B-R) / (RGB MAX RGB MIN ) If(S==B) H = (R-G) / (RGB MAX RGB MIN ) 3.2 Blurring Algorithm Blurring an image reduces sharpness of the image thus making the image easier to detect. Pixels are mixed with each other using following algorithm - for (int y=1;y<heightofimage-1;y++) 60

3 for (int x=1;x<widthofimage-1;x++) Sum = 0; for (int yy=y-1;yy<=y+1;yy++) //get the values of surrounding pixels for(int xx=x-1;xx<=x+1;xx++) col = in Pixels[yy][xx]; b = col & 0xff; g = (col >> 8) & 0xff; r = (col >> 16) & 0xff; sum += (r+g+b)/3; // adding grayscale component to sum. // average of 8 surrounding pixels and centre r = g = b = sum / 9; out Pixels[y][x] = (b (g<<8) (r<<16)); // Storing the calculated values 3.3 Thresholding Algorithm Thresholding is mainly used to Convert multicolored image into binary image. Colour is assigned based on particular threshold value. This results in separation of background and foreground of the image. avg = (r + g + b) / 3; // grayscale if(avg< th) Pix = 0; // pure black else pix = 0xFFFFFF; // pure white 3.4 Blob Detection Algorithm Blob detection concerns with detecting our point of interests. After separating foreground and background we detect the blob for our image i.e head using discrete convolution. 4. ADDITIONAL SOFTWARES With head-mouse being the major part of the system, following two softwares are used in unison to make it a complete software solution for a handicapped or blind user. 4.1 Speech Recognition Software Speech Application Program Interface(SAPI) is used in this technology for giving speech commands to the computer system. SAPI is a freeware software from Microsoft and has a database for storing speech commands. We can feed the required commands in the software in one time. The software compares real time speech commands with the database to execute appropriate action. SAPI provides an excellent accuracy in recognizing speech commands and triggering the corresponding command using command prompt. 4.2 Screen Reader Software A JSAPI Text-To-Speech(TTS) software is used for screen reader functionality. Screen Reader can be used by a blind reader for reading documents, web pages, s etc. Screen Reader can read offline documents and it can also use RSS feeds to give latest updates from internet. Screen readers are also available in regional languages. 5. HELPING HANDS SOFTWARE 1] User Interface - 61

4 5] Speech Test 2] Manage Speech Commands 6] Complete Interface 3] Mail Settings For Blind User 4] Head Mouse Camera Test 62

5 6. ACKNOWLEDGMENTS I would like to thank Mr. Nirav Shah, Mr. Piyush Bhattad, Mr. Rahul Pardeshi and entire team of helping hands software for actively contributing in the software development. I would also like to thank Columbia Infosys for sponsoring this project. 7. REFERENCES [1] Bowman, M., Debray, S. K., and Peterson, L. L Reasoning about naming systems.. [2] Ding, W. and Marchionini, G A Study on Video Browsing Strategies. Technical Report. University of Maryland at College Park. [3] Fröhlich, B. and Plate, J The cubic mouse: a new device for three-dimensional input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems [4] Tavel, P Modeling and Simulation Design. AK Peters Ltd. 63