THE IMPLEMENTATION OF CUVOICEBROWSER, A VOICE WEB NAVIGATION TOOL FOR THE DISABLED THAIS

Transcription

1 THE IMPLEMENTATION OF CUVOICEBROWSER, A VOICE WEB NAVIGATION TOOL FOR THE DISABLED THAIS Proadpran Punyabukkana, Jirasak Chirathivat, Chanin Chanma, Juthasit Maekwongtrakarn, Atiwong Suchato Spoken Language Systems Group Department of Computer Engineering, Faculty of Engineering Chulalongkorn University Thailand proadpran.p@chula.ac.th, ch.jirasak@gmail.com, newbiess@gmail.com, beer_ju16@hotmail.com, atiwong.s@chula.ac.th ABSTRACT Surfing the web is, today, a daily activity for most. While we utilize the web as a means of publishing information to the world and accessing information from wherever, less fortunate people do not have that privilege and this includes the blind and the motor-handicapped. This paper explains how we have designed and built the CUVoiceBrowser, a web browser that can be controlled by voice in the Thai language to serve the Thai disabled group. While many have worked on text-to-speech capabilities in reading web pages to the blind, we focus on taking commands from the blinds and the motorhandicapped, in Thai, to navigate the web and to search for information on the web. Our prototype shows high accuracy rate of over 80% when users speak their web navigation commands, and over 70% when Thai characters are inputted by voice. KEY WORDS Web browser, speech recognition, text-to-speech, and assistive technology 1. Introduction Web accessibility centered around voice input will one day allow anyone to use the web without training [1]. More importantly, this technology will allow the blinds to participate in the growing web community. A summary of the progress of web accessibility is given by Asakawa [2]. Many researchers have built prototypes to demonstrate the workability of Voice-web concepts. Hemphill, et.al., have developed voice controlled navigation using speaker independent speech recognition that supports a speakable hotlist, speakable links and smart pages in the speech user agents [3]. Parente [4] has developed a prototype of audio enriched link that will provide a summary of web pages for those web pages accessed by the blind. Most of the work done in this area is based on the English language. However, there have been experimental projects attempted by other non- English systems. Brondsted, et.al., have built the prototype of a Danish voice-controlled utility for internet browsing targeting motor-handicapped users who have difficulty in using the standard keyboard and/or standard mouse [5]. And Lopez, et.al., [6] have offered a tool to help the visually-impaired to surf web in Spanish but does not have a voice-recognition capability in place. To date, IBM ViaVoice and Homepage Reader are the only commercial tools that support the Thai language. While ViaVoice does recognize Thai language, it is not a tool to navigate the web. Although IBM Homepage Reader does read homepages or the web aloud in Thai, it does not have a feature to recognize commands from users. The blind and the motor-handicapped share similar difficulties when it comes to surfing the web. They cannot use a keyboard or a mouse to input commands. This fact instantly limits their use of the web. In fact, the blind might be in a better position if they could use a Braille keyboard together with text-to-speech software. Unfortunately, those whose hands are not capable of typing will find the keyboard to be of no use. In addition, the Thai disabled community is not necessarily proficient in English. This further limits them when seeking knowledge through the web. Hence, this project was initiated to build a web browser, the CUVoiceBrowser, that understands simple web navigation commands in Thai. Although both the blind and the motor-handicapped are considered as the users of this tool, the design of this tool is more focused on the blind who have more limitations in terms of viewing the web. 2. Design and Implementation Criteria The design of CUVoiceBrowser web navigation tool has taken into consideration the criteria that both general and particular Thai handicapped people are the users. We allow simple voice commands that are appropriate in the Thai language which may not be exact translations from English. Particularly the blind, as they are not able to see the screen, are unable to understand even fundamental concepts like normal people. These include frame, color, blinking text, pictures, etc. This simple framework does not reduce the capabilities nor make it harder for the motor-handicapped. Our design criteria mainly focus on ease of use and ease of

2 understanding. Taking these criteria into account, the resulting design is as follows: 2.1 Minimal Training To minimize the training needed, the tool only requires the users to listen to the guidelines at the beginning once the user opens the program. It tells the user to locate Alt and the Space bar which they will use to activate each command. It will also list available commands that users may use. These guidelines are kept as a help menu that the user can review at anytime. As a result, users will need no formal training in the use of this tool. The structure of Help and guideline is shown in Figure 1. Whenever the users want to activate the help function, they can say Help in Thai at anytime. Help.html Introduction (Welcome) Control: How to record (Ctrl + Spacebar) How to pause Essential commands (Read, Read Next, Read previous) Link command (Goto link) Guideline (one.html) (How to use the tool step by step) How to record sound How to input addresses and goto addresses What happens when finishing loading Other commands Techniques (two.html) All commands (three.html) Program limitation (four.html) Figure 1 Structure of Help and Guideline for the users 2.2 Web Navigation Commands In developing the Thai language commands, we have attempted to use the fewest set of commands as possible. Also, we have tried to select commands that do not sound similar to each other so that accuracy will be higher. One more consideration we would like to note here is that the commands in Thai are not direct translations from English. This is to reduce the frustration that might be experienced by users should they have to learn new technical jargon. As a result, the following groups of functions have been defined and implemented. The commands enclosed in are in Thai. Here, it is presented in English for the purpose of publishing. a) Control Functions: i. Open a new page - Open new tab ii. Close page - Close the current tab iii. Next page - Change to next tab iv. Exit - Exit browser b) Page access functions: i. Previous - Back to previous page ii. Next - Forward to next page iii. Reload - Refresh the current page iv. Stop loading stop loading the new page v. Input address Entering input mode, voice input of characters and special characters forming URL vi. Go Go to the specified URL vii. Search Entering search mode, voice input of search item then Web Search Engine will be activated c) Web reading functions i. Read Read the current paragraph ii. Read next Read the next paragraph iii. Read previous Read the previous paragraph iv. Read again Read from the beginning of the current page v. Stop reading Stop reading the text vi. Next link Read and select next link vii. Previous link Read and select the previous link viii. Open link Go to the specified link d) Other commands i. Help Go to help menu ii. Bookmark Go to bookmark page to store the desired page There are altogether 21 commands to navigate the web. The users do not need to memorize all the commands as the tool will ask the users what they would like to do at each stage. The options will be given at each stage or the expected response will be intuitive enough without instruction. Please note again that all the commands are in Thai. Among the 21 commands, the most used are Read, Read next, and Search. 2.3 Language Input The highlight and the challenge of this tool is the ability for users to input the desired information in Thai using their voice, in addition to English. This is particularly useful when users want to search for something. They may use Thai, English, number, or a special character as the input and they can say one character at a time to the tool. The tool will recognize it and echo the result back to the user. If it is incorrect, the user can repeat that letter again. This paper will not go into details on the English character set. However, it is worth explaining how the Thai language is characterized and how we have designed our system to manage this task.

3 Figure 2 Architecture of the Tool Thai language In the Thai language, there are 46 consonants, 21 vowels, and 4 tone indicators to indicate 5 tones. Out of 46 consonants, there are 44 in use. Our tool allows the use of all 44 consonants, 21 vowels, and 5 tone indicators. The complication arises because some of the characters sound exactly the same although the autography of the letters is different. Therefore, to capture the right one, the user is asked to say that letter along with the main word sample attached to that letter. Though it may sound complicated, it is rather trivial for Thais since it is how the language is taught in school when Thais start to learn. The format of this input is similar to A-Apple, B-Boy in English. However, when the user wants to input a vowel, they say the word vowel in Thai, followed by the desired vowel English language When users want to input an English character, they only have to speak that letter without saying the sample word as they have to do when inputting Thai. There are two reasons for this approach. The first is that there is no duplicate sound from those 26 letters. The second is that it makes it automatic for the tool to understand whether the user is inputting English or Thai character Numbers and Special Characters Numbers 0 to 9 can be inputted by saying them in Thai. Again, since there is no duplicate sound, the tool will understand that the user wants a number once it is said. For special characters, most are allowed and they are listed in the guidelines for users to see what can be used. Examples are -, #,?, ~,, (comma), :, +, -,. Users will say these in Thai as well. 2.4 Utilities We have designed our speaking Help function not only to minimize training process, but also to serve as a utility for users. In addition to the Help menu, it is important to select utility that is useful to the users. To save users time, users can also bookmark their favorite pages. Favorites are kept for them provided that user sets a keyword. Keyword may be anything such as the name of the newspaper, name of the bank, etc. Once a bookmark is set, user is able to go directly to their bookmark in one step by saying the keyword of the bookmarked page. Another option to reach the kept favorite pages if user cannot remember the keyword is to say bookmark. The tool will read aloud the list in the bookmark one by one. Once the desired page is read, user

4 can say open link, and the tool will automatically navigate to that page. 3. Architecture of the Tool There are three main components when building this CUVoiceBrowser; the browser, the recognizer and the text-to-speech. We have used Microsoft Visual Studio, to build the browser and its functions. In the case of the recognizer, we built it using HMM technique and use HTK as a tool. For text-to-speech, we simply called a library from IBM ViaVoice to do the work. In this paper, we will only give details on the browser and the recognizer that we built. The overall structure of the architecture is shown in Figure 2. Browser Microsoft Foundation Class (MFC) has been used to build our browser. As shown in Figure 2, WebBrowserView handles the displaying of webpage. The navigation tasks are done by calling the functions in WebBrowserView and the function is sent to WebBrowserCore which executes and controls each navigation task. When the user presses Ctrl+Spacebar, the WebBrowserView will accept the event from the keyboard and understand that the user now wants to input a voice command. Then, the MainFrame function will call Recorder which will record the voice command from the user before sending it to the Recognizer. The Recognizer in turn calls HTK to execute the built-in module for Thai and English commands which will perform the recognition task. The recognizer here does the entire job of recognizing what the users say, including web navigation commands and the character input; Thai, English, and special characters, and numbers. The output from the Recognizer is a string of text that is one of the commands that will be sent to MainFrame to accomplish what the user asks the tool to do. At this stage, it will go back to the web browser. The WebBrowserCore function will send HTML string to WebBrowserView in order for it to display the screen. After that, the reading function is performed when HTMLTranslator translates the source code that it has obtained from WebBrowserView. It understands which part of the text is to be read and which part is the link determined from the associated tags. Within HTMLTranslater, there is a Function List. It keeps the text to be read in the List. When the user asks the tool to read, the Function List will send a text string to ViaVoice to read it to the user. Links are also read and the tool will ask the user if they want to go to the next link. A sample of the CUVoiceBrowser screen is shown in Figure 3. Figure 3 Sample of screenshot of CUVoiceBrowser Recognizer As stated briefly earlier, we have used the Hidden Markov Model (HMM) technique and the Hidden Markov Toolkit (HTK) to build the recognizer. Seventy five Thai phonemes, including syllable initial phonemes, vowel phonemes, and coda phonemes, were used as the basic sound units for the vocabulary in the recognizer s dictionary. Each acoustic model specifically corresponded to a context-dependent triphone, which corresponded to one of the seventy five Thai phonemes with specific preceding and following phonemes. We collected the needed utterances from a total of 30 native Thai speakers, equally distributed between male and female, to train our acoustic models corresponding to the 75 phonemes. Each speaker was asked to say all of the commands used in the system as well as every Thai and English letter and number. Their voices were recorded using a computer headset and digitized at 16kHz with 16 bit resolution. The topology of each HMM is a five-state left-to-right model with three emitting states. Thirty nine-dimensional feature vectors, consisting of Mel-frequency cepstral coefficients (MFCCs) together with their deltas and accelerations, were used to represent observations from speech frames. Gaussian mixtures with two components were used to govern the emitting probabilities of the emitting states. A Baum-Welch re-estimation algorithm with flat-start strategy was used to estimate the required parameters for every HMM and Gaussian mixture. The Token Passing algorithm was used in the decoding phase to find the most likely hypothesis from the HMM-based triphone network generated by associated task grammar, which could be either navigation command or character filling.

5 4. Discussion and Conclusion In this paper, we have presented the design and implementation of a prototype for a voice-controlled tool using the Thai language that targets Thai who are motorhandicapped and/or blind novice Thai web browsers. The tool is called the CUVoiceBrowser. This prototype is the first example web accessibility technology being implemented in Thailand. In testing the effectiveness of the prototype, we found the accuracy to be satisfactory. The greatest accuracy is found when users use web navigation commands where the accuracy is in the high 80%. With Thai character input, accuracy is approximately 70%. The weakness is when users input English characters where the accuracy is lower than 60%. This might be attributed to the deficiency in English speaking of Thai native speakers who are in our test group. Our future research will concentrate on the systematic processing of non structured commands and combinations of commands. We envision implementing a system with feedback from the web interface so as to guide the users more effectively and also help the user to resolve conflicting accessibility paths. The feedback will also increase the speed with which the handicapped user will use the web pages. Furthermore, we have worked to improve on the portability of the tool and the cost of using it so that all visually-impaired and motor-handicapped Thais will have access to the tool. We also wish to employ information retrieval techniques to summarize the Thai webpage for the users. Acknowledgements The authors thank IBM (Thailand) who supplied the ViaVoice library module to help this project accomplish text-to-speech capability. References [1] Frost, R., Call for a Public Domain SpeechWeb, Communications of the ACM, Nov 05, Vol.48, No.11, pp [2] Asakawa, C., What s the Web Like If You can t See It? W4A at WWW2005, 10th May 2005, Chiba, Japan. [3] Hemphill, C.T. and Thrift, P.R., Surfing the Web by Voice. ACM Multimedia 95-Electronic Proceedings, November 5-9, 1995, San Francisco, CA, USA. [4] Parente, P., Audio Enrished Links: Web Page Previews for Blind Users. ASSET 04, Oct 18-20, 2004, Atlanta, Georgia, [5] Brondsted, T. and Aaskoven, E., Voice-Controlled Internet Browsing for Motor-handicapped Users, Design and Implementation Issues, Interspeech [6] Lopez R. A., Krischning A. I., Finder and Reader of Web Pages in Spanish for People with Visual Disadvantages, Proceedings of the 16th IEEE Conference on Electronics, Communications and Computers, 2006.