Keywords Thai Braille Translation, Linux, User Interface, Front-end operation, XML, Braille Computer Code

Size: px

Start display at page:

Download "Keywords Thai Braille Translation, Linux, User Interface, Front-end operation, XML, Braille Computer Code"

Lisa Harrington
5 years ago
Views:

Linux User Interface and Front-end Operation for the Visually Impaired Thanathip Limna, Chomphunut Sae-tang, Chatchai Jantaraprim, Pichaya Tandayya and Wiraman Niyompol Department of Computer

1 Linux User Interface and Front-end Operation for the Visually Impaired Thanathip Limna, Chomphunut Sae-tang, Chatchai Jantaraprim, Pichaya Tandayya and Wiraman Niyompol Department of Computer Engineering Prince of Songkla University and Rachasuda College, Mahidol University, THAILAND ABSTRACT This paper presents a work that supports the visually impaired to be able to use the PSU Braille computer more easily. As the PSU Braille computer uses Linux as an operating system which can be unfamiliar to many users, this work focuses on the text-based user interface and front-end operation especially designed for the visually impaired and to support Braille devices including Braille printers, keyboards and displays. Along with the hardware development of the PSU Braille computer, the group has also developed translation software with Thai and English processing capability. The work in this paper also includes the integration of all translation programs, i.e. English, Thai, Math, Chemistry to Braille and vice versa, using the Extensive Markup Language (XML) and Braille Computer Code. In addition, the XML technology has also been applied extensively for classifying and collecting documents in the system. Keywords Thai Braille Translation, Linux, User Interface, Front-end operation, XML, Braille Computer Code 1. INTRODUCTION There are a few Thai Braille translation programs. However, they run on the Microsoft operating software and the types of documents they support are limited. For example, some supports only Braille grade 1 or only Thai or English. They usually don t cover all Braille, grade 1 and 2, or rather support only one language while nowadays documents combine more than one language. Not many of the translation programs support nongeneral documents such as text that includes mathematic and scientific equations or music notation. As they were developed on the Microsoft Windows by sighted people, they usually have a Graphics User Interface that relies on a screen reader to inform the visually impaired people how to navigate in the program. The PSU Braille computer project is to develop a prototype for low-cost Braille computer notebooks with Thai and English processing capability. The work deals with both hardware and software for the visually impaired (Figure 1). In terms of software, we focus on using and creating open source programs. This work is developed for facilitating the visually impaired and sighted people to easily use integrated Braille translation documents that can be composed of Thai and English Braille text and mathematical and scientific Braille text to a web-based standard formats including Mathematical Markup Language (MathML) and Chemistry Markup Language (CML). This work also includes designed XML formats that support document integration, translation, correction and word segmentation. This work aims to support Braille input and output on the PSU Braille computer Figure 1. Braille Computer Project 2. Background 2.1 PSU Braille Computer The PSU Braille computer [7] supports Braille input and output or Braille keyboard and display as well as general input and output devices such as computer keyboard and monitor, and USB peripherals. Currently, it supports a Braille display with 32 Braille cells and a 4 rows x 20 characters Liquid Crystal Display (LCD), and 9-key Braille keyboard (8 keys and a space bar). It also supports Thai and English processing. 2.2 NFBTRANS Program The NFBTRANS [12] program is an open source written in C that is developed by the National Federation of the Blind (NFB) in the United States. It can run on both Linux and Microsoft Windows operating systems. It can translate English to Braille and Braille to English in both grades (1 and 2). In addition, it supports Braille text printing on embossers. The program is used in this work for English and Braille translation capability. 2.3 Extensive Markup Language (XML) Extensive Markup Language or XML [13] is a language for supporting and displaying web-based documents that can document formatted text. It has structures to specify various types of documents following the W3C formats. It also has rooms for extensible formats. 2.4 PSU Thai Braille Translator The PSU Thai Braille translator [10-11] which is written in C and runs on a Linux operating system has been developed by the

2 Braille and Assistive Technology Research Group at Prince of Songkla University in collaboration with Ratchasuda College. It supports Thai Braille to Thai ASCII translation and vice versa. It is derived from the NFBTRANS program to inherit the English processing capability. It also aims to support Thai Braille in both grades (1 and 2). This Thai translation is not as easy as it is in English or languages with Roman characters which can use a table method. Thai Braille has more complex grammars due to more consonants, vowels and intonation marks. 2.5 PSU Braille Math Translator The PSU Braille Math Translator [5-6] is developed for translating mathematical symbols, numbers, variables and equations in forms of MathML which is a standard web format for displaying mathematical documents for sighted people into Braille math text for visually impaired people and vice versa. Currently, it supports up to the high school level of mathematics. 2.6 PSU Scientific Braille Translator The PSU Scientific Braille Translator [3] is developed for translating Braille scientific equations and formula into standard web format documents and vice versa. In addition, it supports the CML that includes displaying molecule structures and chemical equations. under the GNU General Public License. Besides basic emulation, Nano also has the following other features: GNU autoconf support. Goto line and replace functions (without needing command-line flags). Interactive replace function and spell checker. Auto-indent support. Slang curses-wrapper support. Variable displayed tab width. Regular expression search and replace. Toggles for command-line flags from within the editor XML Parser The XML Parser [2] used in this work is developed by Dr. Ir. Frank Vanden Berghen. It is small, simple, cross-platform, free and fast. Parsing steps: The XML parser loads a full XML file into memory. Then, it parses the file and generates a tree structure representing the XML file. After that, the tree can easily be explored to get the data. The tree can also be modified using "add" and "delete" functions. A formatted XML string can then be generated from a subtree. Memory management is totally transparent through the use of smart pointers. 2.7 SWATH Program SWATH [6] (Smart Word Analysis for THai) is a word segmentation for Thai. SWATH offers 3 algorithms: Longest Matching, Maximal Matching and Part-of-Speech Bigram. The program supports various file input formats such as html, rtf, LaTeX as well as plain text. SWATH is used in this work for Thai word segmentation before translation. 2.8 Data Structure for Controlling Terminal: System V System V is a Unix system for controlling terminals. It has a data structure for freely controlling behaviors of the terminal, for example creating or editing time, signaling or display using some functions to assign variables of the data structure termios in file termios.h. The functions that control terminals are categorized into the following groups: Output modes Input modes Control modes Local modes Control codes for controlling the terminal The data structure for controlling the System V terminal is applied for controlling the terminal to be able to support keys pressed on the PSU Braille computer. In the system, we design to use only one character to activate the function and makes the character invisible on screen (for the sighted people). 2.9 GNU Nano GNU Nano [1] is a small and friendly text editor. It is used in this work because it is an open source so that we can add features like Braille translation. Nano is a clone of the Pico text editor covered Figure 2. Use Case Diagram of the System 3. DESIGN This work integrates all Braille translation engines in the project together. The visually impaired users can use the program to read, edit and translate documents (Figure 2). This program will also allow users to be able to easily change document translation configuration such as the grade of Braille text and the number of lines per page, and the number of characters per line. The program development can be divided into 3 parts: User Interface XML formatting, translation and export file Editor 3.1 User Interface As this work aims to support the application interface for the PSU Braille Computer which uses the Linux OS and text only (for the visually impaired people), the applications in this work therefore need to be based on Linux OS and text mode. The user can select activities through the menu as shown in the activity diagram in Figure 3.

$If it is a Thai text Function thaiwordcut is called for word segmentation before writing into an output file. For English, Function putxmlcharacter is for checking special characters, i.e., \n, \t, <, >, &.$

3 Press Enter > Press m > Press q > Press t > Press Space bar or c > < Press q < Press m Press Enter > Press c > Press Space bar or t > whether it is Thai or English. If it is a Thai text Function thaiwordcut is called for word segmentation before writing into an output file. For English, Function putxmlcharacter is for checking special characters, i.e., \n, \t, <, >, &. At the end, the end tag will be added in the XML output file. Figure 3. Activity Diagram for the System 3.2 Document Formatting and Braille Translation This part of the program deals with formatting documents from Braille text to the XML format. The Thai word segmentation is used in this case. After translating the Braille document, we also need to prepare a format for printing on a printer using word segmentation, both English and Thai. The Thai segmentation can be divided in 2 cases: grade 1 and grade 2. Figure 5. Class Diagram for Part of Translating Figure 4. Overview of the Braille Translation A Braille translation engine is called as a child process for more efficiency. The child process that runs a translation engine contacts a key process that separates texts with the XML parser and distribute them to related translation engine. The translated texts will then be assembled back through a Unix pipe. Finally, the translated document will be formatted for printing on an embosser or displayed on a Braille display. The design is shown in the class diagram (Figure 5) Class InsertTag Class InsertTag is used for translating a text file to an XML file. Figure 6 shows the sequence diagram of the class. The constructor of Class InsertTag gets the arguments which include the names of input and output files, and English and Thai Braille grades used. Function InsertTag will be called for adding an XML header into the output file. Then, Function texttoxmlprocess repeatedly reads the input file for checking the tag opening < (for MathML and CML data). If it is the tag opening, Function findmltag will be called for checking the type of the XML tag whether it is CML or MathML. Function findendmltag is for checking the end tag of that tag. Function checktypeoftext is for checking the text Figure 6. Sequence Diagram of Class InsertTag The documents that have been identified by XML tags can be also displayed on a web browser by applying Extensible Style Sheet Language (XSL) [14] as shown in Figure 9.

3.2.3 Class Export Class Export is for extracting Braille Computer Code from an XML file that contains Braille Computer Code text. The sequence diagram of the class is shown in Figure 11. Figure 7.

Thai word segmentation Figure 9. Display with XSL 3.2.

4 3.2.3 Class Export Class Export is for extracting Braille Computer Code from an XML file that contains Braille Computer Code text. The sequence diagram of the class is shown in Figure 11. Figure 7. Part of an XML file with grade-1 Thai word segmentation Figure 8. Part of an XML file with grade-2 Thai word segmentation Figure 9. Display with XSL Class ParseTag Class ParseTag helps in translating an XML file which contains text for sighted people to Braille Computer Code for visually impaired people or vice versa. The class s sequence diagram can be shown in Figure 10. Figure 10. Sequence diagram of Class ParseTag First, the constructor of Class ParseTag creates a pipe for process intercommunication and then calls Function ParseTag for adding the XML header to the output file. It also creates a child process for calling a related translation engine using Function xmlprocess and Function calltranslation. The translation engine gets extracted texts for translation from the XMLParser. Function convertcharacter helps in translating special charater, i.e., &, <, >, to an XML format. These steps are repeated until the end of the input file. Figure 11. Sequence diagram of Class Export First, the constructor of Class Export collects arguments that includes the names of the input and output files, the number of cells, the number of lines the page number requirement which will be put in the header, and the header name. It then calls Function openfile to open the input and output files. If a header is needed, Function insertheader will be called every time a new page is started. Then, Function XMLParser will be used for parsing text and passing it to Function parsecell for checking the number of cells before writing the text into the output file. Function newline will be called if there is a line feed. In case of the reverse translation, the steps will be the same excluding Function insertheader and Function checkcell Mode of Translation The functions of the program is divided into 5 modes for ease of use as shown in Table 1. Table 1. Mode of translation Mode Work 1 Translate a document for sighted people to an XML file with Braille text (brlxml) 2 Translate a Braille XML file to a document for sighted people 3 Extract a Braille file (BCC) from a Braille XML file or extract an XML document to a text file 4 Translate a text file into a Braille file. 5 Translate a Braille XML document into a text file. 3.3 Editor The editor in this work is GNU Nano because it is a text-based editor and it provides short cut keys for faster use. As an open source, the editor can be modified to call translation engines. In this case, we use the short cut key Ctrl+ t to enable translation (Figure 12).

5. CONCLUSION This work focuses on integrating all Braille translation programs, modifying the interface to suit the visually impaired, and also designing a common format that helps with documenting

5 5. CONCLUSION This work focuses on integrating all Braille translation programs, modifying the interface to suit the visually impaired, and also designing a common format that helps with documenting and translating. It as well supports the display on PSU Braille computer. Figure 12. GNU Nano with translation option 4. Input Formats and Translation Engines This section describes the formats of input documents and translation programs provided in this work. 4.1 Documents for Translation The input documents that can be translated with the translation engines must conform to the formats in Table 2. As MathML and CML documents are written in English, the program may confuse them with normal English documents. Therefore, the tags for MathML and CML in Table 2 will help the program to distinguish which translation engine the document is related to. If MathML or CML documents do not conform to the formats in Table 2, they will be treated as normal English documents. Figure 13 shows an example of a MathML file and Figure 14 shows the result of the translation in a Braille file. Data Thai English MathML CML Table 2. List of input documents accepted Accepted Format Thai characters English characters Data must be in between Tags <mathml> and </mathml> Data must be in between Tags <cml> </cml> Figure 13. Example of a MathML document 4.2 Translation Engines Provided There are 4 translation engines provided. The software can currently translate documents in Thai and English and documents with symbols and equations or structures in Mathematics and Chemistry as listed in Table 3. Figure 14. The Braille file translated from the MathML document in Figure 13 Table 3. List of Braille translation engines provided Data Provided Thai Thai to Thai Braille (Grade 1 and 2) Thai Braille to Thai (Grade 1) English English to English Braille (Grade 1 and 2) English Braille to English (Grade 1 and 2) (Nfbtrans version) Math MathML to Braille Math Braille Math to MathML Scientific CML to Braille Chemistry Braille Chemistry to CML 6. ACKNOWLEDGMENTS This work has been funded by National Electronics and Computer Technology Center (NECTEC). The host university is Prince of Songkla University, partly joint with Mahidol University and the Thailand Association of the Blind. 7. REFERENCES [1] GNU nano. Online 11 November [2] Ir. Frank Vanden Berghen. Small, simple, cross-platform, free and fast C++ XML Parser. Online 16 November [3] Kitiya Tiravikarn. Scientific Braille Translation (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, [4] Kongsakul Chumkong, Operation Software for the blind (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, [5] Nuthnaree Wonghirundecha. MathML to Braille Math Translation (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, 2005.

6 [6] Paisam Charoenpomsawat. Swath Manual. Online 11 November [7] Pichaya Tandayya, Suntorn Witosurapot, Worraprot Chukumnird, Chatchai Jantaraprim, and Wiraman Niyompol, Technical Report of the Development of a Braille Computer System for the Visually Impaired, submitted to NECTEC, [8] Pimorn Senakat. Braille Math to MathML Translation (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, [9] Pulse Data and VisuAide. BrailleNote mpower BT 32. online 25 June T.asp. [10] Somchai Petkleang. Thai and English to Braille Translation (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, [11] Tachit Panwasin. Braille to Thai and English Translation (in Thai). Student Project, Department of Computer Engineering, Prince of Songkla University, Songkhla, [12] The National Federation of the Blind. NFBTRANS. online 11 November [13] XML. Online. 15 August [14] Extensible Style Sheet Language (XSL). Online.

MusicXML to Braille Music Translation

MusicXML to Braille Music Translation Aphisada Inthasara, Ladawan Mipansaen, Pichaya Tandayya, Chatchai Jantaraprim and Patimakorn Jantaraprim Department of Computer Engineering, Prince of Songkla University,