MusicXML to Braille Music Translation

Size: px

Start display at page:

Download "MusicXML to Braille Music Translation"

Stephen Randall
6 years ago
Views:

MusicXML to Braille Music Translation Aphisada Inthasara, Ladawan Mipansaen, Pichaya Tandayya, Chatchai Jantaraprim and Patimakorn Jantaraprim Department of Computer Engineering, Prince of Songkla

1 MusicXML to Braille Music Translation Aphisada Inthasara, Ladawan Mipansaen, Pichaya Tandayya, Chatchai Jantaraprim and Patimakorn Jantaraprim Department of Computer Engineering, Prince of Songkla University, THAILAND ABSTRACT This paper presents the development of the Braille Music and MusicML Translation. The input can be from a keyboard instrument, a Braille music file or a MusicML file. It will also play music from the input file. The program runs on Microsoft Windows and provides a graphic user interface which works with the screen reader JAWS. This work will help the visually impaired and the sighted people in learning and teaching music.. 1. INTRODUCTION The visually impaired people communicate with one another by perceiving sound and touching. Music is a relaxing way of communication, entertainment and relaxation. However, it is easy for the visually impaired people to learn and play music. Music notation [1] is rather complex and are mostly collected in forms of notation for sighted people. MusicXML is an extensible markup language that allows collecting and playing music on the web [2]. This work focuses on translation from the MusicXML document to Braille Math. The work also concerns getting input from a electronic keyboard and playing notes following the MIDI format. 2.5 XML Parser XML Parser is a parser simply written in C++. It is small, easy to use and flexible [14]. It supports several encodings including the Thai font TIS-620. The parsing process starts from that the XML parser loads a full xml file into memory. Then, it parses the file and generates a tree structure representing the xml file. After that, the data can be collected by traversing nodes in the tree. There are 3 parts of functions for extract data from the XML structure as below. 1. Start Tag is for extracting data from opening tags such as Element name, Attribute, Attribute Value. 2. Text is for extracting data in between the opening tag and end tag. 3. End Tag is for extracting data from end tags such as Element name. 3. SYSTEM DESIGN Figure 1 shows the overview diagram of the software. There are two modes to get input 2. BACKGROUND 2.1 MusicXML Formats MusicXML or Music Extensible Markup Language [2] provides formats for music notation following the W3C. The important parts of the music notation are step, octave and type. This three parts are also used in the Music Instrument Digital Interface (MIDI) standard [3-11]. 2.2 NFBTRANS Program The NFBTRANS program [12] is an open source written in C that is developed by the National Federation of the Blind (NFB) in the United States. It can run on both Linux and Microsoft Windows operating systems. It can translate English to Braille and Braille to English in both grades (1 and 2). In addition, it supports Braille text printing on embossers. The program is used in this work for English and Braille translation capability. In this work, the program is used for translating characters (grade 1). 2.3 Standard MIDI File Format The Music Instrument Digital Interface (MIDI) [3-11] is used as a format for transmitting and exchanging sound signals. It can collect music notes for songs. 2.4 Braille Computer Code Braille Computer Code is a Braille system for being used on computers. Thai Braille has rooted from English Braille [13]. Figure 1 Overview diagram of the software In Mode 1, the program receives the musicxml input file to be translated into a Braille Computer Code (BCC) file using the NFBTRANS program and also creating a MIDI file (.midi) for playing the song. In Mode 2, the program receives the input file from a keyboard instrument and then translates it into Braille Computer Code (BCC) using the NFBTRANS program. It also creates MIDI files (.midi extension) and plays the song [15-17]. There are main functions in the program. 1. MusicXML to Braille music translation 2. Music playing

part Inform the order of notes (number) Inform the key used Inform the notes used 1. get the data parsed from the MusicXML input file that can be translated into Braille Computer Code (BCC) 2.

2 Table 1. Table Structure of Music XML Element name Score-partwise Identification Part-list Part Measure Attribute Note Root element Function Identify the detail of the song Inform id and part name Musical part Inform the order of notes (number) Inform the key used Inform the notes used 1. get the data parsed from the MusicXML input file that can be translated into Braille Computer Code (BCC) 2. categorize the data for the format of a MIDI file 3. translate the data into a MIDI file for playing The Music XML format is shown in Figure 4 and described in Table 1. Figure 2 Sequence Diagram for the translation Figure 4 MusicXML format Figure 3 Sequence diagram for playing music The MusicXML to Braille music translation is shown in Figure 2 and 3. The input MusicXML file will be parsed through Functions XmlNodeRef(), parsetosetmember() and XmlParser [14]. The information between tags will be checked if it is a note. Next, the information will be translated into a Braille computer code using the NFBTrans version In case of playing music from the input file, there are three tasks to do this work. Figure 5 Music XML notation Figure 5 compares the Music XML notation to normal music note pattern as follows. The first part (1) is to announce the id and part name, Music in this case. The second part (2) notes an octave key used. In this case, a Sol key which starts from the second line of the 5 musical lines is used.

3 The third part (3) informs the time signature used. In this case, it is 4-4. The forth part (4) informs the note used. In this case, it is Do in the forth octave. Figure 6 shows the use case diagram of the program. The user interface has been designed so that the user can edit, translate and display documents. The input can be Braille music or MIDI. Figure 7 shows the graphic user interface of the program. The red points 1-8 in Figure 8 are the interfaces that can be explained as follows. 1) Select the instrument according to the MIDI standard. 2) Virual keyboard 3) Keypress data 4) Data sent from the keyboard informing alphabetical notes, octaves and type. 5) Translated XML data 6) Braille music text 7) Buttons for translating from XML to Braille 8) Sound control panel Figure 7 User Interface Figure 6 Use case diagram of the program The translation steps are shown in Figure 8 (next page). In case of translating an Music XML file, the XML Parser is used for parsing tags and musical data before being translated into a Braille music file. The program gets a Music XML file or MIDI input from a MIDI keyboard controller. The data from a keyboard will be checked in terms of type of note, which octave, and duration of key pressed. This information will be written in text files, i.e., Alphabetical, Octave and Type (Figure 9) in order to help organizing the data before being translated into a Music XML file. Figure 9 Files involved in translation There are 2 types of input collection as follows. Input from a USB MIDI Keyboard Controller After connecting a MIDI keyboard controller to a computer, the program will check whether the keyboard driver is correct. If not, the user can select a correct one from the list provided. This is shown in Figure 10. The steps for checking notes are shown in Figure 11. After a key is pressed, the data will be sent from the MIDI keyboard controller to the connected computer. Then, the program checks the step and octave, and looks up the note table. Input from a Music XML file After loading a Music XML file, the user can check and edit the file before translation. However, the user must know the exact format of the Music XML in order to correctly edit the file. Otherwise, the translation program may not translate it correctly.

4 Figure 8 Translation steps

5 Figure 10 Input collection from a USB MIDI keyboard controller 4. CONCLUSION This work presents an application program that can translate MusicXML documents into Braille Computer Code (BCC) documents and play music from the input file. In order to do so, an XML parser is used for parsing the information from the tags. After being categorized, the parameters are used for translating into a BCC format file by NFBTrans or playing the music following the MIDI format. It can also get input from a MIDI keyboard controller for translating to BCC. The program provides a Graphic User Interface that works with the screen reader Jaws to ease both the sighted and the visually impaired. The program connects to the MIDI keyboard controller hiding the configuration work. The delay in during play time is rather low. In summary, it can collect input from the MIDI keyboard controller to build a Music XML file as well as a MIDI file. It can also play MIDI files.. 5. ACKNOWLEDGEMENTS This work has been funded by National Electronics and Computer Technology Center (NECTEC). The host university is Prince of Songkla University, partly joined with Mahidol University and the Thailand Association of the Blind. 6. REFERENCES [1] MUSIC LESSON II Theory Reference, [2] MusicXML Definition, [3] The MIDI File Format Spec, [4] The MIDI-Compatible Part of MusicXML, [5] Wrapper Library for Windows MIDI API, [6] Note names, MIDI number and frequencies, [7] The International MIDI association, Standard MIDI-File Spec. 1.1, The International MIDI Association, Los Angeles, USA [8] MIDI, [9] Improv How to: Basic MIDI Output, [10] Virtual Midi Control Interface, [11] MIDI Software, [12] Nfbtrans version 7.70, Website by Nation Federation of the Blind (NFB), [13] Braille Music Code 1997 Table of Contents, [14] Source code XML Parser, [15] How to code a program that can play sound, [16] How to play and record sounds, [17] Sound in C++,

6 Figure 11 Steps for checking notes

Keywords Thai Braille Translation, Linux, User Interface, Front-end operation, XML, Braille Computer Code

Keywords Thai Braille Translation, Linux, User Interface, Front-end operation, XML, Braille Computer Code Linux User Interface and Front-end Operation for the Visually Impaired Thanathip Limna, Chomphunut Sae-tang, Chatchai Jantaraprim, Pichaya Tandayya and Wiraman Niyompol Department of Computer Engineering