Niusha, the first Persian speech-enabled IVR platform

2010 5th International Symposium on Telecommunications (IST'2010) Niusha, the first Persian speech-enabled IVR platform M.H. Bokaei, H. Sameti, H. Eghbal-zadeh, B. BabaAli, KH. Hosseinzadeh, M. Bahrani, H. Veisi, A. Sanian Speech Processing Lab, Sharif University of Technology, Tehran, Iran {Bokaei, Babaali, Bahrani, Veisi}@ce.sharif.edu, Sameti@sharif.edu ASR-Gooyesh Pardaz Company, Tehran, Iran {h.eghbalzadeh, kh.hosseinzadeh, a.sanian}@asr-gooyesh.com Abstract This paper introduces Niusha, the first Persian speech-enabled IVR platform. This platform uses Persian recognizer and Persian text-to-speech synthesizer engines in order to interact with users. The platform is designed in a way that it can simply be customized in various domains and its components are adjustable with new words. Keywords-component; speech-enabled IVR systems; VoiceXML; dialogue system; I. Introduction Since the invention of computer, human-computer interaction has been one of the most interesting areas from both academic and industrial viewpoints. The ease of this communication is a basic need for a user of any computer systems. According to phenomenon of data explosion, one of the most commonly used computer systems are information systems such as information kiosks where a user refers to it to gain information in a specific domain. The simplest way to communicate with an information system is to use natural language. For this purpose, spoken dialogue systems are developed that communicate with a user in an interactive environment in order to provide suitable information for the user. A typical dialogue system consists of five distinct modules: automatic speech recognizer, spoken language understanding module, dialogue manager, text generator and text to speech synthesizer. These modules are not perfect and have some errors in generating their outputs. Because of these errors, a commercial dialogue system is not developed yet and academic studies are conducted to improve the accuracy of each module separately. To palliate the need of dialogue systems, Interactive Voice Response (IVR) systems are emerged instead which consists of the same five modules, but each module is implemented in a more limited level and thus the accuracy is improved. Traditionally, touch tone IVR systems are used where a menu is read for the user and he/she uses the buttons on the phone keypad to interact with the system according to the read menu. With improvement of speech recognition module specifically in limited domains, a distinct kind of IVR systems are emerged. These systems are speech enabled IVR systems where the user can say his/her choice and the system recognizes the speech and acts accordingly. With the development of this kind of systems, IVR systems are getting closer to the ultimate dialogue system. In this paper we aim to introduce Niusha, the first Persian speech enabled IVR platform. The main module of an IVR system is its interaction process manager. With the use of VoiceXML (VXML) standard for implementing this unit, the whole system can be adapted in different domains easily. The rest of this paper is organized as follows. In Section II speechenabled IVR systems and the VoiceXML standard are introduced. In Section III Niusha is introduced and the distinct parts of this system are investigated. In section IV the main features of Niusha are introduced and finally in Section V the discussion is concluded and the future works are introduced. II. Concepts In this section we briefly introduce two most important concepts: Interactive voice response systems and VoiceXML standard. A. Interactive Voice Response systems Interactive voice response (IVR) is an automated telephony system that interacts with callers, gathers information and provides the requested information to the caller. An IVR system accepts a combination of voice input and touch-tone keypad selection and provides appropriate responses in the form of voice, fax, callback, e-mail and perhaps other media. An IVR system interacts with its user according to a pre-defined scenario designed in tree structure. User is moved to different states according to his/her answer to the questions being asked by the system. The first generation of IVR systems is the touch-tone IVR systems that read a menu and the caller selects an appropriate choice by pressing a number on the phone keypad. Apparently, this kind of IVR system is incapable to deal with some scenarios. An important limitation of touch-tone IVR systems is that the number of choices must be less than 9. Listening to a menu with several choices exhausts the caller. Usually, a menu with 3 or 4 choices is acceptable. According to this limitation and along with performance improvements in 978-1-4244-8185-9/10/$26.00 2010 IEEE 591

speech recognition modules, specifically in limited domains, the second generations of IVR systems are emerged that use an automatic speech recognition module to recognize user utterances. With incorporating the speech recognizer engine in the system the caller can say his/her purpose as well as pressing the appropriate key in order to select a choice. These IVR systems are called the speech-enabled IVR systems. A speech-enabled IVR system breaks the touch-tone version limitation and the user can freely choose a choice with the use of natural language and speech. B. VoiceXML standard VoiceXML (VXML) is the W3C's standard XML format for specifying interactive voice dialogues between a human and a computer. It allows voice applications to be developed and deployed in an analogous way to HTML for visual applications. Just as HTML documents are interpreted by a visual web browser, VoiceXML documents are interpreted by a voice browser. This standard is designed for creating audio dialogues that feature synthesized speech, digitized audio, recognition of spoken and DTMF key input, recording of spoken input, telephony, and mixed initiative conversations. In fact, VoiceXML is a description language which describes the procedure of a voice application such as speech enabled IVR system. Below is a short example of a VoiceXML application which simply uses text to speech synthesizer (TTS) module to produce the specified utterance Hello World for the user: <?xml version="1.0"?> <vxml version="2.1" xmlns="http://www.w3.org/2001/vxml"> <form> <block>hello World!</block> </form> </vxml> So far, the VoiceXML 2.1 is finalized and adopted as a W3C recommendation. VoiceXML 3.0 will be the next major release of VoiceXML, with new major features. It includes a new XML state chart description language called State Chart XML (SCXML). III. Niusha In this section we introduce Niusha, The first Persian speech-enabled IVR platform. Also system architecture is described and each different module is studied. As it was explained earlier, an IVR system is a dialogue system the main purpose of which is to interact with users in order to gain information about the user s goal and to act accordingly. For example in a Bank system, different operations are often defined such as cash deposit or withdrawal, payments etc. The overall schema of an IVR system is like a dialogue system and is illustrated in Figure 1. Generally, an IVR system is a small-scale model of a dialogue system where each component is implemented in a lower complexity level. Since an IVR system is used in a limited domain, such as the banking domain, this limitation affects the implementation level of each component. For example a speech recognizer which is used in a limited domain, like the banking domain, is expected to detect only a few words in each state. This limitation simplifies the training process of speech recognition module and improves the accuracy simultaneously. Figure 1. overall schema of an IVR system Generally speaking, Niusha is a platform which also creates tools for a designer to design a scenario of an IVR system. Then this scenario is used later within Niusha platform and constructs a complete IVR system where a user can connect and interact with the system. Two main features of our proposed system are listed below: 1. It supports Persian language: Niusha is the first speechenabled IVR platform which supports Persian language. It uses Persian speech recognizer engine (Nevisa engine) and also Persian text-to-speech synthesizer engine (Ariana engine) to interact with the user. 2. Niusha is a platform for creating and running a complete IVR system in different domains: As it was mentioned earlier, Niusha is a platform rather than a system in a way that it can be used in various domains. A scenario must be presented to this platform in order to construct a complete IVR system. The system designer designs a scenario according to the VoiceXML standard and adds it to the core of Niusha to construct an IVR system. Implementing the system in a new distinct domain is as simple as designing a new scenario for that domain. In addition both engines (Nevisa and Ariana) are statistical and can be adapted with new words for use in a new domain. In this paper a System designer is someone who designs a scenario and a System user is someone who makes a connection to the designed IVR system and interacts with the system. In the next subsection various components of Niusha are introduced according to overall schema illustrated in Figure 1. A. Niusha components As illustrated in Figure 1, typical IVR system architecture is composed of five distinct modules. So far, we have a simple spoken language understanding module. This simple module is to detect certain pre-defined words. It has a priority list of 592

words and selects one simple prior word among them. A more complicated statistical understanding module has also been designed by the authors [1] and is supposed to be added to the system in the future. The other four modules and the way we implement them in Niusha are explained in the following section. 1. Speech recognition module Niusha uses Nevisa engine as its speech recognizer module. Nevisa is the first and only large vocabulary speech recognizer system for the Persian language. This continuous speech recognition system uses the state-of-the-art speech and language modeling techniques and performs adequately as the first product for automatic dictation and telephony environment recognition applications in Persian. MFCC representation with some modifications is used as the set of speech signal features besides a VAD based on signal energy and zero-crossing rate. Nevisa is equipped with out-ofvocabulary capability for applications with medium or small vocabulary sizes. Powerful robustness techniques are also utilized in the system. Model-based approaches like PMC, MLLR, and MAP, feature robustness methods such as CMS, PCA, RCC, and VTLN, and speech enhancement methods like spectral subtraction and Wiener filtering, along with their modified versions, were diligently implemented and evaluated in the system. More about this engine can be found in [2]. 2. Dialogue manager Dialogue manager is the heart of a dialogue or an IVR system. Its main goal is to direct interaction in a way that important information is gathered from the user and to decide on the best response to the user based on this information. This unit acts according to a pre-defined scenario. The scenario can be assumed as a finite state machine whose states correspond to the IVR process states and whose transitions correspond to the conditions for changing the states. A part of the whole bank application is illustrated in Figure 2. Welcome prompt is played for the user in the welcome state. An unconditional transition is made to the next state where an input is given from the user and the next state is chosen according to this input. As mentioned earlier, VoiceXML standard is so far the best computer knowledgeable language for describing an IVR scenario [3]. In order to use this standard, the system has to have an interpreter capable to read VoiceXML standard tags and act accordingly. For this purpose we use Bladeware- VXML, an open source VoiceXML standard interpreter. This interpreter supports VoiceXML 2.1, the latest stable version of VoiceXML standard, but has its own defects. We fix these problems by adding several new tags, customize it in order to support the Persian language and prepare it in order to be used as the core of the dialogue manager unit. 3. Natural language generator The goal of NLG unit is to generate the proper responses for the user. Usually this component is implemented in a template based manner. Some pre-defined templates are provided. Each template needs some parameters to become a complete sentence. For example assume a greeting template like Welcome <person name> to the system This template becomes a sentence when the person name is added to it. In Niusha this component is implemented according to VoiceXML standard. For example we have: <prompt> Welcome <value expr= personname /> to the system</prompt> The person name is stored in runtime variable named personname and used for completing the sentence. The completed sentence is sent for text-to-speech synthesizer module in order to be played for the user. Figure 2. Part of "bank" application scenario 4. Text-to-speech synthesizer For the TTS component, the recently developed Ariana TTS engine is used. It is a Persian TTS engine which contains two parts. The first part named TTP, obtains phonetic information from the pure text. Combination of a 90k lexicon and a stemmer is applied for collecting various phonetic candidates of each word and an HMM-based algorithm estimates the best candidate. The second part converts the extracted phonetic information to the synthesized speech waveform. This part uses cluster unit selection method for synthesizing speech waveform. This method is implemented in Festival speech synthesizer system [3]. B. Niusha architecture Through architectural perspective Niusha is composed of four distinct components, Niusha gateway, Niusha designer, Niusha simulator, Niusha TAPI wrapper. These components and their responsibilities are described later. The connection between these components is done with socket programming. Therefore each component can be placed in a separate computer and keep on functioning without any interruption using the definite protocol. Each component is described next. 593

1. Niusha gateway Niusha gateway is the main engine of our whole system. All Niusha units and their connectivity are shown in Figure 3 and described below. Niusha browser/simulator manager: This unit manages all the system settings such as component s individual IP and ports, files path setting and etc Niusha call handler: This unit is responsible for the following tasks: o To check all line statuses and their connectivity. o To provide call waiting service in case the lines are busy. o To organize the calls in their design order. o o To switch calls over the lines. To assign different ports to different calls and different conversation in multi-lines. 2. Niusha designer This part is a tool to enable the user to design a scenario with a graphical interface. Complicated scenarios can be designed with just drag and drops of pre-defined modules. This graph is then converted to the tags of VoiceXML standard and is prepared to be used within the Niusha platform. Each component is mapped to a series of VoiceXML tags. For example a menu can be designed with menu tag where a prompt is played for the user and a choice is selected according to user decision. For putting such a menu in a scenario, the designer has to only drag a menu component and drop it in the grid. The corresponding tags are then produced as illustrated in Figure 4. Figure 4. a menu component and its corresponding VoiceXML tags Figure 3. Niusha gateway architecture Niusha resource manager: It manages all the resources (such as fax, SMS, TTS, etc ) that should be authorized. Each resource may have certain limitations which have to be controlled in this unit. Niusha VoiceXML interpreter: This unit interprets all the tags in the designed scenario and acts accordingly. Resources: To perform specified action, Niusha is presented with some modules. These modules are considered as resources which have to be controlled by the recourse manager. Some of these resources are: fax, SMS, email, database, Speech recognition engine, text-tospeech engine, etc 3. Niusha simulator Before putting a scenario on line, it must be checked if it works properly. This can be done with Niusha simulator. This tool is a computer application which gets a VoiceXML application and interacts with the user through microphone and speaker. 4. Niusha TAPI wrapper This wrapper is designed for telephony boards. Specifically Niusha is customized to use Dialogic Dive media board for telephone interface, but it can easily be adapted to other media boards. IV. Niusha capabilities Niusha is designed in a way to supports the main capabilities of available IVR systems. The main capabilities are listed in this section. These specifications are accessible in 594

VoiceXML level. It means that their properties can be set at the scenario design step. 1. Connecting to any available database and execute any valid query: we define a new tag which handle execution of database queries. This tag contains two properties. One for database name and the other for the query. An engine like SQL server is used to perform the specified query. 2. Speech recognition module configuration: To have a more accurate speech recognition engine, we have to limit the number of considered words. This can be done separately for each state since the words used in each state are different. Three properties are set for speech recognition engine to use: a. Language model b. Lexicon c. Phonetic of each word in lexicon With specifying these values in each node separately, the speech recognition engine is customized for that node specifically. 3. Utilizing barge in (with sound and DTMF): Assume a long prompt which has to be played for users. Barge in gives this capability for the user to select a choice before the playing prompt is finished. This can be done with both DTMF and voice. In the other words, during playing of prompt, user can say his/her choice or press the corresponding phone key. Then the prompt is skipped and the choice is selected. This property can be set for each state separately. 4. Final silence detection: an energy-based algorithm is implemented which differentiates between silence and speech signals. This module is used to detect silence in utterances. We assume that the user utterance is finished when specified epoch is detected as silence and then the record operation is terminated. This feature can be set for each state separately and its value indicates the silence time interval that the system has to wait in order to terminate the recording process. 5. Taking multi-digit numbers from the user: In some of the states the system has to get a multi digit number from the user. The desired number specification, such as maximum and minimum length, can be set in VoiceXML for the corresponding state. 6. Defining skip character: a character can be set to be considered as the skip character. The skip character function is to skip playing the prompt, but no choice is made. The prompt is skipped and system waits for an input from the user. 7. Time and date indication: to indicate current time & date or any other ideal time or date, a tag is defined which gets the runtime date and time and play it with Ariana TTS engine. limitations of traditional touch-tone IVR systems the need to this kind of speech enabled systems is growing. The system platform was explained from two distinct viewpoints. First we introduce an overall schema of a typical IVR or dialogue system and the implementation of these modules in our Niusha platform are explained. From architectural viewpoint, we illustrated that our platform consists of 4 components. These components are introduced and their responsibilities are described. At last, we introduced the main features that our platform can support. With these features, a designer can create complicated scenarios and use the dialogue engine in any specific domain. For future we want to embed our designed statistical understanding module in Niusha and go one step further to an ultimate dialog system. REFERENCES [1] Bokaei M. H., Sameti H., Bahrani M., Babaali B., "Segmental HMM-based part-of-speech tagger," International conference on audio language and image processing ICALIP2010, Shanghai, China, 2010. [2] Sameti H., Veisi H., Bahrani M., Babaali B. and Hosseinzadeh K., Nevisa, a Persian Continuous Speech Recognition System. In Communications in Computer and Information Science, Advances in Computer Science and Engineering, 13th International CSI Computer Conference, CSICC 2008 Kish Island, Iran, Vol. 6, pp. 485-492, Springer Berlin Heidelberg, 2008. [3] Bruce L., VoiceXML for Web-based distributed conversational applications, Communications of the ACM, v.43 n.9, p.53-57, Sept. 2000 [4] Black A., Taylor P., and Caley R. The Festival Speech Synthesis System. University of Edinburgh, Centre for Speech Technology Research, edition 1.4, for festival version 1.4.0, 1999. V. Conclusion In this paper we introduced the first Persian speechenabled IVR system, called Niusha. According to the 595