A Review of Network Server Based Distributed Speech Recognition

Transcription

1 A Review of Network Based Distributed Speech Recognition YIMIN XIE, JOE CHICHARO, JIANGTAO XI & CHUN TUNG CHOU School of Electrical, Computer and Telecommunications ering University of Wollongong Northfields Avenue, Wollongong NSW 2522, AUSTRALIA Abstract: - Speech is the most natural means for people to communicate with each other. It is also regarded as the preferred method of future interaction between people and computer devices and applications in general. Distributed speech recognition technology facilitates the extension towards mobile devices by increasing their functionality and ease of use. This paper reviews the key aspects of distributed speech recognition technology and discusses some of the associated applications and technical challenges. A feasibility study is undertaken to establish the likely performance of a network server based speech recognition system. The objective is to provide speech recognition for users of mobile devices such as the Smartbadge [1]. Preliminary simulation results for the proposed system are presented which suggest that a single speech recognition engine can support a multi-user speech recognition system and still achieve a response time similar to a single user system. Key-Words: - Distributed speech recognition, Speech recognition front-end, based speech recognition. 1. Introduction Speech is the ultimate, ubiquitous interface. The ability to enter information or to control a process using one's voice offers real operational benefit to a wide range of users. Research on speech recognition has a long history, with the first work in this area dating back to the 1870s [2]. There have been substantial improvements in the capabilities of automatic speech recognition algorithms and products in the past few years, with more than 60 current vendors supplying a wide range of commercial speech recognition products world-wide [3]. In quiet office environments, the accuracy rate of most these recognition systems is reported to around 94% or higher with a vocabulary size of up to 20,000 words [4]. Speech recognition technology is being used in many applications, such as telephone call automation, network-based voice messaging, automatic transaction processing, timetable and directory enquiries and dictation [2, 5]. Telephone based Interactive Voice Response (IVR) system such as BT's Brimstone corporate telephone directory has been implemented and used in areas such as call control, voice mail and simple information retrieval [6, 7]. The use of voice-control interfaces to access information and services is becoming increasingly widespread and moving towards Internet-based services as speech recognition technology matures. Increasingly, people want the ability to access information while on the move. This demand will shift the use of speech for access to information and services from the wire line world to mobile wireless world. In this case a small portable device such as Personal Digital Assistant (PDA) or mobile phone is used to access data services through the natural user interface such as speech. Even though many on-chip speech recognition systems have been integrated into some command and control applications such as entertainment control, self-dialing cellular phones and even robot pet toys, the vocabulary of these systems is limited. Large vocabulary speech recognition systems require fast computation and large memory capacity to be able to work effectively and smoothly. These requirements are far beyond the capacity of most mobile devices and handheld computers. Distributed speech recognition is considered a desirable solution for providing a natural interface between users and mobile devices. Distributed speech recognition technology enables a new generation of interactive voice response applications operating over the mobile telephone and IP network. A growing number of e- businesses are targeting this vast market by deploying voice portal technologies that enable consumer access to web site content and transaction from the telephone. Voice access to web content and services provides a great opportunity to offer a wide

2 variety of services not only to users with computers but also to anyone with a telephone or a mobile phone or even a simpler mobile device such as a Smartbadge. 2. Distributed Speech Recognition A signal processing front-end is a common component of all speech recognition system [8], which extracts a set of acoustic features from the sampled speech. These feature vectors are then passed to a speech recognition engine where the utterances are recognized. The functional block diagram of a distributed speech recognition (DSR) system is illustrated in Figure 1. In this case the signal processing front-end is moved into the mobile device. This device is connected to a network server, or in case of multi-language system, to a bank of network servers, or a number of distributed servers through the data network. The speech recognition engines are located in the network servers. Hence users of mobile devices will feel as if the speech recognition system is on their mobile device. Signal Processing Front-end (Mobile Device) Data Network Back-End Recognizer (Network ) Back-End Recognizer (Network ) Back-End Recognizer (Network ) Fig. 1. Distributed speech recognition Moving the front-end processing of speech recognition system to the mobile device enables the speech features to be packeted into a stream of feature vectors, which can be transmitted to the network server where the actual speech recognition takes place. There are two common methods for computing acoustic features for signal processing front-end: a Filter-bank model and a Linear Predictive Coding (LPC) model. The choice of which method to use is based on the type of implementation, computational resources and memory available on the device. To support distributed speech recognition and other related applications and services, a standard for the distributed speech recognition front-end is being recommended and developed to ensure compatibility and interoperability. In particular, the Aurora DSR working group within the European Telecommunications Standards Institute (ETSI) has been actively developing this standard for the cellular phone. The recommended front-end system is illustrated by a simplified block diagram shown in Figure 2. Input Speech Feature Extraction Feature Compression Transmission Channel Bit Stream Formatting Fig. 2. Distributed speech recognition front-end The feature extraction algorithm used is a Mel- Cepstrum algorithm [9], which computes a set of feature vectors from the sampled speech. Each short-time analysis frame consists of 13 static cepstral coefficients and a log-energy coefficient. These feature vectors are then passed to a feature compression function in order to reduce the amount of data needed to represent the extracted feature vectors. The algorithm used in the feature compression is a split Vector Quantisation (VQ), which groups the static Mel Cepstral coefficients into pairs. Each pair of coefficients is then quantized using its own codebook (with a typical size of 64). The codebook indices are used as feature vector representations for each speech frame. The bit stream formatting function creates packets of compressed speech feature vectors together with error protection code in a fixed length message (144 bytes), which can be transmitted to the recognizer in the central server [9]. The advantages of this approach are that it can eliminate channel transmission errors, maintain the overall recognition performance and enable easy update of technologies and services provided. 3. Distributed Speech Recognition Across Internet The use of distributed speech recognition technology in Internet based services is a promising means of providing robust speech interface for people to interact with computer devices, especially in the mobile world and is one important step towards ubiquitous computing. It facilitates the implementation of distributed speech recognition

3 technology over an IP network and enables the use of existing network infrastructure and services. However, there are a number of advantages and disadvantages for deploying distributed speech recognition over an IP network. 3.1 Advantages 1. Sending the compressed speech feature vectors gives a useful increase in performance as well as a reduction of overall computational costs, when it is compared with directly sending speech in the form of codec parameters [10, 11, 12]. For example, coded speech using GSM encoding at 13kb/s gives a word error rate of 14.5% while VQ based compressed speech feature vectors at 2kb/s gives a word error rate of 6.63% [12]. This lower bit rate is of great benefit to handhold mobile devices such as wireless PDA. 2. Distributed network engine servers can share the computational resources between end users and enable the easy update of technologies and services provided. In other words, users can take advantage of the latest speech recognition engine and frequently updated vocabulary with out the massy upgrade of their own systems. 3. Increases the functionality and ease of use and hence creates greater end user opportunities and demand for new services. For example, providing directory service for location aware handhold mobile devices while users are on the move or driving a vehicle. 3.2 Disadvantages 1. The connectionless IP protocol is not particularly well suited for transmitting realtime data as there is not yet guaranteed Quality of Services (QoS) through the entire network. Therefore, as traffic in the network increases the transfer of data slows down and may make the service unusable. 2. Buffering large amounts of speech feature vectors for compression and transmission causes delays in the system. 3. The short battery life and limited processing power inherent in mobile devices results in a trade-off between complexity and reliability. 3.3 Challenges Distributed speech recognition has been tested with the packet-based Internet Protocol and Voice over IP (VoIP) [10]. The results indicated that the commonly used Transmission Control Protocol (TCP) and User Datagram Protocol (UDP) for distributed speech recognition faces number of challenges (such as packet loss, packet efficiency and delay). 1. The TCP protocol will deliver every packet but not in a specific time while UDP protocol allows smaller delays in packet delivery time but at the expense of probable loss in some of the speech feature vectors when network traffic is high. The degradation in speech recognition performance will be significant if the packet loss is high [10]. This calls for the design of more sophisticated packet loss recover technique if multiple speech frames are compressed and sent within a single packet. 2. The recommended standard [9] for distributed speech recognition packets multiframes of speech feature vectors into a fixed length message (144 Bytes), which represents 240ms of speech. It will introduce a large processing delay if this multi-frame of speech feature vectors is sent one at a time. Reducing the frame size can reduce the latency introduced by the frontend processing, but the packet efficiency will be lower. Therefore, there is a trade-off between front-end processing delay and packet efficiency. In addition, to facilitate the implementation of distributed speech recognition over an IP Network and offer low cost voice interactive response services to the end user, one of the issues need to be addressed is: What are the scalability constraints associated with distributed speech recognition? 4. Proposed System Speech recognition engines are system resource intensive and it is not acceptable (in terms of efficiency) to have a one to one ratio between users and speech recognition engines. This paper seeks to answer the question of the optimal number of users per speech recognition engine for a given system resource. Recent advances in speech recognition technology, particularly improvements in the accuracy of large vocabulary speech recognition and increases in available bandwidth to the desktop or wireless terminals enable us to consider a network server based speech recognition system. The proposed system aims to provide speech recognition services for multiple users of a mobile device such as Smartbadge on a local network.

4 The system as shown in Figure 3 consists of the following components: Front-End Terminal, Speech Agent and a band of Speech Recognition s. Mobile Device (Front-end Terminal) Mobile Device (Front-end Terminal)... Mobile Device (Front-end Terminal) Back-end Speech Recognition.... Speech Agent Back-end Speech Recognition Fig. 3. Network server-based DSR system Front-end Terminal: The front-end terminal can be a fixed terminal or a wireless mobile device or appliance running a small user agent program on behalf of an application. The user agent program is responsible for capturing and transmitting the parameterized feature vectors of the speech to the Speech Agent and passing back the decode text to the application. It only encapsulates and transmits the speech feature vectors when an utterance is detected via speech detection techniques. Speech Agent: The speech server agent consists of a multi threaded server agent program, which is supported by single or multiple back-end speech recognition engines. Also, the server agent is responsible for admission control and workload balancing if it is supported by multiple engines. agent running on a network server responds to the user agents and creates a new thread for each new user and accepts data from the user agent upon receipt of service requests. When the end of utterance signal is received, the server agent places the entire speech data into a task queue. The task queue handler will allocate the task to the first available speech recognition engine. The speech recognition engine responds to it by returning the decoded text. The server agent either sends back the decoded text to the user agent or performs further operations depending on the services provided. If a voice response is preferred, the Text To Speech (TTS) technique is employed to convert the decoded text to speech, and the speech is then sent to the user. Speech Recognition s: There are a number of different configurations for the back-end speech recognition engines. These engines can be organized physically to reside on multiple machines, i.e. a cluster system. They can also be built on one or more high performance machines which employ symmetric multiprocessor architecture. Remark: Although the proposed system has one speech server agent which organizes all the incoming requests, this system can be extended to include a number of geographically distributed agents and engines over the network to enhance the scalability and reliability of the entire system. The latter configuration can improve the response time if the customers are dispersed over a large area. 5. Preliminary Simulation Results From the user s point of view, response time is one of the most important performance characteristics. Response time is defined as the time between the end of spoken command and the system response (i.e. replies with decoded text). This experiment was conducted on a 550 MHz Pentium III machine with 128MB RAM and used the IBM ViaVoice speech recognition engine. The distributed speech recognition system was implemented as a TCP/IP client/server pair. The grammar file used was the Kiosk Example [13], which contains 2664 possible sentences or commands. The lengths of these sentences varied from 0.5 second to 6 seconds. Two hundred sentences were randomly selected and used to analyze the distribution of their length and processing time for a given system resource. The simulation analyzed the response time for one or more terminals connected to the system and examined the worst case (user issued commands one after another as soon as he or she received the response). In order to simulate the system, data was collected from a performance analysis of a user connected to a speech server agent with a single speech recognition engine. This data was then analyzed to estimate the performance of more than one terminal communicating to a network speech recognition engine through speech server agent. The simulation results are shown in Figures 4 to 6.

5 Fig. 4. Response time for a network server-based speech recognition system Fig. 5. Distribution of response time for a system which supports 10 users simultaneously. Fig. 6. Distribution of response time for a system which supports 15 users simultaneously. In order for a system to be practical, we assumed that the delay for front-end processing is less than 250ms and transfer delay is less than 200ms each way [9, 10, 14]. Also, the overall response time must be less than 2 seconds, as any longer delay may cause concern and lead to user frustration, especially in an application where speech is the only interface. For the given system resources (550 MHz Pentium III CPU with 128MB RAM) the processing time for each spoken command was found to be slightly less than one tenth of the length of the individual command. Our results show that ten users can be supported simultaneously within a 2 second limit 99% of the time (see figure 5) and 15 users can be supported 95% of the time (see figure 6). Although this result is system resource dependent, it indicates that more than ten users can be connected to a network server with a single speech recognition engine and still achieve a reasonable average response time similar to a single user system. If the number of concurrent users increases from 15 to 20 the average response time will increase significantly and the speech engine will be saturated. Note that the above analysis considers the worst case scenario, the speech server will be able to support more users if we take statistical multiplexing into account. 6. Conclusions and Future Work Distributed speech recognition technology enables a new generation of interactive voice response applications operating over the mobile telephone and IP network and makes use of the existing network infrastructure and services. Utilizing VQ based compressed speech feature vectors gives a useful increase in the performance as well as a reduction of overall computational costs. However, a more sophisticated packet loss recover technique is required if the UDP protocol is considered. Also there is a trade-off between front-end processing delay and packet efficiency. We proposed a network server-based architecture for delivering speech recognition services. This architecture can offer more cost/performance service to voice interactive applications for a small office environment and allows easy update of the new technologies. Also its capacity can be extended by adding more engines into a cluster server system to enhance the scalability and reliability of the speech network server. The ongoing experiment will examine the feasibility of providing multi-language speech recognition service over a wide area network and focus on the workload balancing on the multiple DSR engine server, the scalability of the distributed speech recognition network server and its applications. References: [1] P. Beadle et al. Location Aware Mobile Computing In ICT97, April 1997.

6 [2] J. A. Markowitz. Using speech Recognition, Prentice Hall, [3] R. K. Moore. Dictation and Voice Control: automatic speech recognition in the marketplace. In Speech and Language ering - State of the Art, page(s) 7/1-7/4, [4] P. Woodland. Speech Recognition In Speech and Language ering State of the Art, Volume 9. No.5, page(s) 2/1-2/5, Oct [5] F. A. Westall. Review of speech recognition for telecommunications. In Electronics and Communication ering, Volume 9 page(s) , Oct [6] D. J. Attwater and S. J. Whittaker. The Brimstone Corporate Directory Enquiries Application. In Advances in Interactive Voice Technologies for Telecommunication services, page(s) 5/1-5/8, [7] S. J. Whittaker and D. J. Attwater. The designed of complex telephony applications using large vocabulary speech technology In Spoken Language ICSLP 96, volume 2, page(s) , [8] L. Rabiner and B. H. Jaung. Fundamentals of Speech Recognition, Prentice Hall, [9] ETSI Draft ES V1.1.1 Distributed Speech Recognition; Front-end Feature Extraction Algorithm; Compression Algorithm, Feb [10] B. Milner et al. Robust Distributed Speech Recognition across IP Network. In Interactive Spoken Dialogue Systems for Telephony Applications, page(s) 6/1-6/6, [11] D. B. Roe and J. G. Wilpon. Whither speech recognition: the next 25 years: In IEEE communications Magazine, volume 31, Nov, [12] V. V. Digalakis, L. G. Neumeyer and M. Perakakis, Quantization of Cepstral Parameters for Speech Recognition over the World Wide Web, In IEEE Journal on Volume 17, Issue 1, Page(s): 82 90, [13] SMAPI Developer s Guide, IBM ViaVoice SDK for Windows Version 1.5, IBM, second edition, January URL: [14] S. Ramanathan and P. V. Rangan. Architectures for Personalized Multimedia In IEEE Multimedia, volume 1, page(s) 37-46, 1994.