irobotrock: A Speech Recognition Mobile Application Reema Pimpale Prabhat Narayan Anand Kamath

Size: px

Start display at page:

Download "irobotrock: A Speech Recognition Mobile Application Reema Pimpale Prabhat Narayan Anand Kamath"

Francine Miles
5 years ago
Views:

1 irobotrock: A Speech Recognition Mobile Reema Pimpale Prabhat Narayan Anand Kamath

2 Outline Introduction Technologies Current Approaches Our Solution Users ( Domain) Our Approach Pending Functionality Future Enhancements References

3 Introduction Remotely accessing machine to do tasks such as Play music Type Take notes Browse web Motivation To increase user friendliness by using Speech Recognition.

4 Technologies ios Operating system developed by Apple in use on their mobile devices, such as the iphone Python Cross platform programming language CMUSphinx A speech recognition toolkit which has a number of packages for different tasks and applications. PocketSphinx A package of CMUSphinx SphinxBase A package of CMUSphinx Acoustic Model is used by a speech recognition engine to recognize speech. LMTool Web based tool for generating the language model

5 Current Approaches Latest technologies iphone Siri Android Iris Features supported by the above technology: Send text messages Listen to music Call contacts Send View a map Visit websites

6 Our Solution Recognize the speech and connect to your computer. Open a website on your computer by giving a voice input on your phone. Type and send an from your computer using voice input from your phone. In short, mobile device acts as a remote control to connect and carry out tasks on your computer.

7 Users ( Domain) Extremely useful for handicapped people. Useful for people on the go.

8 Our Approach Researched the available speech recognition libraries Designed the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application

9 Researche d the available speech recognition libraries Designing the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application Client side - Uses the open source components of CMUSphinx to provide very complicated voicerecognition functionality. Server side Uses Python to dynamically load additional classes at run-time and allow enhanced functionality without requiring future users to touch any previously developed code

10 Researched the available speech recognition libraries Designed the high level architectur e design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application

11 Researched the available speech recognition libraries Designing the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application

12 Researched the available speech recognition libraries Designing the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application Uses the CMUSphinx toolkit and implement the architecture for the client. The main components are PocketSphinx: lightweight recognizer library written in C. Sphinxbase: support library required by Pocketsphinx. (Part of the PocketSphinx component). Dictionary: (Language Model and Acoustic model)

13 Researched the available speech recognition libraries Designing the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application Written in Python Basic TCP server that listens for messages from client on an assigned port Waits for keywords to activate specific functionality Ex. "MESSAGE" triggers composition component All subsequent messages received will be appended to a message string Waits for "BYE" keyword to end message and send

14 Researched the available speech recognition libraries Designing the high level architecture design Designed the Sequence diagram to understand the data flow Build the for the iphone (Client ) Build the server Test the application Live Demonstration

15 Pending Functionality Dynamically load additional components enabled on the server. New component will be written as a Python class with a defined set of callbacks. Each class defines trigger messages and defines how following messages should be handled to perform any required task

16 Future Enhancements Currently uses unencrypted TCP connection. Security can be enabled using TLS (Transport Layer Security) to encrypt messages. Plugin based system can be extended in any way that the user desires

17 RESULT Successfully implemented a mobile application using speech recognition which connects to the computer remotely and is able to send an via the voice input given through the phone.

18 References Carnegie Mellon University, CMU Sphinx Open Source Toolkit for Speech Recognition, March Huggins-Daines, D.; Kumar, M.; Chan, A.; Black, A.W.; Ravishankar, M.; Rudnicky, A.I.;, "Pocketsphinx: A Free, Real-Time Continuous Speech Recognition System for Hand- Held Devices," Acoustics, Speech and Signal Processing, ICASSP 2006 Proceedings IEEE International Conference on, vol.1, no., pp.i, May Gulic, M.; Lucanin, D.; Simic, A.;, "A digit and spelling speech recognition system for the Croatian language," MIPRO, 2011 Proceedings of the 34th International Convention, vol., no., pp , May K. F. Lee, H. W. Hon, and R. Reddy, "An overview of the SPHINX speech recognition system," IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 38, no.1 pp , Jan Cohen, J. Embedded speech recognition applications in mobile phones: status, trends and challenges. Proc. ICASSP 2008, IEEE Press (2008), pp Python Software Foundation, The Python Standard Library, November

19 Questions? Thank You

Open-Source Speech Recognition for Hand-held and Embedded Devices

PocketSphinx: Open-Source Speech Recognition for Hand-held and Embedded Devices David Huggins Daines (dhuggins@cs.cmu.edu) Mohit Kumar (mohitkum@cs.cmu.edu) Arthur Chan (archan@cs.cmu.edu) Alan W Black