Speech in Mobile and Pervasive Environments

Size: px

Start display at page:

Download "Speech in Mobile and Pervasive Environments"

Kory Berry
5 years ago
Views:

3 Speech in Mobile and Pervasive Environments

4 Wiley Series on Wireless Communications and Mobile Computing Series Editors: Dr Xuemin (Sherman) Shen, University of Waterloo, Canada Dr Yi Pan, Georgia State University, USA The Wiley Series on Wireless Communications and Mobile Computing is a series of comprehensive, practical and timely books on wireless communication and network systems. The series focuses on topics ranging from wireless communication and coding theory to wireless applications and pervasive computing. The books provide engineers and other technical professionals, researchers, educators, and advanced students in these fields with invaluable insight into the latest developments and cutting-edge research. Other titles in the series: Misic and Misic: Wireless Personal Area Networks: Performance, Interconnection, and Security with IEEE , January 2008, Takagi and Walke: Spectrum Requirement Planning in Wireless Communications: Model and Methodology for IMT-Advanced, April 2008, Pérez-Fontán and Espiñeira: Modeling the Wireless Propagation Channel: A simulation approach with MATLAB, August 2008, Ippolito: Satellite Communications Systems Engineering: Atmospheric Effects, Satellite Link Design and System Performance, August 2008, Lin and Sou: Charging for Mobile All-IP Telecommunications, September 2008, Myung and Goodman: Single Carrier FDMA: A New Air Interface for Long Term Evolution, October 2008, Wang, Kondi, Luthra and Ci: 4G Wireless Video Communications, April 2009, Cai, Shen and Mark: Multimedia Services in Wireless Internet: Modeling and Analysis, June 2009, Stojmenovic: Wireless Sensor and Actuator Networks: Algorithms and Protocols for Scalable Coordination and Data Communication, February 2010, Liu and Weiss, Wideband Beamforming: Concepts and Techniques, March 2010, Riccharia and Westbrook, Satellite Systems for Personal Applications: Concepts and Technology, July 2010, Qian, Muller and Chen: Security in Wireless Networks and Systems, March 2014,

5 Speech in Mobile and Pervasive Environments Nitendra Rajput and Amit A. Nanavati IBM Research, New Delhi, India A John Wiley & Sons, Ltd., Publication

6 This edition first published John Wiley & Sons Ltd. Registered office John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, United Kingdom For details of our global editorial offices, for customer services and for information about how to apply for permission to reuse the copyright material in this book please see our website at The right of the author to be identified as the author of this work has been asserted in accordance with the Copyright, Designs and Patents Act All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted by the UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Designations used by companies to distinguish their products are often claimed as trademarks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Library of Congress Cataloging-in-Publication Data Rajput, Nitendra. Speech in mobile and pervasive environments / Nitendra Rajput and Amit A. Nanavati. p. cm. Includes bibliographical references and index. ISBN (cloth) 1. Speech processing systems. 2. Cell phone systems. I. Nanavati, Amit A. II. Title. TK7882.S65R dc A catalogue record for this book is available from the British Library. ISBN: (H/B) Typeset in 10.5/13pt Times by Laserwords Private Limited, Chennai, India

7 For, Z ooz oo & Po o To, Family & Friends

9 Contents About the Series Editors List of Contributors Foreword Preface Acknowledgments xiii xv xvii xix xxiii 1 Introduction Application design Interaction modality Speech processing Evaluations 5 2 Mobile Speech Hardware: The Case for Custom Silicon Introduction Mobile hardware: Capabilities and limitations Looking inside a mobile device: Smartphone example Processing limitations Memory limitations Power limitations Silicon technology and mobile hardware Profiling existing software systems Speech recognition overview Profiling techniques summary Processing time breakdown Memory usage Power and energy breakdown Summary Recognizers for mobile hardware: Conventional approaches Reduced-resource embedded recognizers Network recognizers Distributed recognizers An alternative approach: Custom hardware 38

10 viii CONTENTS 2.5 Custom hardware for mobile speech recognition Motivation Hardware implementation: Feature extraction Hardware implementation: Feature scoring Hardware implementation: Search Hardware implementation: Performance and power evaluation Hardware implementation: Summary Conclusion 49 Bibliography 50 3 Embedded Automatic Speech Recognition and Text-to-Speech Synthesis Automatic speech recognition Mathematical formulation Acoustic parameterization Landmark-based approach Acoustic modeling Unit selection Hidden Markov models Language modeling Modifications for embedded speech recognition Feature computation Likelihood computation Applications Car navigation systems Smart homes Interactive toys Smartphones Text-to-speech synthesis Text to speech in a nutshell Front end Back end Rule-based synthesis Data-driven synthesis Statistical parameteric speech synthesis Embedded text-to-speech Evaluation Summary 94 Bibliography 94

11 CONTENTS 4 Distributed Speech Recognition Elements of distributed speech processing Front-end processing Device requirements Transmission issues in DSR Back-end processing ETSI standards Basic front-end standard ES Noise-robust front-end standard ES Tonal-language recognition standard ES Transfer protocol Signaling RTP payload format Energy-aware distributed speech recognition ESR, NSR, DSR 111 Bibliography Context in Conversation Context modeling and aggregation An example of composer specification Context-based speech applications: Conspeakuous Conspeakuous architecture B-Conspeakuous Learning as a source of context Implementation A tourist portal application Context-based speech applications: Responsive information architect Conclusion 133 Bibliography Software: Infrastructure, Standards, Technologies Introduction Mobile operating systems Voice over internet protocol Implications for mobile speech Sample speech applications Access channels Standards Standards: VXML 144 ix

12 x CONTENTS 6.6 Standards: VoiceFleXML Brief overview of speech-based systems System architecture System architecture: VoiceFleXML interpreter VoiceFleXML: Voice browser A prototype implementation SAMVAAD Background and problem setting Reorganization algorithms Minimizing the number of dialogs Hybrid call-flows Minimally altered call-flows Device-independent call-flow characterization SAMVAAD: Architecture, implementation and experiments Splitting dialog call-flows Conclusion Summary and future work 188 Bibliography Architecture of Mobile Speech-Based and Multimodal Dialog Systems Introduction Multimodal architectures Multimodal frameworks Multimodal mobile applications Mobile companion MUMS TravelMan Stopman Architectural models Client server systems Dialog description systems Generic model for distributed mobile multimodal speech systems Distribution in the Stopman system Conclusions 214 Bibliography 214

13 CONTENTS 8 Evaluation of Mobile and Pervasive Speech Applications Introduction Spoken interaction Mobile-use context Speech and mobility Evaluation of mobile speech-based systems User interface evaluation methodology Technical evaluation of speech-based systems Usability evaluations Subjective metrics and objective metrics Laboratory and field studies Simulating mobility in the laboratory Studying social context Long- and short-term studies Validity Case studies STOPMAN evaluation TravelMan evaluation Discussion Theoretical measures for dialog call-flows Introduction Dialog call-flow characterization m,q,a -characterization m,q,a -complexity Call-flow analysis using m,q,a -complexity Conclusions 257 Bibliography Developing Regions Introduction Applications and studies VoiKiosk HealthLine The spoken web TapBack Systems Challenges 278 Bibliography 278 Index 281 xi

About the Series Editors Xuemin (Sherman) Shen (M 97-SM 02) received a BSc degree in electrical engineering from Dalian Maritime University, China in 1982, and the MSc and PhD degrees (both in

15 About the Series Editors Xuemin (Sherman) Shen (M 97-SM 02) received a BSc degree in electrical engineering from Dalian Maritime University, China in 1982, and the MSc and PhD degrees (both in electrical engineering) from Rutgers University, New Jersey, USA, in 1987 and 1990 respectively. He is a Professor and University Research Chair, and the Associate Chair for Graduate Studies, at the Department of Electrical and Computer Engineering, University of Waterloo, Canada. His research focuses on mobility and resource management in interconnected wireless/wired networks, UWB wireless communications systems, wireless security, and ad hoc and sensor networks. He is a co-author of three books, and has published more than 300 papers and book chapters on wireless communications and networks, control and filtering. Dr. Shen serves as a founding area editor for IEEE Transactions on Wireless Communications; editor-in-chief for Peer-to-Peer Networking and Application; associate editor for IEEE Transactions on Vehicular Technology, KICS/IEEE Journal of Communications and Networks, Computer Networks, ACM/Wireless Networks and Wireless Communications and Mobile Computing. He has also served as a guest editor for IEEE JSAC, IEEE Wireless Communications and IEEE Communications Magazine. Dr. Shen received the Excellent Graduate Supervision Award in 2006, and the Outstanding Performance Award in 2004 from the University of Waterloo, the Premier s Research Excellence Award (PREA) in 2003 from the Province of Ontario, Canada, and the Distinguished Performance Award in 2002 from the Faculty of Engineering, University of Waterloo. Dr. Shen is a registered Professional Engineer of Ontario, Canada.

FUZZY LOGIC WITH ENGINEERING APPLICATIONS

FUZZY LOGIC WITH ENGINEERING APPLICATIONS Third Edition Timothy J. Ross University of New Mexico, USA A John Wiley and Sons, Ltd., Publication FUZZY LOGIC WITH ENGINEERING APPLICATIONS Third Edition FUZZY