The Connected World and Speech Technologies: It s About Effortless

Transcription

1 The Connected World and Speech Technologies: It s About Effortless Jay Wilpon Executive Director, Natural Language Processing and Multimodal Interactions Research IEEE Fellow, AT&T Fellow

2 In the Beginning Genesis 1:1 2

3 Front 3 view of the spoken digit recognizer (J. Suzuki & K. Nakata, Radio Research Labs, Japan, 1961)

4 It would be too simple to say that work in speech recognition is carried out simply because one can get money for it....the attraction is perhaps similar to the attraction of schemes for turning water into gasoline, extracting gold from the sea, curing cancer, or going to the moon John R. Pierce, Executive Vice President at AT&T Bell Labs (1969) 4

5 Good Morning America

6 Virtual Assistants Siri / Google Now Start at 37 seconds 6

7 AT&T Evolution of Speech Industry Evolution Invention of Industrial Leading Technologies & Services The The New Frontier Please say Collect call please AT&T, how may I help you? I have a problem with my bill You are running late for your meeting and you need gas Thanks, can you find a good restaurant for Sunday brunch Beyond Simple Commands Command & Control ( Press or say ) Large-vocabulary Speech Services Operator services Enterprise services Natural Language Services How May I Help You Customer care Enterprise services Watson Speech mashup Intelligent Multimodal Services Home automation Connected car Cloud API Big data analytics Emerging devices, wearables M2M Video and television Pioneered speech service industry Resulted in billions of dollars in savings Paradigm shift in user experience and automation Seamless communications, information access, commerce Connected Life

8 Connected World Ecosystem Speech and multi-modal technologies universally accessible to all devices and industries through a wide ecosystem of partners Voice Biometrics Machine Learning Speech Recognition Dialog Manager Large Enterprises Text to Speech Language Processing Devices Video 3 rd Party Developers Systems Integrators Solution Providers Security App Developers Cloud Providers Consumer Applications Healthcare Connected Car Home Automation Industries Customer Service Finance

9

10 Multimodal Virtual Assistants Effortless access to information, services, transactions anywhere, anytime Cloud-based virtual assistant manifests itself appropriately based on the device and environment Web Social Media Connected Car - Personalized - Adaptive - Connected Natural and effective Personal Data Emerging Devices - Spoken dialog - Language understanding - Multimodal interaction Retail Digital Home Office

11 The Science Behind the Service Language Generation Text to Speech Speech Recognition Dialog Speech and Language Technology Suite Biometrics Machine Learning Speech Translation Natural Language Processing

12 Speech Industry is Approaching a Cross Road Market Trends Market Challenges Hello Hola Growth in connected devices (smartphones, tablets, connected cars, wearables, home automation) Increased adoption of and access to cloud computing Desire to enhance and differentiate the consumer experience (virtual assistants, multimodal solutions) Demand for business intelligence uses increasing (speech, video, text) Low customer adoption due to poor user experience and performance Low accuracy in the real world Complexity due to variety of languages and devices Competing / complementary input technologies (touchscreen, keyboard, multi-touch and visual gestures) Slow development/adaption of applications

13 Its all about Effortless Scientists / Engineers Software vendors Platform vendors Application developers Customers 13

14 Speech/Language Industry Ecosystem - Today Invention Software Vendors Platform Vendors App Developers Ideas Experiments Trial Ideas Experiments Trial Multiple code bases Multiple version Distribution Integration Testing Expensive, Inefficient and Slow Inspiration time Hardware management Integration Optimization Testing Version code Prof. Services Distribution (Oh, did I add testing?) Software maintenance Platform dependence Testing Optimization Version code Prof. Services Slow Expensive and Slow

15 Speech/Language Industry Ecosystem - Today Invention App Developers Ideas Experiments Trial Ideas Experiments Trial Inspiration time Software maintenance Platform dependence Testing Optimization Version code Prof. Services Slow 15

16 Its all about Effortless Invert the Food Chain Customers Intuitive, useful, secure, personalized, anywhere, anytime, device agnostic Application developers technologies easy to integrate, scalable tools, simple interfaces, limited need for professional services Platform providers technologies easy to integrate, interfaces simple and stable, tools Software Vendors platform independence, scalable engines, tools, simple and stable interfaces Scientists / Engines it works, fast to market, platform semiindependence, easy to update 16

17 Getting to Effortless Start with the Customer Invention App Developers Ideas Experiments Trial Ideas Experiments Trial Inspiration time Innovative, Fast, Device Agnostic Software maintenance Platform dependence Testing Optimization Version code Prof. Services Fast 17

18 Network Connectivity Cloud Computing Pag e 18 Bandwidth

19 Speech/Language Industry Ecosystem - Moving Invention App Developers Ideas Experiments Trial Ideas Experiments Trial Inspiration time Cloud Engines APIs and libraries One customer Single code base Low maintenance Simple testing Virtual on-demand machines Optimized Personalization Pricing << Software licensing Fast Innovative, Fast, Device Agnostic Software maintenance Platform dependence Testing Optimization Version code Prof. Services Fast 19

20 Aggressive Market Segmentation Google Apple Microsoft Amazon AT&T Nuance Speech APIs Speech APIs Android ios Windows Android All All Google Now Siri Cortana Kindle mva Nina 20 Device/OS dependent Device/OS independent

21 Getting to Effortless Independence Cloud Engines APIs and Libraries Android ios Windows Linux Other Application Developers 21

22 Getting to Effortless Scientists and Engineers Are we in the? Golden Age of Engineering for Speech Technologies Age of Invention for Language Technologies Data is driving innovation 22

23 Technical Challenges to Move Industry Forward Invention of Industrial Leading Technologies & Services The The New Frontier Please say Collect call please AT&T, how may I help you? I have a problem with my bill You are running late for your meeting and you need gas Thanks, can you find a good restaurant for Sunday brunch Beyond Simple Commands Command & Control ( Press or say ) Large-vocabulary Speech Services Operator services Enterprise services Natural Language Services How May I Help You Customer care Enterprise services Speech mashup Intelligent Multimodal Services Home automation Connected car Cloud API Big data analytics Emerging devices, wearables M2M Video and television Pioneered speech service industry Resulted in billions of dollars in savings Paradigm shift in user experience and automation Seamless communications, information access, commerce Connected Life

24 Multimodal Virtual Assistants Seamless communications, information access and commerce anywhere & anytime Market Opportunities Phones/tablets/PCs: unified access to communications, information and transaction Enterprise customer care moving from IVR to mobile web E-commerce/retail sales Connected car & home Entertainment: easy access to content everywhere Wearable devices: new industry, hands free control Entertainment Home Automation Wearables Connected Car Enterprise Verticals Customer Care

25 Multimodal Virtual Assistants Getting to Effortless Seamless communications, information access and commerce anywhere & anytime Challenges Addressed by Scientists/Engineers It has to work: Everywhere, be intuitive, be available whenever needed Always on: Handsfree, ROBUST, speech detection, sensors everywhere Multimodality: Connected devices need multimodal technologies Context-awareness: Pragmatics of the world around each user Conversational interaction: Effortless user experience require cooperative, personalized & multimodal human-machine dialog Secure and Personalized: Knows its me and learns about me adaptive Proactive: Reasoning, M2M, not just Q/A Incrementality: Users expect fluid/engaging interaction Rapid application authoring: Enterprises demand easy cold start for new domains and rapid support for ongoing improvements (e.g., tools); unsupervised (semi-supervised) training Internationalization: multilingual, translation Data, data, data: Vast amounts (content, personal, dynamic) ingest, model, adapt, learn

26 AT&T Watson Multimodal Intelligent Virtual Assistant 26

27 AT&T Watson Multimodal Intelligent Virtual Assistant Planning a night out on the town events, movies and restaurants Watson VA Features mva Multimodal Concierge Multimodal input and output effortless Supports recognition, understanding, generation and gesture Integration of ASR, NLP and Dialog maximizes performance Natural conversation supports contextually-aware NLU and reference resolution Supports Targeted Clarification: error handling through machine learning and natural language generation Built for rapid authoring and adaptation Dynamic hierarchical language models (DHLM) enable cold start for new domains and rapid adaptation Bootstrap new domains with generic NLP pipeline Watson s Semantic Abstraction Layer (SAL) facilitates easy expansion to new domains and new content 19

28 AT&T Translator TOUCH (Fox TV Show) 28

29 Driving to Intelligence Language Generation Text to Speech Speech Recognition Dialog Data Biometrics Machine Learning Speech Translation Natural Language Processing

30 Natural Language Processing (NLP) NLP Analysis Intent classification semantics Named entities Part-of-speech Syntactic parse Coreference labels Text normalization Sentiments Topic detection Information extraction Emotion detection Anomaly detection Summarization Sentence planning System intent Semantic frames NLP Generation

31 Intelligence Information extraction Basic Steps Input sentence Sentiment analysis This authentic Venezuelan food is delicious homemade and great on price and service Problem Detection My phone is not calling or receiving messages This girl told me Verizon is down Classify Sentiment/No Sentiment Problem/No Problem Identify triggers This authentic Venezuelan food is delicious homemade and great on price and service My phone is not calling or receiving messages This girl told me Verizon is down Identify related targets This authentic Venezuelan food is delicious homemade and great on price and service My phone is not calling or receiving messages This girl told me Verizon is down Summarize Aspect/attribute Venezuelan food Venezuelan food: Price Opinion authentic delicious homemade great Problem object My Phone Verizon Problem not calling or receiving down Venezuelan food: Service

32 35

33 In the Beginning Genesis 1:1 36

34 Rethink Possible