Lucida Sirius and DjiNN Tutorial

Size: px

Start display at page:

Download "Lucida Sirius and DjiNN Tutorial"

Ursula Booth
6 years ago
Views:

1 Lucida Sirius and DjiNN Tutorial Speakers: Johann Hauswald, Michael A. Laurenzano, Yunqi Zhang Organizers: Johann Hauswald, Michael A. Laurenzano, Yunqi Zhang, Lingjia Tang, Jason Mars

2 Before We Begin Conference wifi SSID: HPCA_PPoPP_CGO Password: BARCELONA2016 Suggested prerequisites VirtualBox Chrome VM with demo software lucida-djinn-tutorial.tgz (USB sticks) Hardware resources ~6GB RAM, 50GB disk 2

3 Schedule 9:00-9:15 Welcome Section 1 - Lucida 9:15-9:30 Introduction to Lucida Section 2 Lucida Suite 9:30-9:45 Core Algorithmic Components of IPAs 9:45-10:00 Hands-on: Lucida Suite Section 3 - DjiNN and Tonic 10:30-10:45 Deep Learning in Intelligent Web Services 10:45-11:00 Tonic Suite: Background and Hands-on Section 4 - LucidaEco 11:00-11:15 Introduction to LucidaEco 11:15-12:00 Hands-on: Build Your Own IPA 3

4 Topic: Introduction to Lucida Speakers: Johann Hauswald

5 5

6 Rise of the Wearables 40% $80bn 6

7 Questions Arise What are the impacts on our data centers? Can we scale to millions (or billions) of users? How do we design the right systems? 7

8 The Hard Place: Build an end to end intelligent personal assistant. Sirius [ASPLOS 15] 7

9 Answer Display Answer Image Matching Voice Server Mobile Users Image Database Automatic Speech-Recognition Query Classifier Execute Action Question-Answering Search Database Action Question or Action Image Data Question Image 9

10 Lucida Hierarchy Users Users Voice Command (VC) Voice Query (VQ) Voice Command Voice Query (VC) (VQ) Automatic-Speech Recognition (ASR) Automatic-Speech Recognition (ASR) Query Taxonomy Voice-Image Query (VIQ) Question Answering (QA) Question Answering (QA) Signal Processing HMM/GMM or HMM/DNN Voice-Image Query (VIQ) Natural Language Processing Regular Stemmer Expression Conditional Random Fields Query Taxonomy Image Matching (IMM) IPA Services Image Matching (IMM) Image Processing IPA Services Tasks Feature Extraction Algorithmic Feature Description Components 10

11 Speech Acoustic Model Feature Extraction Input Layer Language Model Hidden Layers Output Layer Word Dictionary Trained Data Speech Decoder Scored HMM states HMM DNN Scoring or GMM scoring Viterbi Search Feature Vectors Who was elected 44th President? ASR 0.9 IMM Image SURF Feature Description Calculate Hessian Matrix Haar Wavelet Build Scale-Space Orientation Assignment Find Keypoints Keypoints Keypoint Descriptor Three Horsemen of Lucida QA Descriptor Database Input Filter Who was elected 44th president? Image Descriptors Question-Answering Document Selector Regex elected 44th Stemmer president CRF Scores Document Filters SURF Feature Extraction Barack Obama Websearch ^[0-9,th]$ 44th -ed elect elected VERB 44th president NUM N 11

12 Project Status Recent Publication Open source community BayMax [ASPLOS 16] lucida-ai.slack.com Student projects Independent studies Summer interns 12

13 Topic: Core Algorithmic Components of IPAs Speaker: Yunqi Zhang

14 How does Lucida work? Users Voice Command (VC) Automatic-Speech Recognition (ASR) Voice Query (VQ) Voice-Image Query (VIQ) Question Answering (QA) Image Matching (IMM) Query Taxonomy IPA Services CMU Sphinx 16

15 How does Lucida work? Users Voice Command (VC) Voice Query (VQ) Voice-Image Query (VIQ) Automatic-Speech Recognition (ASR) Question Answering (QA) Signal Processing Natural Language Processing Image Matching (IMM) Image Processing Query Taxonomy IPA Services Tasks 16

16 How does Lucida work? Signal Processing Natural Language Processing Image Processing Tasks Gaussian Mixture Model (GMM) or Deep Neural Network (DNN) 85% or 78% Regular Stemmer Expression Conditional Random Fields 85% Feature Extraction Feature Description 97% Algorithmic Components 7 kernels: 92% of total execution of Lucida 16

17 What is Lucida Suite? Suite of 7 workloads extracted from the end-to-end Lucida intelligent personal assistant Suite includes: Standalone workloads Pretrained models (when applicable) Inputs for all workloads Available at: 17

18 Objectives of Lucida Suite Represent emerging datacenter workloads speech recognition, computer vision, natural language processing Cover key components in the application Suite represents 92% of the total cycles Implementations Single thread CPU Pthread CPU GPU Easy to use Implemented in C/C++ and CUDA 18

19 Lucida Suite Service Workload Task Platforms Image Matching Feature Extraction Computer Vision Pthread, GPU Feature Description Computer Vision Pthread, GPU Automatic Speech Recognition GMM Scoring Speech Recognition Pthread, GPU DNN Scoring Speech Recognition Pthread, GPU Question Answer Stemmer Natural Language Processing Pthread, GPU Regular Expression Natural Language Processing Pthread Conditional Random Fields Natural Language Processing Pthread 19

20 Image Matching

21 Image SURF Feature Extractor SURF Feature Descriptor Calculate Hessian Matrix Haar Wavelet Build Scale-Space Orientation Assignment Find Keypoints Keypoints Keypoint Descriptor Image Descriptors Descriptor Database Accelerated workload: 97% 21

22 Automatic Speech Recognition

23 Automatic Speech Recognition (ASR) Recognition Process Input Layer Hidden Layers Output Layer Speech HMM DNN Feature Extraction Feature Vectors GMM Pre-processing ca p i Viterbi Algorithm t al 23

24 Automatic Speech Recognition (ASR) Recognition Process Input Layer Hidden Layers Output Layer 24

25 Question Answering

26 Question Answering Input Filters Who was elected 44th president of the United States? Document Database Candidate Phrases: (*) was elected (*) was the 44th Document Filters Answer Selector Barack Obama e.g., wikipedia Sentences in Candidate Documents: wikipedia/barack_obama wikipedia/presidents_of_the_us 26

Barack Obama Candidate Phrases: (*) was elected (*) was the 44th Document Filters e.g.

27 Question Answering Input Filters Who was elected 44th president of the United States? Document Database 85% of the execution Conditional Random Fields Stemmer Regular Expression Barack Obama Candidate Phrases: (*) was elected (*) was the 44th Document Filters e.g., wikipedia Answer Selector Sentences in Candidate Documents: wikipedia/barack_obama wikipedia/presidents_of_the_us 26

28 Topic: Hands-on: Lucida Suite Speaker: Yunqi Zhang

29 Lucida Suite 28

30 Setup Prerequisites Install VirtualBox Install Chrome Copy the tutorial VM USB thumb drive 29

31 Configure VirtualBox Unpack lucida-djinn-demo.tgz into ~/VirtualBox VMs/ Add lucida-djinn-demo to VirtualBox Machine -> Add Port forwarding Machine -> Settings -> Network -> Adapter 1 -> Advanced -> Port Forwarding 30

32 Stand up the VM Start! username/password: demo/demo open Terminal application cd ~/hpca-demo/source-code/lucida/lucida-suite Follow along with Yunqi! 31

33 Topic: Deep Learning in Intelligent Web Services Speaker: Michael A. Laurenzano

34 10,000 foot view of Deep Neural Networks (DNN) What is deep about a DNN? More layers Often, bigger layers also Example DNN in Lucida ASR Input Layer Hidden Layers Output Layer HMM State p(hmmstate AcousticModel) 33

35 Why DNN? Emerging in a number of domains Big data analytics Video/image/audio recognition Language semantics/translation Many components of IPAs like Lucida can leverage DNN 34

36 DNNs in the IPA pipeline Answer Display Answer Image Matching Voice Server Mobile Users Image Database Automatic Speech-Recognition Query Classifier Execute Action Question-Answering Search Database Action Question or Action Image Data Question Image 35

37 Studying DNNs as an architect is a challenge 36

38 Studying DNNs as an architect is a challenge What should my DNN do? Network configuration How many layers? How to configure those layers? Input Layer Hidden Layers Output Layer Interconnections? Activation functions? Model training HMM State p(hmmstate AcousticModel) When are the results accurate enough? How to choose a training set? 36

39 DNN as a Service Image Classification Unified, highly optimized appliance for DNN Digit Recognition Facial Recognition Speech Recognition Natural Language Processing 37

Introducing DjiNN and Tonic [ISCA 15] DjiNN Infrastructure for DNN as a service Extensible to a number of DNN services CPU-optimized implementation GPU-optimized implementation Tonic

40 Introducing DjiNN and Tonic [ISCA 15] DjiNN Infrastructure for DNN as a service Extensible to a number of DNN services CPU-optimized implementation GPU-optimized implementation Tonic Suite 7 complete DNN services covering important IPA use cases Configured networks and trained models are provided No machine learning expertise required 38

41 Topic: Tonic Suite Components Speaker: Johann Hauswald

CHK Name entity recognition (NER) NER FACE Speech Recognition (ASR) Task Automatic speech recognition (ASR) Natural

42 Tonic Suite Image Processing Facial recognition (FACE) Digit recognition (DIG) Speech Processing Image classification (IMC) Users Part-of-speech tagging (POS) DIG It s business, Superman Natural Language Processing Task POS Chunking (CHK) CHK Name entity recognition (NER) NER FACE Speech Recognition (ASR) Task Automatic speech recognition (ASR) Natural Language Processing IMC Image Task business (noun) Superman (P. noun) It s (VP, B-NP) business (NP, I-NP) Superman (PERSON) 40

43 Deep Neural Networks (DNNs) Feature Maps Inference 0.1 Spiderman 0.7 Superman 0.2 Batman Input Convolutional layer Pooling layer Fully Connected layer Network Architecture 41

44 Deep Neural Networks (DNNs) speech features Inference Who, is, this who is this w 0 w 1 w k w 0 w 1 w k w 0 w 1 w k word vectors Fully Connected layer Who (PRONOUN) is (VERB) this (PRONOUN) Network Architecture 41

45 Tonic Suite Applications IMC Users Image Task DIG DjiNN DNN Service FACE DNN Architecture Speech Recognition (ASR) Task It s business, Superman POS CHK NER business (noun) Superman (P. noun) It s (VP, B-NP) business (NP, I-NP) Superman (PERSON) IMC DIG POS FACE CHK ASR NER Trained Models Natural Language Processing Task 42

46 Basic characterization Single CPU Not all DNN services are created equal Accelerating DNN can provide significant speedup 43

47 Tonic Suite 44

48 Topic: Introduction to LucidaEco Speaker: Michael A. Laurenzano

49 What is LucidaEco? The next step in the evolution of Lucida Open source end-to-end IPA ecosystem Rich set of infrastructure and AI tools Easily compose customized, scalable, production-quality IPAs Speech Services Image Services Text Services Frontend Applications Image Speech Query Pipeline Manager Response Text Micro-Services 46

50 Micro-service Architecture Designed to achieve Scalability Modularity Extensibility Independently deployable intelligent services Common interface - create, learn, infer Speech Services Image Services Text Services Frontend Applications Image Speech Query Pipeline Manager Response Text Micro-Services 47

Micro-service Design Common interface to each micro-service Create - new knowledge base/model Learn - add information to KB/model Infer - retrieve knowledge Inter-service communication RPC

51 Micro-service Design Common interface to each micro-service Create - new knowledge base/model Learn - add information to KB/model Infer - retrieve knowledge Inter-service communication RPC abstraction (Facebook s Thrift) for most websockets for streaming audio Language/platform issues hidden from micro-service Virtualized, platform-independent deployment docker (container-based) 48

52 Query Pipeline Manager Orchestrate the pipeline of IPA components Goals Optimize computational footprint Hardware accelerators and heterogeneity Micro-service co-location Speech Services Image Services Text Services Frontend Applications Image Speech Query Pipeline Manager Response Text Micro-Services 49

53 LucidaEco is Growing Micro-service ecosystem ASR (Kaldi, Sphinx, different languages) QA (OpenEphyra, Qanta, YodaQA) Machine Translation, Sentiment, Summarization Image matching, object recognition Frontend apps web, mobile, tablet; others coming LucidaEco as a research platform AI algorithms/implementations in an end-to-end pipeline Scalability - acceleration, virtualization, high-bw communication substrates Burgeoning community of developers/researchers Join us! 50

54 Topic: Build Your Own IPA Speakers: Michael A. Laurenzano and Johann Hauswald

55 The Architecture of Your IPA Open- Ephyra QA Your KB Kaldi ASR Docker Container 1 Docker Container 2 VM (VirtualBox) Host Machine 52

56 Setup Prerequisites Install VirtualBox Install Chrome Copy the tutorial VM USB thumb drive 53

VirtualBox Machine -> Add Port forwarding Machine ->

57 Configure VirtualBox Unpack lucida-djinn-demo.tgz into ~/VirtualBox VMs/ Add lucida-djinn-demo to VirtualBox Machine -> Add Port forwarding Machine -> Settings -> Network -> Adapter 1 -> Advanced -> Port Forwarding 54

58 Run Your IPA Start! username/password: demo/demo open Terminal application cd ~/hpca-demo/source-code/lucida/ Running the demo $ docker-compose up ON THE HOST: 55

59 Build Your Own IPA 56

60 Thank You!!! Tutorial Website lucida.ai Lucida Suite Tonic Suite LucidaEco 57

Sirius: An Open End-to-End Voice and Vision Personal Assistant and Its Implications for Future Warehouse Scale Computers

Sirius: An Open End-to-End Voice and Vision Personal Assistant and Its Implications for Future Warehouse Scale Computers Johann Hauswald, Michael A. Laurenzano, Yunqi Zhang, Cheng Li, Austin Rovinski,