Voice-controlled Home Automation Using Watson, Raspberry Pi, and Openwhisk

Transcription

1 Voice-controlled Home Automation Using Watson, Raspberry Pi, and Openwhisk

2 Voice Enabled Assistants (Adoption)

3 Voice Enabled Assistants (Usage)

4 Voice Enabled Assistants (Workflow) Initialize Voice Recording Record Audio Apply DSP algorithms Transcribe Audio to text Interpret text result Execute action

5 Hotword Background process that listens for specific speech pattern Ok, google Alexa Hey Siri Able to run on devices with limited resources (Arduino, Raspberry Pi)

6 Hotword

7 Microphone Array Consists of multiple microphones Able to record all channels simultaneously Separate streams are processed to increase transcription confidence

8 Watson Speech to Text Input audio via HTTP or Websocket curl -X POST -u <username>:<password> --header "Content-Type: audio/flac " { "confidence": 0.891, "transcript": "Turn on the light in the living room" }

9 Conversation Intent Action user wants to trigger Entity Context for action Dialog Determines response, next step

10 Conversation Service {... input": [ { text": turn on the fan }] } {... entities": [ { confidence": , value": fan }], intents": [ { confidence": , intent": turnon }] }

11 Conversation

12 Demo Architecture

13 OpenWhisk provides an elegant solution OpenWhisk is a cloud platform that executes code in response to events

14 Why Serverless? Focus on code, not maintenance Less overhead Reusability Scales on demand Usage based pricing

15 Respond to events Database changes Sensor value Cognitive trends Scheduled tasks Data from Mobile devices

16 Triggers, actions, rules (and packages) o Services define the events they emit as triggers. o o o Developers associate actions to handle the events via rules. Packages are used to bundle and distribute sets of actions

17 Triggers T A class of events that can happen Data changes Social events User input Device readings Location updates

18 Actions A Code that runs in response to an event (that is, an event-handler)

19 Actions A Can be written in a variety of languages, such as JavaScript, Python, Java, and Swift function main(msg) { return { message: 'Hello, ' + msg.name + ' from ' + msg.place }; };

20 Actions A Can be composed to create sequences that increase flexibility and foster reuse AA := A1 + A2 + A3 AB := A2 + A1 + A3 AC := A3 + A1 + A2

21 Rules R An association of a trigger to an action in a many to many mapping. R := T A

22 Packages P A shared collection of triggers and actions A read A write T changes Open Source A post T topic A translate Third Party A forecast Yours T commit A myaction T myfeed

23 OpenWhisk enables event driven applications 2 Cloudant GitHub OpenWhisk 1 Weather Event Providers Slack An event occurs, for example Commit pushed to GitHub repository Data entered in Cloudant JS Swift Docker Which triggers execution of associated OpenWhisk action

24 OpenWhisk awaits events, fetches mapped code, runs it in a container 1 Trigger 2 Pool of actions OpenWhisk Engine JS Swift Docker 3 Running action Running action Running action

25 Watson IoT Platform (MQTT) Platform enables apps to share and consume data collected by authenticated devices, sensors.. Provides real-time and REST APIs to communicate with your devices and consume the data

26 CitizenM

27 WiFi vs RF Outlet

28 RF Outlet (Schematic)

29 IR Outlet (Schematic)