How Agencies are Developing Accessible E-Government. By Stephen Condon Software Engineer Fig Leaf Software

Transcription

1 How Agencies are Developing Accessible E-Government By Stephen Condon Software Engineer Fig Leaf Software 1

2 Objectives 2 Define a custom skill for Amazon Alexa Define Custom Intents with parameters (slots) Define Effective Utterances Choose an appropriate invocation name for your custom skill Handle requests to your skill by writing and deploying ColdFusion Web Services Define a configuration for your Alexa skill Test your skill in a simulator and on an Amazon device. Submit your Alexa skill for certification to make it available to Amazon customers

3 Creating Skills for Amazon Alexa 3 Alexa is a voice assistant that animates a series of devices and apps from Amazon.com Echo Echo Dot Echo Show (New!) Tap Fire TV Capable of voice interaction via far-field voice recognition, music playback, home automation, purchasing of goods Has a robust ecosystem of "skills" developed by third-parties using Amazon's Alexa Skills Kit. Companion App for Smartphones and Tablets Coming Soon: Support for Facebook Messenger, Slack, Twilio, and more!

4 NPS Alexa Skill- coming REAL SOON! 4 Returns a park's description Returns all active alerts for a park Returns contact information for a park Returns directions to a park Returns up to three events for a date for a park Returns the last three news releases published within the last month for a park Returns parks in a state Returns a random fact for a park Returns a random fact for a random park

5 Using the Amazon Skills Kit (ASK) 5 Enables you to teach Alexa how to respond when a user asks it a question. Custom Skill API Smart Home Skill API Flash Briefing Skill API Video Skill API All skills require that you build a cloud-based service that handles the requests for your skill type. NOTE: This presentation has code samples for the Custom Skill API and Flash Briefing API

6 Supporting Skill Cards 6 In addition to returning an audio response, Alexa can also optionally return visual "cards" to the user's Amazon Alexa app. Skill detail cards describe what your skill does and how to use it. Home cards describe or enhance the user s voice interactions with Alexa. NOTE:

7 Developing Custom Skills 7 Create a custom web service or use Amazon AWS Lambda. AWS Lambda is a service that lets you run code in the cloud without managing servers. You can write Lambda functions in Java, Node.js, or Python. Alternately, you can write your own web service and host it with any cloud hosting provider, such as Heroku. Non-Lambda (e.g. ColdFusion) functions must be accessible via HTTPS You will also need to define a custom skill model

8 Defining a Skill 8 Intents Custom Slot Types Utterances An Invocation Name A Cloud-Based Service A Configuration

9 Specifying Intents 9 { "intents": [ { "intent": "getparkevents", "slots": [ { "name": "date", "type": "AMAZON.DATE" }, { "name": "park", "type": "LIST_OF_PARKS" } ] }, { "intent": "getparkdirections", "slots": [ { "name": "park", "type": "LIST_OF_PARKS" } }, { "intent": "AMAZON.HelpIntent" }, { "intent": "AMAZON.NoIntent" }, { "intent": "AMAZON.YesIntent" } ] }

10 Custom Slot Types Define an enumeration for a parameter that's passed to your web service. Helps the Alexa service more accurately parse your speech. 10

11 Specifying Utterances 11 Utterances link words or phrases to an intent and slots. Registered it the Alexa Developer Skills Portal. For example: GetParkContacts phone number for {Park} GetParkContacts tell me the phone number for {Park} GetParkContacts what is the phone number for {Park} GetParkDescription tell me about {Park} GetParkDescription describe {Park} GetParkDirections directions to {Park} GetParkDirections tell me the directions to {Park} GetParkDirections give me directions to {Park} GetParkDirections what are the directions to {Park} GetParkEvents events for {Park} GetParkEvents {Date} events for {Park}

12 Choosing the Invocation name 12 There are three ways that users may say your invocation name Invoking the skill with a request "Alexa, Ask NPS Ranger about tomorrow s events for Denali" "Alexa, Talk to NPS Ranger and get tomorrow s events for Denali" "Alexa, tell me about tomorrow s events for Denali using NPS Ranger" Invoking the skill without a request "Alexa, open NPS Ranger" "Alexa, ask NPS Ranger" "Alexa, start NPS Ranger" Invoking the skill using just the invocation name "Alexa, NPS Ranger"

13 Choosing the Invocation Name 13 One-word invocation names are not allowed, unless the invocation name is unique to your brand/intellectual property. Invocation names which are names of people or places are not allowed, unless they contain other words in addition to the name. Two-word invocation names are not allowed if one of the words is a definite article ( the ), indefinite article ( a, an ) or preposition ( for, to, of ).

14 Handling Function Inputs 14 { "session": { "sessionid": "The User's Session ID", "application": { "applicationid": "Your Skill ID" }, "attributes": {}, "user": { "userid": "Some Amazon User Id" }, "new": false }, "request": { "type": "IntentRequest", "requestid": The request ID", "intent": { "name": "GetParkEvents", "slots": { "Date": { "name": "Date", "value": " " }, "Park": { "name": "Park", "value": "Acadia" } } } }

15 Speech Synthesis Markup Language (SSML) 15

16 Function Output 16 { "version": "1.0", "response": { "outputspeech": { "ssml": "<speak><p>there is one event: </p><p> At an unspecified time today, Become a Junior Ranger: Pick up a free Junior Ranger Activity Booklet at any visitor center, discover the park, and earn your badge!. </p><p>can I help you with anything else?</p></speak>", "type": "SSML" }, "card": { "content": Become a Junior Ranger: Pick up a free Junior Ranger Activity Booklet at any visitor center, discover the park, and earn your badge!", "title": "Events at Rocky Mountain" }, "speechletresponse": { "outputspeech": { "ssml": "<speak><p>there is one event: </p><p> At an unspecified time today, Become a Junior Ranger: Pick up a free Junior Ranger Activity Booklet at any visitor center, discover the park, and earn your badge!. </p><p>can I help you with anything else?</p></speak>" }, "card": { "content": Become a Junior Ranger: Pick up a free Junior Ranger Activity Booklet at any visitor center, discover the park, and earn your badge!" }, "reprompt": { "outputspeech": { "text": "Can I help you with anything else?" } }, "shouldendsession": false } }, "sessionattributes": {} }

17 Server-Side Handler Considerations Handle the 4 types of requests LaunchRequest IntentRequest SessionEndedRequest (optional) AudioPlayer and PlaybackController (optional) Keep a History / Session Info (Alexa connection is stateless) Handle Standard Intents HelpIntent YesIntent NoIntent StopIntent Convert Amazon Data Types to CF Data Types Date formats 17

18 Putting it all together Register your Skill in the Amazon Skills Kit Configure the Interaction Model Configuring the Endpoint Testing the Skill Debugging the Skill 18

19 Configuring the Interaction Model 19

20 Configuring the Endpoint 20

21 Testing the Skill 21

22 Debugging the Skill Use cflog copiously when debugging your code. NPS uses its API to pull in the data that is returned to the user. Servers managed by agencies usually require discussions with your agency s infrastructure teams to make sure data is accessible, even from other servers under your agency s control. Plan on having an extended beta testing period before publishing your skill. Different agencies have different concerns, but for NPS, we have quite a few parks with names that Alexa does not handle well, which beta testing caught. This allows you to add altered SSML to phonetically spell some of the troublesome park names. 22

23 Code Walkthrough 1/4 23

24 Code Walkthrough 2/4 24

25 Code Walkthrough 3/4 25

26 Code Walkthrough 4/4 26

27 YOUR TURN Demo 27 The National Park Service Download the framework from

28 Flash Briefing API Alexa devices have a Flash Briefing, which is a configurable set of content that provides a quick overview of news, weather, etc. Invocation: Alexa, what s my Flash Briefing or Alexa, what s the news Defaults to using the Reuters TV and Weather flash briefing skills Provides original content to potential customers on a daily basis. You can configure your Flash Briefing under Settings in the Alexa companion app The Flash Briefing API accepts either audio clips or text that Alexa converts to speech 28

29 Creating a Flash Briefing Skill Flash Briefing Skills are simpler to configure than Custom Skills The Flash Briefing API accepts RSS or JSON data The Flash Briefing accepts up to 5 items The name of the Flash Briefing skill will not be used for invocation. Users find it in the Alexa companion app when configuring their Flash Briefing 29

30 Configuring the Flash Briefing Flash Briefing skills do not allow you to modify the interaction model. Error Handling you need to add text for Alexa to speak to the user if the Flash Briefing feed encounters an error 30

31 Configuring the Flash Briefing Alexa uses the preamble to introduce the content you re providing in the Flash Briefing You can configure multiple feeds per Flash Briefing Each feed needs a type (text or audio) The feed needs an update frequency, which tells Alexa how often to check in for new content The URL is the endpoint for the feed The Feed Icon is used on the Alexa Companion app when your feed is called 31

32 Flash Briefing Output Flash Briefings can take JSON or RSS JSON is Amazon s preferred format. Need to return JSON using <cfcontent type= application/json; charset: utf-8 > so that we get the right content type and encoding Sample output: 32

33 Flash Briefing Output uid Unique identifier for the record, in the example in the last slide, it s a UUID generated by CreateUUID() updatedate A date object following the format yyyy-mm-ddthh:mm:ss:lz, and needs to be provided as a UTC timestamp titletext Populates the title of the card on the Alexa companion app maintext the content for Alexa to read off as part of the Flash Briefing, and is also used as card text in the Alexa companion app redirectionurl the URL associated with the content 33

34 Flash Briefing Function 34

35 Flash Briefing Demo Flash Briefing Skill using the NPS API ( We read off the most recent news release published on 35

36 Summary (1 of 2) The Alexa Skills Kit (ASK) enables you to add custom skills, home automation skills, and flash briefing skills. The Alexa mobile companion app enables you to configure Alexa hardware, enable skills, and view skill cards. Skills are comprised of: Intents Utterances An Invocation Name A cloud-based service A configuration 36

37 Summary (2 of 2) Custom intents are defined in JSON and may support optional parameters, or slots. Whenever possible, use built-in intents to ensure consistent behavior across all Alexa skills. One-word invocation names are not allowed, unless the invocation name is unique to your brand/intellectual property. Include Speech Synthesis Markup Language (SSML) in your response to adjust how Alexa pronounces words and phrases. Thoroughly test your skills in the simulator and on a device before submitting them to Amazon for approval. A device must be registered under the same address as your Developer's account for it to be used in testing. Verify that your endpoint is receiving data from your Alexa service. 37