Introduction to Siri Shortcuts

Transcription

1 #WWDC8 Introduction to Siri Shortcuts Session 2 Ari Weinstein, Siri Willem Mattelaer, Siri 208 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

2

3 Shortcuts let you expose the capabilities of your apps to Siri.

4

5

6

7

8

9

10

11

12

13 Adopting shortcuts Optimizing for suggestions Privacy Media shortcuts

14 Adopting Shortcuts

15 Creating Shortcuts

16 Creating Shortcuts Define Shortcut

17 Creating Shortcuts Define Shortcut Donate Shortcut

18 Creating Shortcuts Define Shortcut Donate Shortcut Handle Shortcut

19 What Makes a Great Shortcut? Every shortcut should: Accelerate the user to perform a key function of your app Be of persistent interest to the user Be executable at any time

20 Shortcuts APIs NSUserActivity Intents

21 Choosing an Adoption Strategy Use NSUserActivity if your shortcut: Opens something in your app Represents showing items that you index in Spotlight or offer for Handoff

22 Choosing an Adoption Strategy For the best experience, adopt Intents. Intents-based shortcuts can: Run inline, without launching your app Include custom voice response or a custom UI Include granular predictions

23 NSUserActivity

24 NSUserActivity Define your shortcut Declare a type in NSUserActivityTypes in your Info.plist <key>nsuseractivitytypes</key> <array> <string>com.myapp.name.my-activity-type</string> </array>

25 // How to donate a shortcut let useractivity = NSUserActivity(activityType: "com.myapp.name.my-activity-type") useractivity.iseligibleforsearch = true useractivity.iseligibleforprediction = true useractivity.title = "Activity" useractivity.userinfo = ["key": "value"] useractivity.suggestedinvocationphrase = "Let's do it" let attributes = CSSearchableItemAttributeSet(itemContentType: kuttypeitem as String) let image = UIImage(named: "myimage")! attributes.thumbnaildata = image.pngdata() attributes.contentdescription = "Subtitle" useractivity.contentattributeset = attributes viewcontroller.useractivity = useractivity

26 // How to donate a shortcut let useractivity = NSUserActivity(activityType: "com.myapp.name.my-activity-type") useractivity.iseligibleforsearch = true useractivity.iseligibleforprediction = true useractivity.title = "Activity" useractivity.userinfo = ["key": "value"] useractivity.suggestedinvocationphrase = "Let's do it" let attributes = CSSearchableItemAttributeSet(itemContentType: kuttypeitem as String) let image = UIImage(named: "myimage")! attributes.thumbnaildata = image.pngdata() attributes.contentdescription = "Subtitle" useractivity.contentattributeset = attributes viewcontroller.useractivity = useractivity

27 // How to donate a shortcut let useractivity = NSUserActivity(activityType: "com.myapp.name.my-activity-type") useractivity.iseligibleforsearch = true useractivity.iseligibleforprediction = true useractivity.title = "Activity" useractivity.userinfo = ["key": "value"] useractivity.suggestedinvocationphrase = "Let's do it" let attributes = CSSearchableItemAttributeSet(itemContentType: kuttypeitem as String) let image = UIImage(named: "myimage")! attributes.thumbnaildata = image.pngdata() attributes.contentdescription = "Subtitle" useractivity.contentattributeset = attributes viewcontroller.useractivity = useractivity

28 // How to donate a shortcut let useractivity = NSUserActivity(activityType: "com.myapp.name.my-activity-type") useractivity.iseligibleforsearch = true useractivity.iseligibleforprediction = true useractivity.title = "Activity" useractivity.userinfo = ["key": "value"] useractivity.suggestedinvocationphrase = "Let's do it" let attributes = CSSearchableItemAttributeSet(itemContentType: kuttypeitem as String) let image = UIImage(named: "myimage")! attributes.thumbnaildata = image.pngdata() attributes.contentdescription = "Subtitle" useractivity.contentattributeset = attributes viewcontroller.useractivity = useractivity

29 // How to donate a shortcut let useractivity = NSUserActivity(activityType: "com.myapp.name.my-activity-type") useractivity.iseligibleforsearch = true useractivity.iseligibleforprediction = true useractivity.title = "Activity" useractivity.userinfo = ["key": "value"] useractivity.suggestedinvocationphrase = "Let's do it" let attributes = CSSearchableItemAttributeSet(itemContentType: kuttypeitem as String) let image = UIImage(named: "myimage")! attributes.thumbnaildata = image.pngdata() attributes.contentdescription = "Subtitle" useractivity.contentattributeset = attributes viewcontroller.useractivity = useractivity

30 // How to handle a shortcut func application(_ application: UIApplication, continue useractivity: NSUserActivity, ([Any]?) -> Void) -> Bool { if useractivity.activitytype == "com.myapp.name.my-activity-type" { // Restore state for useractivity and userinfo } }

31 NSUserActivity

32 Intents

33

34 Messaging Workouts Climate and Radio Ride Booking Photo Search Lists Notebook Car Commands Payments VOIP Calling

35 Custom Built-In Messaging Workouts Climate and Radio Ride Booking Photo Search Lists Notebook Car Commands Payments VOIP Calling

36 Intents Define your shortcut Create an intent definition file in Xcode to customize a built-in SiriKit intent or define a new custom intent. Go to File -> New File, and choose SiriKit Intent Definition File.

37

38

39

40

41

42

43

44

45

46

47

48 public class Intent: INIntent { public var items: [INObject]? public var deliverylocation: CLPlacemark? } public protocol IntentHandling: NSObjectProtocol { public func handle(intent: Intent, (IntentResponse) -> Void) optional func confirm(intent: Intent, (IntentResponse) -> Void) }

49 Intents Define your shortcut Target selection Your intent definition should be added to any target where the intents are used If you're using a framework, only generate classes for the framework No Generated Classes No Generated Classes Intent Classes Intent Classes Intent Classes Framework No Framework

50 // Donate your shortcut let intent = PlaceOrderIntent() intent.items = order.items.map({ INObject(identifier: $0.id.uuidString, display: $0.menuItem.displayName) }) intent.deliverylocation = order.destinationlocation let interaction = INInteraction(intent: intent, response: nil) interaction.donate { error in // Handle error }

51 // Handle your shortcut func application(_ application: UIApplication, continue useractivity: NSUserActivity, ([Any]?) -> Void) -> Bool { if useractivity.activitytype == "Intent", let intent = useractivity.interaction?.intent as? Intent { // Show ordering UI, pre-populated with the fields from the intent } }

52

53 // Handle your shortcut class IntentHandler: INExtension, IntentHandling { func confirm(intent: Intent, completion: (IntentResponse) -> Void) { completion(intentresponse(code:.ready, useractivity: nil)) } func handle(intent: Intent, completion: (IntentResponse) -> Void) { // Order the soup completion(intentresponse(code:.success, useractivity: nil)) } }

54 Intent Handling Notes Implement an intents extension to run your shortcut in the background, without launching your app Always implement -continueuseractivity, even if you have an intents extension

55 INRelevantShortcut Expose Shortcuts to the Siri Watch Face by providing INRelevantShortcut objects Optionally, include relevance information as a hint as to when your shortcuts should be shown on the Watch Face Works even if you don t have a Watch app Siri Shortcuts on the Siri Watch Face Hall 3 Wednesday :00AM

56 Demo Adopting Shortcuts Willem Mattelaer, Siri

57 Soup Chef Made NSUserActivity eligible for prediction

58 Soup Chef Made NSUserActivity eligible for prediction Defined a custom intent

59 Soup Chef Made NSUserActivity eligible for prediction Defined a custom intent Donated the intent

60 Soup Chef Made NSUserActivity eligible for prediction Defined a custom intent Donated the intent Handled the intent

61 Optimizing for Suggestions

62 How is a Shortcut Suggested?

63 How is a Shortcut Suggested? Order Tomato Soup

64 How is a Shortcut Suggested? Order Tomato Soup

65 How is a Shortcut Suggested? Time

66 How is a Shortcut Suggested? Order tomato soup with croutons Time Monday, :57 AM

67 How is a Shortcut Suggested? Order tomato soup with croutons Order tomato soup with red pepper Monday, :57 AM Monday, 6:06 PM Time

68 How is a Shortcut Suggested? Order tomato soup with croutons Order tomato soup with red pepper Order tomato soup with croutons Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Time

69 How is a Shortcut Suggested? Order tomato soup Order tomato soup Order tomato soup with croutons with red pepper with croutons Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

70 How is a Shortcut Suggested? Order tomato soup with croutons Order tomato soup with red pepper Order tomato soup with croutons Order tomato soup with croutons Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

71 How is a Shortcut Suggested? NSUserActivity NSUserActivity.userInfo soup quantity scrollposition

72 How is a Shortcut Suggested? NSUserActivity Time

73 How is a Shortcut Suggested? NSUserActivity NSUserActivity 79.0 Time

74 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity Time

75 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity Time

76 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity Shortcut Suggestion Time

77 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity Shortcut Suggestion Time

78 How is a Shortcut Suggested? NSUserActivity requireduserinfokeys Minimal amount of information necessary for restoration

79 How is a Shortcut Suggested? NSUserActivity requireduserinfokeys Minimal amount of information necessary for restoration Used to find patterns

80 How is a Shortcut Suggested? NSUserActivity Time

81 How is a Shortcut Suggested? NSUserActivity NSUserActivity 79.0 Time

82 How is a Shortcut Suggested? NSUserActivity NSUserActivity Time

83 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity Time

84 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity Time

85 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity Shortcut Suggestion Time

86 How is a Shortcut Suggested? NSUserActivity NSUserActivity NSUserActivity NSUserActivity NSUserActivity Shortcut Suggestion Time

87 How is a Shortcut Suggested? Intents Shortcut Types

88 How is a Shortcut Suggested? Intents soup quantity options

89 How is a Shortcut Suggested? Intents soup quantity options soup soup soup quantity quantity options options Shortcut Types

90 How is a Shortcut Suggested? Intents Time

91 How is a Shortcut Suggested? Intents [ croutons ] Time Monday, :57 AM

92 How is a Shortcut Suggested? Intents [ croutons ] [ croutons ] Time Monday, :57 AM

93 How is a Shortcut Suggested? Intents [ red pepper ] [ croutons ] [ croutons ] Monday, :57 AM Monday, 6:06 PM Time

94 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ red pepper ] Monday, :57 AM Monday, 6:06 PM Time

95 How is a Shortcut Suggested? Intents [ croutons ] [ croutons ] [ red pepper ] [ croutons ] [ red pepper ] Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Time

96 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Time

97 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

98 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

99 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

100 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

101 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] [ red pepper ] [ croutons ] Shortcut Suggestion Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

102 How is a Shortcut Suggested? Intents [ croutons ] [ red pepper ] [ croutons ] [ croutons ] Shortcut Suggestion [ red pepper ] [ croutons ] [ croutons ] Monday, :57 AM Monday, 6:06 PM Tuesday, :59 AM Friday, 2:00 PM Time

103 What Makes a Good Donation? Likely to be repeated

104 What Makes a Good Donation? Likely to be repeated Ensure the payload you re donating is consistent across donations

105 What Makes a Good Donation? Likely to be repeated Ensure the payload you re donating is consistent across donations Don t include timestamps

106 What Makes a Good Donation? Show my meetings for June 5th Show my meetings for today

107 What Makes a Good Donation? Likely to be repeated Ensure the payload you re donating is consistent across donations Don t include timestamps Donate once and only once for each user action

108 Custom Intents Parameters Use enums when the values for a parameter are clearly bounded

109 Custom Intents Parameters Use enums when the values for a parameter are clearly bounded New Localization Workflows with Xcode 0 Executive Ballroom Wednesday 0:00AM

110 Custom Intents Parameters Custom results in an INObject

111 Custom Intents Parameters Custom results in an INObject Combines an identifier with a display string

112 Custom Intents Parameters Custom results in an INObject Combines an identifier with a display string Identifier can reference an internal object

113 Custom Intents Parameters Custom results in an INObject Combines an identifier with a display string Identifier can reference an internal object Display string conveys that object to the user

114 Custom Intents Parameters Custom results in an INObject Combines an identifier with a display string Identifier can reference an internal object Display string conveys that object to the user Prevents possible implicit dependencies between parameters

115 Custom Intents Parameters public class MyIntent: INIntent { public var objectidentifier: String? public var objectstring: String? }

116 Custom Intents Parameters public class MyIntent: INIntent { public var objectidentifier: String? public var objectstring: String? } public class MyIntent: INIntent { } public var object: INObject?

117 Custom Intents Parameters public class MyIntent: INIntent { public var objectidentifier: String? public var objectstring: String? } public class MyIntent: INIntent { } public var object: INObject?

118 Shortcut Display Titles, subtitles and images Understandable Represent what will happen Building for Voice with Siri Shortcuts Hall 2 Wednesday 0:00AM

119 Shortcut Testing

120 Shortcut Testing Developer settings on device

121 Shortcut Testing Developer settings on device Set up a shortcut with Siri and edit the Xcode scheme to test

122 Shortcut Testing Developer settings on device Set up a shortcut with Siri and edit the Xcode scheme to test

123 Shortcut Testing Developer settings on device Set up a shortcut with Siri and edit the Xcode scheme to test Create a custom shortcut in the Shortcuts app

124

125 Privacy

126 Deleting Donations User expectations Prevent suggesting content that s no longer relevant

127 Deleting Donations NSUserActivity Deleting user activities with corresponding Spotlight items

128 Deleting Donations NSUserActivity Deleting user activities with corresponding Spotlight items useractivity.contentattributeset?.relateduniqueidentifier = "my-identifier" let searchableindex = CSSearchableIndex.default() searchableindex.deletesearchableitems(withidentifiers: ["my-identifier"]) { (error) in }

129 Deleting Donations NSUserActivity Deleting user activities with corresponding Spotlight items useractivity.contentattributeset?.relateduniqueidentifier = "my-identifier" let searchableindex = CSSearchableIndex.default() searchableindex.deletesearchableitems(withidentifiers: ["my-identifier"]) { (error) in }

130 Deleting Donations NSUserActivity NEW Persistent identifier property on NSUserActivity Use the identifier to later delete specific NSUserActivities

131 // Deleting NSUserActivity using persistent identifier NEW let useractivity = NSUserActivity(activityType: "my-activity-type") // Configure user activity... let persistentidentifier = NSUserActivityPersistentIdentifier("persistent-identifier") useractivity.persistentidentifier = persistentidentifier NSUserActivity.deleteSavedUserActivities(withPersistentIdentifiers: [persistentidentifier]) { } NSUserActivity.deleteAllSavedUserActivities { }

132 // Deleting NSUserActivity using persistent identifier NEW let useractivity = NSUserActivity(activityType: "my-activity-type") // Configure user activity... let persistentidentifier = NSUserActivityPersistentIdentifier("persistent-identifier") useractivity.persistentidentifier = persistentidentifier NSUserActivity.deleteSavedUserActivities(withPersistentIdentifiers: [persistentidentifier]) { } NSUserActivity.deleteAllSavedUserActivities { }

133 // Deleting NSUserActivity using persistent identifier NEW let useractivity = NSUserActivity(activityType: "my-activity-type") // Configure user activity... let persistentidentifier = NSUserActivityPersistentIdentifier("persistent-identifier") useractivity.persistentidentifier = persistentidentifier NSUserActivity.deleteSavedUserActivities(withPersistentIdentifiers: [persistentidentifier]) { } NSUserActivity.deleteAllSavedUserActivities { }

134 Deleting Donations Intents Identifier and group identifier properties on INInteraction Use the identifier or group identifier to later delete one or more INInteractions

135 // Donating let interaction = INInteraction(intent: intent, response: response) interaction.identifier = "my-identifier" interaction.groupidentifier = "my-group-identifier" interaction.donate { (error) in } // Deleting the donation INInteraction.delete(with: ["my-identifier"]) { (error) in } INInteraction.delete(with: "my-group-identifier") { (error) in } INInteraction.deleteAll { (error) in }

136 // Donating let interaction = INInteraction(intent: intent, response: response) interaction.identifier = "my-identifier" interaction.groupidentifier = "my-group-identifier" interaction.donate { (error) in } // Deleting the donation INInteraction.delete(with: ["my-identifier"]) { (error) in } INInteraction.delete(with: "my-group-identifier") { (error) in } INInteraction.deleteAll { (error) in }

137 // Donating let interaction = INInteraction(intent: intent, response: response) interaction.identifier = "my-identifier" interaction.groupidentifier = "my-group-identifier" interaction.donate { (error) in } // Deleting the donation INInteraction.delete(with: ["my-identifier"]) { (error) in } INInteraction.delete(with: "my-group-identifier") { (error) in } INInteraction.deleteAll { (error) in }

138 // Donating let interaction = INInteraction(intent: intent, response: response) interaction.identifier = "my-identifier" interaction.groupidentifier = "my-group-identifier" interaction.donate { (error) in } // Deleting the donation INInteraction.delete(with: ["my-identifier"]) { (error) in } INInteraction.delete(with: "my-group-identifier") { (error) in } INInteraction.deleteAll { (error) in }

139 Media Shortcuts

140 Play Media Intent NEW INPlayMediaIntent

141 Play Media Intent NEW INPlayMediaIntent Launches app in background

142 Play Media Intent NEW INPlayMediaIntent Launches app in background Suggested in playback controls on Lock Screen

143 Play Media Intent NEW INPlayMediaIntent Launches app in background Suggested in playback controls on Lock Screen Works on HomePod

144 Upcoming Media Use INUpcomingMedia API to tell the system about new media the user hasn t yet consumed Great for podcasts, TV shows

145 Summary Enables powerful new experiences Provides new ways to engage users Adopt with NSUserActivity or Intents See Building for Voice with Siri Shortcuts

146 More Information Building for Voice with Siri Shortcuts Hall 2 Wednesday 0:00AM Siri Shortcuts Lab Technology Lab 7 Wednesday 2:00PM Siri Shortcuts Lab Technology Lab Thursday :00AM

147