Modern User Interaction on ios

Transcription

1 App Frameworks #WWDC17 Modern User Interaction on ios Mastering the UIKit UIGesture System Session 219 Dominik Wagner, UIKit Engineer Michael Turner, UIKit Engineer Glen Low, UIKit Engineer 2017 Apple Inc. All rights reserved. Redistribution or public display not permitted without written permission from Apple.

2 Multi-Touch

3 The UIGesture system System gesture interaction Playing nice with Drag and Drop

4 The UIGesture system System gesture interaction Playing nice with Drag and Drop

5 The UIGesture system System gesture interaction Playing nice with Drag and Drop

6 Basics

7 Basics UITouch UIGesture

8

9

10

11 Touch Handling

12 Tap Gesture Touch Handling

13 Tap Gesture Touch Handling

14 UITouch Tap Gesture Touch Handling

15 UITouch Tap Gesture Touch Handling

16 UITouch Tap Gesture Touch Handling

17 possible UITouch Tap Gesture Touch Handling

18 possible UITouch Tap Gesture Touch Handling

19 possible UITouch Tap Gesture Touch Handling ended

20 possible UITouch Tap Gesture Touch Handling ended ended

21 ended UITouch Tap Gesture Touch Handling ended ended

22 ended UITouch Tap Gesture Touch Handling ended ended

23 ended UITouch Tap Gesture Touch Handling ended ended cancelled

24 // Influencing responder based touch handling class UIGesture : NSObject { } open var delaystouchesended: Bool // default is true. open var cancelstouchesin: Bool // default is true. open var delaystouchesbegan: Bool // default is false.

25 // Influencing responder based touch handling class UIGesture : NSObject { } open var delaystouchesended: Bool // default is true. open var cancelstouchesin: Bool // default is true. open var delaystouchesbegan: Bool // default is false.

26 Gesture s first

27 Tap Gesture Touch Handling

28 Pan Gesture Tap Gesture Touch Handling

29 Pan Gesture Tap Gesture Touch Handling

30 UITouch Pan Gesture Tap Gesture Touch Handling

31 possible possible UITouch Pan Gesture Tap Gesture Touch Handling

32 possible possible UITouch Pan Gesture Tap Gesture Touch Handling

33 possible possible UITouch Pan Gesture Tap Gesture Touch Handling moved

34 possible possible UITouch Pan Gesture Tap Gesture Touch Handling moved moved

35 possible UITouch Pan Gesture Tap Gesture Touch Handling moved moved

36 possible UITouch Pan Gesture Tap Gesture Touch Handling moved moved moved

37 failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved moved

38 failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved moved

39 failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved moved cancelled

40 failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended

41 failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended ended

42 ended failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended ended

43 ended failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended ended ended

44 ended failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended ended

45 ended failed UITouch Pan Gesture Tap Gesture Touch Handling moved moved cancelled ended ended

46 Exclusion

47 // Influencing exclusion public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldrecognizesimultaneouslywith othergesture: UIGesture) -> Bool open class UIGesture : NSObject { } open func canprevent(_ preventedgesture: UIGesture) -> Bool open func canbeprevented(by preventinggesture: UIGesture) -> Bool

48 // Influencing exclusion public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldrecognizesimultaneouslywith othergesture: UIGesture) -> Bool open class UIGesture : NSObject { } open func canprevent(_ preventedgesture: UIGesture) -> Bool open func canbeprevented(by preventinggesture: UIGesture) -> Bool

49 // Influencing exclusion public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldrecognizesimultaneouslywith othergesture: UIGesture) -> Bool open class UIGesture : NSObject { } open func canprevent(_ preventedgesture: UIGesture) -> Bool open func canbeprevented(by preventinggesture: UIGesture) -> Bool

50 Pan Gesture Tap Gesture

51 UITouch Pan Gesture Tap Gesture

52 possible possible UITouch Pan Gesture Tap Gesture

53 possible possible UITouch Pan Gesture Tap Gesture moved

54 possible possible UITouch Pan Gesture Tap Gesture moved moved

55 possible UITouch Pan Gesture Tap Gesture moved moved

56 possible UITouch Pan Gesture Tap Gesture moved moved moved

57 gesture(_, shouldrecognizesimultaneously:) -> true possible UITouch Pan Gesture Tap Gesture moved moved moved

58 gesture(_, shouldrecognizesimultaneously:) -> true possible UITouch Pan Gesture Tap Gesture moved moved moved

59 gesture(_, shouldrecognizesimultaneously:) -> true possible UITouch Pan Gesture Tap Gesture moved moved ended

60 gesture(_, shouldrecognizesimultaneously:) -> true possible UITouch Pan Gesture Tap Gesture moved moved ended ended

61 gesture(_, shouldrecognizesimultaneously:) -> true ended possible UITouch Pan Gesture Tap Gesture moved moved ended ended

62 gesture(_, shouldrecognizesimultaneously:) -> true ended possible UITouch Pan Gesture Tap Gesture moved moved ended ended ended

63 gesture(_, shouldrecognizesimultaneously:) -> true ended ended UITouch Pan Gesture Tap Gesture moved moved ended ended ended

64 gesture(_, shouldrecognizesimultaneously:) -> true ended ended UITouch Pan Gesture Tap Gesture moved moved ended ended ended

65 gesture(_, shouldrecognizesimultaneously:) -> true ended ended UITouch Pan Gesture Tap Gesture moved moved ended ended ended

66 Failure Requirements

67 // Failure Requirements class UIGesture : NSObject { } open func require(tofail othergesture: UIGesture) public protocol UIGestureDelegate : NSObjectProtocol { optional public func gesture(_ gesture: UIGesture, shouldrequirefailureof othergesture: UIGesture) -> Bool } optional public func gesture(_ gesture: UIGesture, shouldberequiredtofailby othergesture: UIGesture) -> Bool // Subclasses open class UIGesture : NSObject { } open func shouldrequirefailure(of othergesture: UIGesture) -> Bool open func shouldberequiredtofail(by othergesture: UIGesture) -> Bool

68 // Failure Requirements class UIGesture : NSObject { } open func require(tofail othergesture: UIGesture) public protocol UIGestureDelegate : NSObjectProtocol { optional public func gesture(_ gesture: UIGesture, shouldrequirefailureof othergesture: UIGesture) -> Bool } optional public func gesture(_ gesture: UIGesture, shouldberequiredtofailby othergesture: UIGesture) -> Bool // Subclasses open class UIGesture : NSObject { } open func shouldrequirefailure(of othergesture: UIGesture) -> Bool open func shouldberequiredtofail(by othergesture: UIGesture) -> Bool

69 // Failure Requirements class UIGesture : NSObject { } open func require(tofail othergesture: UIGesture) public protocol UIGestureDelegate : NSObjectProtocol { optional public func gesture(_ gesture: UIGesture, shouldrequirefailureof othergesture: UIGesture) -> Bool } optional public func gesture(_ gesture: UIGesture, shouldberequiredtofailby othergesture: UIGesture) -> Bool // Subclasses open class UIGesture : NSObject { } open func shouldrequirefailure(of othergesture: UIGesture) -> Bool open func shouldberequiredtofail(by othergesture: UIGesture) -> Bool

70 // Failure Requirements class UIGesture : NSObject { } open func require(tofail othergesture: UIGesture) public protocol UIGestureDelegate : NSObjectProtocol { optional public func gesture(_ gesture: UIGesture, shouldrequirefailureof othergesture: UIGesture) -> Bool } optional public func gesture(_ gesture: UIGesture, shouldberequiredtofailby othergesture: UIGesture) -> Bool // Subclasses open class UIGesture : NSObject { } open func shouldrequirefailure(of othergesture: UIGesture) -> Bool open func shouldberequiredtofail(by othergesture: UIGesture) -> Bool

71 Pan Gesture require(tofail:) Tap Gesture

72 UITouch Pan Gesture require(tofail:) Tap Gesture

73 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible

74 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved possible

75 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible

76 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved

77 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed

78 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed possible

79 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible

80 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible ended

81 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible ended ended

82 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible ended ended ended

83 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible ended ended no action ended

84 UITouch Pan Gesture require(tofail:) Tap Gesture possible possible moved no action possible moved changed no action possible ended ended no action ended send action

85

86 163 UIGestures

87

88

89 7 UIGestures

90 Hit Testing

91 Hit Testing

92 Hit Testing Window

93 Hit Testing Window

94 Hit Testing

95 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

96 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

97 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

98 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

99 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

100 // UI subclasses open class UI : NSObject { } open func hittest(_ point: CGPoint, with event: UIEvent?) -> UI? open func point(inside point: CGPoint, with event: UIEvent?) -> Bool class UI : NSObject { var isuserinteractionenabled: Bool var alpha: CGFloat var ishidden: Bool } var ismultipletouchenabled: Bool public struct UIAnimationOptions : OptionSet { } public static var allowuserinteraction: UIAnimationOptions { get }

101 Hit Testing and Animations Presentation layer vs model layer UIPropertyAnimator and ismanualhittestingenabled Building Interruptible and Responsive Interactions WWDC 2014 Advances in UIKit Animations and Transitions WWDC 2016

102 Relevant Gesture s Window

103 Relevant Gesture s Window

104 Relevant Gesture s Window

105 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

106 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

107 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

108 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

109 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

110 // Influencing participation in the interaction public protocol UIGestureDelegate : NSObjectProtocol { } optional public func gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool optional public func gestureshouldbegin(_ gesture: UIGesture) -> Bool class UIGesture : NSObject { var isenabled: Bool } var allowedtouchtypes: [NSNumber] var requiresexclusivetouchtype: Bool

111 New on UIGesture NEW Debugging 11.0, *) open var name: String? // name for debugging to appear in logging

112 Debugging Breakpoint opportunities gesture(_ gesture: UIGesture, shouldreceive touch: UITouch) -> Bool touchesbegan(_ touches: Set<UITouch>, with event: UIEvent?) Things to inspect po touches.first?.gestures po event?.touches(for:somegesture) po touches.first?.view?.superview?.gestures

113 Custom UIGestures Begin late and fail fast! Ignore touches ignore(_ touch:, for event:) Don t forget touchescancelled(_:with:)

114 Gesture System

115 Gesture System Revisit your setups

116 Gesture System Revisit your setups Exclusion and failure requirements

117 Gesture System Revisit your setups Exclusion and failure requirements Are your gesture recognizers on the right views?

118 System Gesture Interaction Glen Low, UIKit Engineer

119

120 Cover Sheet or Killjoy?

121 Life on the Edges

122 Life on the Edges Multitasking / Dock

123 Life on the Edges Slide Over Multitasking / Dock

124 Life on the Edges Cover Sheet Slide Over Multitasking / Dock

125 Life on the Edges Cover Sheet Gesture s and Responders Slide Over Multitasking / Dock

126 Who Gets the Touches? System App Tap Pinch Rotate Long Press

127 Who Gets the Touches? System App System App Tap Pinch Rotate Long Press Pan Swipe Responders

128 System gesture or app gesture? We made an educated guess.

129 NEW System gesture or app gesture? You tell us.

130 Let s Talk Terms NEW class MyController: UIController { // override to return which screen edges to defer system gestures override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { return defercontrolcenter?.bottom : UIRectEdge() } // call whenever your method would return a different screen edge var defercontrolcenter : Bool { didset { setneedsupdateofscreenedgesdeferringsystemgestures() } } }

131 Let s Talk Terms NEW class MyController: UIController { // override to return which screen edges to defer system gestures override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { return defercontrolcenter?.bottom : UIRectEdge() } // call whenever your method would return a different screen edge var defercontrolcenter : Bool { didset { setneedsupdateofscreenedgesdeferringsystemgestures() } } }

132 Let s Talk Terms NEW class MyController: UIController { // override to return which screen edges to defer system gestures override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { return defercontrolcenter?.bottom : UIRectEdge() } // call whenever your method would return a different screen edge var defercontrolcenter : Bool { didset { setneedsupdateofscreenedgesdeferringsystemgestures() } } }

133 Speaking in Tongues Gesture s and Responders override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { } return defercontrolcenter?.bottom : UIRectEdge()

134 Speaking in Tongues Gesture s and Responders override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { } return defercontrolcenter?.bottom : UIRectEdge()

135 Speaking in Tongues Gesture s and Responders Multitasking / Dock override func preferredscreenedgesdeferringsystemgestures() -> UIRectEdge { } return defercontrolcenter?.bottom : UIRectEdge()

136 For Containers Only NEW class MyContainerController: UIController { // override to return which child view controller to consult override func childcontrollerforscreenedgesdeferringsystemgestures() -> UIController? { return myselectedchildcontroller } }

137 Don t Do It, Because Don t mess with the familiar Your recognizers may already get the touches Telling us us doing it

138 Don t Do It, Because Don t mess with the familiar Your recognizers may already get the touches Telling us us doing it Your app is only used casually

139 Playing Nice With Drag and Drop Michael Turner, UIKit Engineer

140 UIDragInteraction NEW Adding UIDragInteraction to a UI within your app is easy let draginteraction = UIDragInteraction(delegate: mydelegate) my.addinteraction(draginteraction)

141 Initiating a Drag

142 Initiating a Drag

143 UIDragInteraction Example UI var interactions UIDragInteraction var gestures UIGesture UIGesture UIGesture UI

144 UIDragInteraction Example UI var interactions UIDragInteraction var gestures UIGesture UIGesture UIGesture UILongPressGesture UIController func longpress(gesture: UIGesture) { switch gesture.state { case.: presentactivitycontroller() case.cancelled: dismissactivitycontroller() default: break } } UI

145 Long Press and Move

146 UILongPressGestures are delayed

147 Long Press, Hold, and Move

148 A beginning drag will cancel the touch

149 Compact Trait Environment

150 Long Press, Hold, and Move

151 Long presses are delayed until the touch ends

152 Long Press and Lift

153 Adding to a Drag

154 Adding to a Drag Control this behavior using UIDragInteractionDelegate func draginteraction(_ interaction: UIDragInteraction, itemsforaddingto session: UIDragSession, withtouchat point: CGPoint) -> [UIDragItem] { // Returning 0 items allows normal touch processing to continue return [] }

155 How to Adapt for UIDragInteraction Examine your existing actions Present modal UI carefully Handle the.cancelled state Your app is interactive during a drag!

156 Summary Leveraging the gesture system Use the new deferred system gesture APIs Working alongside UIDragInteraction

157 Summary Leveraging the gesture system Use the new deferred system gesture APIs Working alongside UIDragInteraction

158 Summary Leveraging the gesture system Use the new deferred system gesture APIs Working alongside UIDragInteraction

159 More Information

160 Past Sessions Building Advanced Gesture s WWDC 2012 Making the Most of Multi-Touch on ios WWDC 2011 Leveraging Touch Input on ios WWDC 2016 Building Interruptible and Responsive Interactions WWDC 2014 Advanced Scrollviews and Touch Handling Techniques WWDC 2014

161 Related Sessions Introducing Drag and Drop Hall 3 Tuesday 11:20AM What s New in Cocoa Touch Hall 3 Tuesday 10:20AM Updating Your App for ios 11 Hall 3 Tuesday 4:10PM Mastering Drag and Drop Grand Ballroom B Wednesday 11:00AM Advanced Animations with UIKit Hall 3 Thursday 3:10PM

162 Labs Cocoa Touch and Haptics Lab Technology Lab C Fri 12:00PM UIKit and Collection Lab Technology Lab B Thu 10:00AM

163