UI Research. Vision Videos HCI and UI Research Topics. UI Research 1

Transcription

1 UI Research Vision Videos HCI and UI Research Topics UI Research 1

2 Envisioning the Future of Interaction Many examples of videos that provide visions of the future - Prediction of the home computer (1966) Apple "Knowledge Navigator" (1987) Sun Microsystems "Starfire",(1987) UI Research 2

3 Microsoft: Productivity Future Vision (2015) - (6:20) UI Research 3

4 Touch Gestures 0:53 2:13 2:30 4:45 UI Research 4

5 Pin-and-Cross - (4:00) UI Research 5

6 Mode Switching Techniques in Touch-based User Interfaces (2017) - (2:00) UI Research 6

7 Pen+Touch Interaction UI Research 7

8 Conté: Multimodal Input Inspired by an Artist's Crayon - UI Research 8

9 Mid-air Bare-hand Input 0:15 0:35 4:55 UI Research 9

10 Myopoint: Pointing and Clicking Using Forearm Mounted EMG and IMU (2015) - (2:30) UI Research 10

11 Gunslinger: Subtle Arms-down Mid-air Interaction (2015) - (3:00) UI Research 11

12 Wearables 3:10 UI Research 3:46 12

13 Doppio: A Reconfigurable Dual-Face Smartwatch for Tangible Interaction (2016) - (5:00) UI Research 13

14 Cito: An Actuated Smartwatch for Extended Interactions (2017) - (5:00) UI Research 14

15 Mixed Reality and Augmented Reality 4:25 4:34 5:08 5:27 UI Research 15

16 IllumiRoom (2013) - UI Research 16

17 HoloDesk: Direct 3D Interactions with a Situated See-Through Display (2011) - (4:00) UI Research 17

18 UltraHaptics: Multi-Point Mid-Air Haptic Feedback for Touch Surfaces - (3:40) UI Research 18

19 inform - Interacting With a Dynamic Shape Display (2013) - (3:30) UI Research 19

20 UI Research 20

21 Rovables: Miniature On-Body Robots as Mobile Wearables (2016) - (0:30) UI Research 21

22 Final Exam Review Winter 2018 Final Exam Review 22

23 What to Expect April 12, 4:00 to 6:30 PM (2.5 hours) No aids allowed (i.e. no calculators, books) Scope: The entire course, focusing on lecture material. The final exam is comprehensive and covers everything. Format: Question and Answer - Short to medium answers - Concepts - Problem-solving - Short coding (reading existing code, writing small amounts of pseudocode, NOT writing syntactically correct or complex code). Final Exam Review 23

24 Introduction History Windowing Systems Drawing Events Java Basics Java GUI Widgets Event Dispatch Event Handling Layout Graphics Transformations Graphics Hit-testing Model-View-Control (MVC) Input Devices Input Performance Direct Manipulation Undo Clipboard, Drag-and-Drop Mobile UI Android Basics Android UI Development Multi-view Applications User-Centered Design Design Process Visual Perception Visual Design Responsiveness Accessibility Final Exam Review 24

25 Introduction What is a User Interface? is the place where a person expresses intention to an artifact, and the artifact presents feedback to the person. What is the difference between an interface and an interaction? External presentation vs. actions by user and system over time Final Exam Review 25

26 History The history of interaction - Batch interfaces - Conversational interfaces - Graphical interfaces What are the characteristics of a GUI interface? What is a WIMP interface? Four visionaries - Vannevar Bush: As We May Think, Memex - Ivan Sutherland: Sketchpad, Lightpen, DM - Douglas Engelbart (On-Line System): Mouse, Hypertext, Copy/Paste - Alan Kay: Xerox Star, Dynabook Final Exam Review 26

27 Windowing Systems Design criteria of X Windows - Display and device independent - Network transparent - Support multiple concurrent displays - Support overlapping windows, 2D graphics, imaging What is a X Client? X Server? What led to this design? Base Window System, Window Manager Final Exam Review 27

28 Drawing Three conceptual models for drawing: - Pixel, Stroke, Region Clipping Depth and ordering (Painter s Algorithm) Double-buffering Final Exam Review 28

29 Events What is Event-driven programming? - Nothing happens unless the something else happens: What is Events? - user presses a key, moves the mouse - window is resized, closed, covered - certain time passes Receiving Events: - XNextEvent vs. Xpending Animation - A simulation of movement created by displaying a series of pictures, or frames. - Repaint times per second ( FPS ) - Don t use more CPU than necessary Final Exam Review 29

30 Java Basics & Java GUI (Almost) Everything is a Class Inheritance Interfaces Adapter Swing Component Hierarchy Graphics and Painting - Each component has a paintcomponent() method, which describes how it paints itself. Final Exam Review 30

31 Widget Widget is a generic name for parts of an interface that have their own behavior (e.g. buttons, progress bars) Capture user input, send and receive events Simple vs. container widgets, special value widgets A logical input device is the essence of what a widget does, its function - a widget is a logical device with an appearance Logical button device: pushed button Logical number device: slider MVC also applies at the widget level, Since widgets can have their own state, events, and appearance. Final Exam Review 31

32 Event Dispatch Widgets are often the target of events - Laid out in a hierarchy (interactor tree). - Lightweight vs. heavyweight widgets Which widget receives an event? - Positional dispatch: top-down vs. bottom-up - Focus dispatch: key, mouse What are some problems with positional-dispatch (e.g. scrollbar)? How does focus dispatch help address these limitations? Final Exam Review 32

33 Event Binding After dispatch to a widget, how do we bind an event to code? Design Goals - Easy to understand (clear connection between event/code) - Easy to implement (binding paradigm or API) - Easy to debug (how did this event get here?) Good performance Mechanisms - Event loop + Switch statement manual binding - Inheritance binding - Listener Interface binding - Listener Adapter binding - Final Exam Review 33

34 Layout Two Interface Layout Tasks Designing a spatial layout of widgets in a container Adjusting that spatial layout when container is resized A Layout Manager provides a layout algorithm to size and position widgets (uses widgets preferred, min and max size) Layout strategies (Java layouts) Fixed layout (null) Intrinsic size (box, flow) Variable intrinsic size (border) Struts and springs (spring, box) Final Exam Review 34

35 Graphics Transformation How do we represent and draw primitives? Shape models (array of points, location, bounds, ) Homogeneous Coordinates add an extra component w to each coordinate How do we manipulate shape models? Affine transformations! Rotation, Scaling, Translation Represent combined transformations using matrix notation Can combine successive operations but order matters! Final Exam Review 35

36 Graphics Hit-testing Line Segment Hit-test - a line model has no thickness - pick a threshold distance from mouse position to line - point to line distance can be computed using vector projection Polyline and Multiple Polyline Hit-testing Rectangle Shape Hit-Test - rectangle shape useful as a bounding box A point inside or outside a polygon? Final Exam Review 36

37 Model-View-Controller (MVC) Why do we need Model-View-Controller (MVC)? Need multiple views of the same document The UI will change over time (e.g. introduce mobile app). Supports code separation and reuse, easy testing. How do Model, View and Controller communicate? The View and Controller are often tightly coupled (i.e. view class contains both). Why? Final Exam Review 37

38 Input Devices Dimensions to Classify Computer Input - Sensing Method, Continuous vs. Discrete, Degrees of Freedom (DOF) What are the tradeoffs between using specific vs general input devices? What considerations went into the design of the QWERTY keyboard? What are the perceived Qwerty problems? Virtual Keyboards, Chording Keyboards, Predictive Text Properties of Positional Input Devices (Absolute vs. Relative Positioning, Direct vs. Indirect Contact, Position vs. Rate Control) What is control-display gain (CD gain)? Why might we adjust it? Final Exam Review 38

39 Input Performance Apply the Keystroke Level Model (KLM) to a particular user interface task (You don t need to memorize the time for K, P, B, H, and M; these will be provided). Apply Fitts Law to a particular target acquisition problem. MT = a + blog 2 (D/W +1) According to Fitts Law, what is better? - menu at the top of the screen versus application window - context versus pie menus - OS X dock expansion versus slowing mouse cursor near target What is the steering law, and how is it relevant to input performance in applications? Final Exam Review 39

40 Direct Manipulation Direct manipulation allows people to feel that they are directly controlling the objects represented by the computer Instrumental Interaction: An interaction model based on how we use tools to manipulate objects in the physical world. - Domain objects (the thing of interest, e.g., data/attributes) vs. Interaction instrument (a mediator between the user and domain objects). - WIMP instruments activated spatially or temporally. Evaluating instruments - Degree of indirection: Spatial/temporal offset between instrument/action - Degree of integration: Ratio of DOF of instrument to DOF of input device: Controlling scroll bar (1DOF) with Mouse(2DOF) = 1/2 - Degree of compatibility: Similarity of action on control device to action on object Final Exam Review 40

41 Clipboard, Drag-and-Drop Methods to enable user-interface level data transfer within an application and between applications - clipboard (copy, cut, paste) - drag-and-drop (drag data from one view/application to another) Data formats - Text, graphics, Final Exam Review 42

42 Mobile UI Device Characteristics - Limited resources - Primarily touch interaction - Mobile form factor Human Accuracy, fat fingers issue, No Hover State in Touch Device Characteristics: Interaction - one app in the foreground - Each app has window that fills the entire screen Android Design Guidelines and tips - Enchant Me - Simplify My Life - Help Users to Enter Information - Final Exam Review 43

43 Android Basics Activities, activity lifecycle Intents - use to request an action from another app component - use intents to pass data between activities use intents to pass data between activities. Views and Viewgroups Pixel density and dp Layout Units - dp = (width in pixels * 160) / screen density Final Exam Review 44

44 Android UI Development and Multi-view Applications Manifest, App Resources, Layouts Linear Layout - Orientation - Fill model - Weight - gravity - Padding/margin Relative Layout - android:layout_alignparenttop - android:layout_centervertical - android:layout_below - android:layout_torightof - MVC in Android Final Exam Review 45

45 User-Centred Design User Mental Model, System Model, (Developer model) Norman s Model of Interaction - Execution: What we do to the system to achieve a goal - Evaluation: Comparing what happened with our intended goal Gulf of Execution and Gulf of Evaluation - Gulf of Execution: Difficulty translating user s intentions into actions allowed by system. Can user carry out their intentions? - Gulf of evaluation: Difficulty in interpreting the state of the system to determine whether our goal has been met. UI Design Principles - Perceived Affordance, Mapping, Constraints, Visibility/Feedback, Metaphor Final Exam Review 46

46 Design Process Interaction Sequences and Interface Schematics Prototype Fidelity - Low fidelity: prototype doesn t look much like final product. - High fidelity: prototype looks and operates like the real product Paper prototyping Wizard of Oz prototypes Breadth vs. Depth Final Exam Review 47

47 Visual Perception Psychophysics: out there vs. in here Temporal Resolution Critical Flicker Frequency: perception of continuous motion Spatial Resolution Visual acuity, pixel density (Retina displays) Color Perception Additive (RGB, HSV) and Subtractive (CMY) color models Cones vs. rods Color blindness Color Presentation Hard to tell colors apart when pale, small/thin, far apart Final Exam Review 48

48 Visual Design Gestalt Principles: Grouping - Proximity, - Similarity - Continuity - Closure - Connectedness Creating Organization and Structure - Visual Design - Grouping - Hierarchy - Relationship - Balance Typography, Type Styles and Anatomy How to achieve simplicity? Final Exam Review 49

49 Responsiveness We can make feedback responsive in two ways: - loading data efficiently so it s available quickly - designing for human perception of time 1. Design the UI to meet human deadline requirements Systems can feel responsive by providing feedback. Poor responsiveness: ignore user input, no clues on duration. Deadlines cause and effect: 0.1 second visual-motor reaction time: 1 second Tricks Use Progress vs. Busy indicators Render / display important information first Fake heavyweight computations during tasks 2. Load data efficiently so that it s available quickly Final Exam Review 50

50 Accessibility People vary in their physical and mental capabilities - The average person is just a statistical ideal Temporary / situational disabilities - Sickness, Underwater diving, Walking while using a smartphone Aging - Reduced motor coordination (fine/gross motor skills), Visual and hearing impairments, Loss of memory Inclusive technologies to address impairments. - What technologies can we design to address these issues? - Visual, Hearing, Motor, Cognitive What is the Curb Cut phenomenon? Name a few examples. Final Exam Review 51

51 Final Words KISS - Keep it simple and succinct - Our goal is not to confuse. Our goal is not to require long answers that we need to spend extra time deciphering. Neatness is your friend - Messy answers can cost you if TAs cannot decipher your answers. If you make a mess, use circle and callouts to clearly indicate what you want graded Read the questions and answer accurately - Give two examples means we grade *the first two* or *the two highlighted* examples. Others are ignored Don t write on the back of the exam! Final Exam Review 52