LECTURE 6, INTERACTION STYLES

Size: px

Start display at page:

Download "LECTURE 6, INTERACTION STYLES"

Frederica Garrison
6 years ago
Views:

1 September LECTURE 6, INTERACTION STYLES HCI &b InfoVis 2017, fjv 1

2 Recapitulation Lecture #5 Computer elements Interaction devices Focus on input Paradigms, Shifts From Batch To Ubiquitous Interaction models Norman Abowd & Beale HCI &b InfoVis 2017, fjv 2

3 Abowd & Beale vs. Norman Norman 7 stages: Helps understanding interaction Abowd & Beale: Translation between languages (user-input-core-output) 4 translations Articulation (1), Performance (2) Presentation (3), Observation (4) S core 3 O output 4 2 I 1 input U task Simple: Virtual Reality, articulaton is direct Complex: Control panels, e.g. Lecture Room lighting HCI &b InfoVis 2017, fjv 3

4 Example Haptic IO HCI &b InfoVis , fjv

5 Example Haptic IO HCI &b InfoVis 2017, fjv 5

6 Focus on Haptic Feedback Epidural Simulator from Yantric Haptic IO-device mounted in mannequin Interfaced with 3D models Feedback Visibility Mental model (building of) Mental model is built from the combination of haptic force feedback and 3D visualization Metaphor is very direct, i.e. a patient HCI &b InfoVis 2017, fjv 6

7 Metaphors The book metaphor Strength and weakness The desktop metaphor History and examples The internet metaphor Strength and weakness The spatial metaphor In development, examples The social metaphor In development, example HCI &b InfoVis 2017, fjv 7

8 Spatial Metaphor Humans are adapted to 3D space Spatial metaphor results in natural interaction Access functionality by acting in realistic space The normal desktop (2.5D) Gaming (3D virtual world: e.g. Quake, 2 nd life) Issue in information organization Virtual space same problems a Real Space Use related problems in improvement of technology HCI &b InfoVis 2017, fjv 8

9 Beyond the desktop Robertson, G. et al. "The Task Gallery: A 3D Window Manager." In Proceedings of CHI 2000 HCI &b InfoVis 2017, fjv 9

10 3D interactions Introduction of new metaphor's gallery, rooms, palette (maze) result of successful use of desktop-window interface; HCI &b InfoVis 2017, fjv 10

11 3D Navigation How well is the mapping of the keyboard realized? A modern approach to space navigation HCI &b InfoVis 2017, fjv 11

12 Spatial Metaphors 3D interaction in the (G)UI is often thought about and implementations are now finding their way to applications and users. Jared White: A real 3D interface may be many years from fruition, but use of computers will be unnecessarily restricted in both cognitive and functional sense, and also in the social sense as long as we are living in a flat, two-dimensional virtual world., August 2004, adapted by fjv. HCI &b InfoVis 2017, fjv 12

13 3D Interaction & Spatial Metaphor Win3D, ClockWise Microsoft Windows desktop See examples Looking Glass Project, SUN Microsystems Linux and Solaris desktop 2004, See examples Croquet Project What if.., desktop independent Open Source 2004, Cf. groupware HCI &b InfoVis 2017, fjv 13

14 Modernized 3D desktop Metaphor Simplex Continuously updated Office zone Multi-media zone Games zone Kids World Play No learning curve Fun Not immersive HCI &b InfoVis 2017, fjv 14

15 HCI &b InfoVis 2017, fjv 15

16 3D DeskTop Project Looking Glass Translucent windows! See what you are doing where Building-Rooms paradigm Interact with 3D objects Multi media driven Object~Action mental model Modern approach to objects See also MacOS-X interface HCI &b InfoVis 2017, fjv 16

17 HCI &b InfoVis 2017, fjv 17

18 HCI &b InfoVis 2017, fjv 18

19 3D Interfaces HCI &b InfoVis 2017, fjv 19

20 3D Interface, includes spatial metaphor HCI &b InfoVis 2017, fjv 20

21 Other Spatial interactions... Make interactive objects physical... HCI &b InfoVis 2017, fjv 21

22 Spatial (3D) Interaction Increasingly present in a lot of interfaces itunes ipod 3D and pseudo 3D interaction Interaction with objects. Wii Leap Motion Kinect Myo Hypertext suggest spatial relations Relates to basic HCI metaphores HCI &b InfoVis 2017, fjv 22

23 Social Metaphor Modeling interface to rules and forms of social behavior Related to spatial metaphor (environment) No concrete object/concept Not functionality but Style/Form Social agents Social Interaction is often collaborative Human Human Human Artificial agents Public virtual spaces, or Digital Hangout Relates to basic HCI metaphors HCI &b InfoVis 2017, fjv 23

24 Social Metaphor 2nd life, a gathering of avatars a gathering of identities HCI &b InfoVis 2017, fjv 24

25 Mismatched Metaphors (1) From the Hall of Shame The use of the trashcan to eject the floppy disk? The trashcan on the desktop? HCI &b InfoVis 2017, fjv 25

26 Mismatched Metaphors (2) From the Hall of Shame VCR buttons to control a printer? HCI &b InfoVis 2017, fjv 26

27 Basic Level InfoVis Metaphores Direct Manipulation Metaphore: DATA are Physical OBJECTS Navigation Metaphor DATA is in SPACE Distances have a meaning. Human Interaction Metaphore: COMPUTERS are PEOPLE Connect with People through Computer networks HCI &b InfoVis 2017, fjv 27

28 The Metaphor for the ipad? What is your metaphor for your ipad? My metaphor is brain (external) My tablet metaphor would be like Jamie's Torch. Left it in office once, I was bereft, daughter said its like I'd left her there! Mobile shopper ;-) Definitely "life partner" ;-) A scrapbook It s my daily computer. Once or twice a week I ll unsheath the MacBook Air for something specific Mine would be a window into the world or a bridge to communicate with others via social networks HCI &b InfoVis 2017, fjv 28

29 Users & Designers Metaphors Graphics Icons Language Documentation Tutorials HCI &b InfoVis 2017, fjv 29

30 Features of Systems Image = UI Familiar to users (see lecture 4) Matches way they think about domain Preferably based on a concrete metaphor Coherent Supports learning by exploration Hides system model (transparency) Reflects current status - changes are notified Supports retention (memorability) Reduces need for training HCI &b InfoVis 2017, fjv 30

31 Therefore Sloan s Law: The changes in new designs have to be dramatically attractive so that unease about the former design will develop This law is about a developing a satisfying Systems Image! HCI &b InfoVis 2017, fjv 31

32 INTERACTION STYLES HCI &b InfoVis 2017, fjv 32

33 Interaction Techniques An interaction technique is a way of using a physical input device to perform a generic task in a human-computer dialogue. Foley et al. [1990] technical definition : An interaction technique is a way to carry out an interactive task. It is defined in the binding, sequencing, and functional levels, and it is based on using a set of input and output devices or technologies. HCI &b InfoVis 2017, fjv 33

Five Primary Interaction Styles Ben Shneiderman 1998 Direct manipulation Menu selection Form filling Command language Natural language These are all in response to the baseline of

34 Five Primary Interaction Styles Ben Shneiderman 1998 Direct manipulation Menu selection Form filling Command language Natural language These are all in response to the baseline of command languages All types not appropriate for all tasks Often useful to blend interactions styles in one UI We restrict to standard interaction (on a desktop) HCI &b InfoVis 2017, fjv 34

35 Interaction Styles & Categories Describe form of communication between user and computer Linguistic styles Command line (natural) Speech Key-Modal styles Function keys Q & A interaction Menu driven Form Fill-in Direct manipulation GUI Forms (Form Fill-in) HCI &b InfoVis 2017, fjv 35

36 Command Language Oldest style of interaction Instructions are expressed to computer directly Parsing or Direct Execution Communication purely textual Options: Single characters Whole words Abbreviations (e.g. cd, ftp) Function keys Combinations of the above HCI &b InfoVis 2017, fjv 36

37 Example Command Line Interface HCI &b InfoVis 2017, fjv 37

38 Issues in Command Language Issues: # of keystrokes required to complete command burden on (operator) memory Main characteristics: grammar required limited set of commands subset of natural language syntax often of form action - object HCI &b InfoVis 2017, fjv 38

39 Guidelines Command Language Meaningful commands & command names based on task analysis Meaningful command use consistent abbreviations Use consistent CL format (syntax) Follow de facto standards Provide way to execute sets of commands batch processes, Pipelines Provide online help, feedback, undo, replay, etc Redundancy is okay (provides customizability) Use extra options to alter behavior slightly reduce memory load e.g unix -r option for 10 commands (11 instead of 20) HCI &b InfoVis 2017, fjv 39

40 Command Language From literature: flexible Advantages appeals to power users allows user initiative allows convenient creation of macros cheap Disadvantages low visibility requires substantial training and memorization not useful for novice users poor error handling (relative) HCI &b InfoVis 2017, fjv 40

41 Text-based Natural Language Alan Turing: Turing Test No difference experienced human/computer HEX: Chatterbot Relevant characteristics Convenience (familiar words) Expressive power (combinations) Ambiguity The house was built by the river by the local council Context Cleaning lady earns 250, Managing director 250(.000,-) HCI &b InfoVis 2017, fjv 41

42 HCI &b InfoVis , fjv

43 Natural Language LUIs Continuous speech recognition (CSR) Natural language understanding common already often narrow in focus (specialized) Query languages almost natural language grammar limited Written language input Part of graphical interface HCI &b InfoVis 2017, fjv 43

44 Example Voice Recognition (in) Phonemes input & recognition: CSR HCI &b InfoVis 2017, fjv 44

45 Voice Output Has to do with Computer-Human Interaction Synthesis of voice BT VODIS: Voice Operated Database Inquiry System KPN Telecom DIAL 118 phone number request Computer operated answer Sound-Devices/Cards have Speech Synthesis Time indication system Read aloud for websites New interfaces use sound/voice for instruction HCI &b InfoVis 2017, fjv 45

46 Example Voice Synthesis (out) Phonemes generation & tone expression HCI &b InfoVis 2017, fjv 46

47 Natural Language From literature: Advantages natural to specify (no need to learn syntax) less intimidating for novices Fits mobile devices Disadvantages can not be generally applied yet requires clarification dialog may require more keystrokes (unless spoken) unpredictable results HCI &b InfoVis 2017, fjv 47

48 Key-Modal Styles Require Input by user/operator Question and Answer Function key interaction Menus HCI &b InfoVis 2017, fjv 48

49 Function-Key Interaction Function keys: Term covers use of number of special input devices used for interaction with a machine e.g. numeric keypads credit card readers (bank-card reader, how many F-keys?) car park barrier operators cash machines, etc. Strict sequence of operations Not much training required Useful in public places HCI &b InfoVis 2017, fjv 49

50 Example Key-Modal Interaction Key-modal, and multi-modal Key-modal style, most used Key-modal, only Backup HCI &b InfoVis 2017, fjv 50

51 Question and Answer Interaction (1) system prompts user, (2) user types answer Often text-based Questions asked one at a time Suitable where short data entry required e.g. wizards, installation software Limited variation in sequence some branching possible Little training needed Limited support for correcting errors Slow in use (depending on user) Control is not with the user HCI &b InfoVis 2017, fjv 51

52 Form Fill-in Includes spreadsheets Designed for clerical workers with little experience Interfaces mimic paper forms (metaphor) Navigation: designed to move insertion point without looking at screen Structured way of gathering info Should be possible to go back If transcription: make screen resemble paper HCI &b InfoVis 2017, fjv 52

53 Form Fill-in From literature: Advantages simplifies data entry requires modest training convenient assistance (recognition vs. recall) allows for formmanagement tools Disadvantages context and application dependent uses menu style for parameter fill-in consumes screen space HCI &b InfoVis 2017, fjv 53

54 Menu Style Interaction Covers toolbars, palettes, text-based menus, etc. Selection possible in multiple ways pointing typing letter or number shortcuts instead of pointing HCI &b InfoVis 2017, fjv 54

55 Characteristics of Menus Offer only those actions that make sense in the current situation (e.g. grey out others) Use based on recognition not remembering Item names and icon must be carefully chosen Not suitable for complex actions e.g. commands with many operands copying a set of files from one disk to another Can be cumbersome way accessing info e.g. if wading through a number of submenus is required Menu may take up screen space, solution: pull-down, pop-up, radial, fish-eye HCI &b InfoVis 2017, fjv 55

56 Styles of Menus There are many types of menus pull-down & pop-up command cascading menus iconic menus tree-structured menus (site maps) pie-menus (pop-up radial) multiple-column menus (common on WWW) Lots of studies have been done how many items/to what depth dynamically rearranging content according to frequency of use role of stable position within a pull-down menu titling (where is the title placed) HCI &b InfoVis 2017, fjv 56

57 User Feedback with Menus Well-designed menu incorporates feedback What options are available? (greyed out) Visual information highlighting current item underlining accelerator keys ticks in front of selected options (sound feedback) walk-through arrows indicating submenus dots... showing that selection refers to further dialogue separator lines to group Feedback on end of selection process (visibility) the appropriate action has occurred HCI &b InfoVis 2017, fjv 57

58 Example Menu Style What options are available? ImageJ: java based image processing HCI &b InfoVis 2017, fjv 58

59 Issues on Single Menu Panel Meaningful item name Top-level panels of menus most error prone It should be easy to go back Undo function is useful (incorporated?) IF undo not possible THEN do not place item at of menu-top OR near frequently used options Critical actions do not use accelerator keys Try out menus with real users HCI &b InfoVis 2017, fjv 59

60 Examples Menus (i) HCI &b InfoVis 2017, fjv 60

Examples Menus (ii) To see updates from friends, photos and feeds, the Zumobi Ziibii interface (http://www.zumobi.

On the left is a static list of text/image items with a gestural swipe used to control paging, and on the right is a

61 Examples Menus (ii) To see updates from friends, photos and feeds, the Zumobi Ziibii interface ( allows users to choose between two styles of presentation. On the left is a static list of text/image items with a gestural swipe used to control paging, and on the right is a dynamic scrolling ticker (called River ) which horizontally scrolls titles and images across the screen. Remark: Newer interaction modes are probed with Menu-structures. HCI &b InfoVis 2017, fjv 61

62 Organizing a Single Menu Panel No right number of items certainly not 7 ± 2 Grouping strategy should be applied categorical conventional frequency of use importance sequence of use alphabetical (if all other fails...) FishEye menu style (cf. direct manipulation) HCI &b InfoVis 2017, fjv 62

63 Menu Style: Topics of Concern 1. Naming of items on option list 2. Depth vs. breadth 3. Categorization 4. Time to selection, cf. Fitt s law: HCI &b InfoVis 2017, fjv 63

64 Menu Selection From literature: Advantages reduces learning time reduces keystrokes structures decision making easy to support error handling Disadvantages too many menus for complex tasks can be slow for frequent users consumes screen space requires rapid display rate HCI &b InfoVis 2017, fjv 64

65 Direct Manipulation Windows, Icons, Menus, Pointers, aka WIMP, Forms, (other objects) Objects can be acted upon directly by user with a pointing device Choice of forms and icons important a. match: designer s representation - user s understanding b. choice of metaphors c. explicit attention to ways in which instantiated and real objects differ d. cultural bias of representation HCI &b InfoVis 2017, fjv 65

66 Archetypal Direct Manipulation HCI &b InfoVis 2017, fjv 66

67 Direct-Manipulation Systems Technologies that derive from word processor: Desktop publication software Slide-presentation software Hypermedia environments Improved macro facilities Spell checker and thesaurus Grammar checkers Integration HCI &b InfoVis 2017, fjv 67

68 Direct-Manipulation Systems Video games Nintendo Wii, Sony PlayStation, Microsoft Xbox - Kinect Field of action is visual and compelling Commands are physical actions whose results are immediately shown on the screen No syntax to remember Most games continuously display a score Direct manipulation in the Sims, Second Life Spore, Myst well received DOOM and Quake controversial HCI &b InfoVis 2017, fjv 68

69 Examples Direct-Manipulation Guitar Hero video game HCI &b InfoVis 2017, fjv 69

70 Direct Manipulation Visual representation of objects and tasks that can be manipulated in arbitrary orders by users: GUI Advantages visually presents task concepts learnability retention over time error avoidance encourages exploration high subjective satisfaction Disadvantages can be difficult to program (especially error handling) non-sighted users requires high-end equipment new developments require to rethink the principles of Direct Interaction HCI &b InfoVis 2017, fjv 70

71 Menus in Direct manipulation Menu Bar (top of

appears where you are pop-up menus - actions for

71 71 Menus in Direct manipulation Menu Bar (top of screen) menu drags down pull-down menu - mouse hold and drag down menu drop-down menu - mouse click reveals menu fall-down menus Contextual menu appears where you are pop-up menus - actions for selected object pie menus - arranged in a circle compass menus

72 Direct-Manipulation Systems: WYSIWYG word processing HCI &b InfoVis 2017, fjv 72

73 Example Direct Manipulation HCI &b InfoVis 2017, fjv 73

74 Example Fish-Eye menus See paper hci.liacs.nl HCI &b InfoVis 2017, fjv 74

75 Example Fish-Eye Menus HCI &b InfoVis 2017, fjv 75

76 Hybrid Systems HCI &b InfoVis 2017, fjv 76

77 Ubiquitous Computing Introduced by Weiser 1991: Invisibly enhancing the world that already exists Computers disappear in environment Goal: Make the interface metaphor visible to user Beyond GUI and WIMP Multi-modal! Sensor based HCI &b InfoVis 2017, fjv 77

78 MEASUREMENT AND EVALUATION OF INTERACTION HCI &b InfoVis 2017, fjv 78

79 Evaluation of Interaction Styles Depending on Context and: Time to learn Speed of performance Rate of errors Retention over time Subjective satisfaction These are typical Usability Aspect Part of usability specifications HCI &b InfoVis 2017, fjv 79

80 Evaluation of Interaction: Fitt s Law Relation between Target Size Distance to Target (GUI) Helps in Efficient Screen Design Fitt s law: T = k log 2 (D/S+0.5) T = time to move hand to target k ± 100ms, D = distance, S = size of target How would this work for newer interactions? HCI &b InfoVis 2017, fjv 80

81 Fitt s Law Experiments T = k log 2 (D/S+0.5) Relation Between Size - Distance HCI &b InfoVis 2017, fjv 81

82 Interaction, Mental Model, Fitt s Law ID 3 = log 2 D S c sin θ HCI &b InfoVis 2017, fjv 82

83 We have acquired a new skill... Link with screen layout Interaction is measurable on the lowest level Distance Target size Speed to the target Usability helps to measure interaction on higher levels of abstraction HCI &b InfoVis 2017, fjv 83

84 Epistemic Actions For Info Vis Exploration is important Epistemic Actions: Actions to support discovery of information Lowest action is exploring the screen, i.e. Eye-movement Interaction in the space with pointing device There is a known approximate time for some of these actions This knowledge can be part of a design cognitive load. HCI &b InfoVis 2017, fjv 84

85 Review #6 Interaction Styles Command Line Natural Language Speech Key-Modal Menus Form fill-in Direct Manipulation Hybrid Styles Evaluation, with Fits Law, Usability HCI &b InfoVis 2017, fjv 85

LECTURE 6, INTERACTION STYLES

September 22 2014 LECTURE 6, INTERACTION STYLES 1 Recapitulation Lecture #5 Computer elements Interaction devices Focus on input Paradigms, Shifts From Batch To Ubiquitous Interaction models Norman Abowd