EECS 4441 / CSE 5351 Human-Computer Interaction. Topic #3 The Computer

Transcription

1 EECS 4441 / CSE 5351 Human-Computer Interaction Topic #3 The Computer I. Scott MacKenzie York University, Canada 1 The Computer A computer system is made up of various elements Each affects the interaction Input devices text entry and pointing Output devices screen (small/large), digital paper Virtual reality special interaction and display devices Physical interaction e.g., sound, haptic, bio-sensing Memory RAM, permanent media, capacity & access Processing speed of processing, networks 2 1

2 ? A Typical Computer System Screen or monitor on which there are windows Keyboard Mouse/touchpad Variations Desktop Laptop PDA, smart phone, tablet The devices dictate the styles of interaction that the system supports If we use different devices, then the interface will support a different style of interaction 3 How Many Computers In your house? PC TV, VCR, DVD, HiFi, cable/satellite TV Microwave, cooker, washing machine Central heating Security system In your pockets? PDA Phone, camera Electronic car key USB memory stick 4 2

3 Interactivity Long ago in a galaxy far, far away batch processing Punched card stacks or large data files prepared Long wait Line printer output And it if is not right Now most computing is interactive Rapid feedback The user is in control (most of the time) Interacting rather than responding 5 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 6 3

4 Text Entry Devices Keyboards QWERTY Chord keyboards Phone keypads Soft keyboards Handwriting (stylus + tablet) Speech (microphone) 7 Keyboards Most common text input device Allows rapid entry of text by experienced users Keypress closes connection causing a character code to be sent Usually connected by cable, but can be wireless 8 4

5 Layout - QWERTY Standardized layout, but Non-alphanumeric keys are placed differently Accented symbols needed for different scripts Minor difference between UK and USA keyboards QWERTY arrangement not optimal for typing Layout to prevent typewriter jamming! Alternative designs allow faster typing but large social base of QWERTY typists produces reluctance to change 9 Layout QWERTY (2) Left Hand Right Hand 10 5

6 Alternative Keyboard Layouts Dvorak Common letters under dominant fingers Biased towards right hand Common combinations of letters alternate hands 10-15% improvement in speed and reduction in fatigue But large social base of QWERTY typists produce market pressure not to change Left Hand Right Hand Demo KeyHandAnalyses (next slide) 11 Qwerty (key-hand analysis) 12 6

7 Dvorak (key-hand analysis) 13 Alternative Keyboard Layouts (2) Alphabetic Keys arranged in alphabetic order Not faster for trained typists Not faster for beginnings either! Used in some field-service terminals (e.g., couriers) where device must fit in hand 14 7

8 Special Keyboards Designed to reduce fatigue for RSI (repetitive stress injuries) One handed use E.g., the Maltron left-handed keyboard 15 Chord Keyboards Only a few keys 4 or 5 Letters typed as combination of keypresses Compact size Ideal for portable applications Short learning time (?) Keypresses reflect letter shape Fast (once trained) But social resistance, long learning curve, fatigue after extended use NEW niche market for some wearables 16 8

9 Chord Keyboard Example - Twiddler See Lyons, K., Starner, T., Plaisted, D., Fusia, J., Lyons, A., Drew, A., and Looney, E. W., Twiddler typing: One-handed chording text entry for mobile phones, Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2004, (New York: ACM, 2004), Chord Keyboard Example - Stenotype There are various ways to combine letters to make different sounds; different court reporters use different theories in their work. Although most writing is similar, most stenographers cannot read another's work, as it is highly personalized. See Stenotype in Wikipedia 18 9

10 Phone Keypad and T9 Entry Multitap hello = [pause] hello = N Questions: How fast? How accurate? T9 predictive entry One key (?) per letter Dictionary guesses the word hello = But 26 am or an Homework. Please read Silfverberg, M., MacKenzie, I. S., & Korhonen, P. (2000). Predicting text entry speed on mobile phones. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI 2000, pp New York: ACM. 19 Demo: BuildKeystrokes 20 10

11 Demo: KSPCWords 21 Demo: T

12 Keystrokes Per Character (KSPC) KSPC is a convenient measure to characterise and compare entry methods that require primitives actions, such a keystrokes, stylus strokes, finger strokes, taps, etc. E.g., KSPC MULTITAP = Homework. Please read MacKenzie, I. S. (2002). KSPC (keystrokes per character) as a characteristic of text entry techniques. Proceedings of the Fourth International Symposium on Human Computer Interaction with Mobile Devices, pp Heidelberg, Germany: Springer-Verlag. 23 KSPC > 1, KSPC < 1 Ambiguous vs. predictive text entry Demo PhoneKeypadExperiment 24 12

13 Demo: PhoneKeypadExperiment 25 Soft Keyboards Easy to re-configure layout Not as strong a case for using QWERTY Goal: maximize entry speed by minimizing stylus/finger movement 26 13

14 Soft Keyboards - OPTI OPTI Questions: What design method? How fast can one type right away? with practice? Faster than QWERTY? Are there faster layouts? The best question of all: Can the questions above be answered without conducting an experiment; i.e., by modeling the interaction? 27 Soft Keyboards OPTI (2) Homework. Please read MacKenzie, I. S., & Zhang, S. Z. (1999) The design and evaluation of a high performance soft keyboard. Proceedings of the ACM Conference on Human Factors in Computing Systems - CHI '99, pp New York: ACM

15 Demo: SoftKeyboardExperiment 29 Scanning Keyboards Soft keyboard keys are highlighted sequentially ( scanned ) Physical key or switch activated when desired key is highlighted; character added to text stream Used in accessible computing (for users with motor impairment) Performance improved using Row-column scanning Optimized letter arrangement Word completion etc

16 Demo: Microsoft On-screen Keyboard 31 Optimized Scanning Keyboard 1 Model for scanning keyboards? 1 Koester, H. H. and Levine, S. P., Learning and performance of able-bodied individuals using scanning systems with and without word prediction, Assistive Technology, 6, 1994,

17 Demo: Scanning Ambiguous Keyboard 33 Handwriting Recognition Text can be entered using a pen or finger and a digitizing tablet Natural interaction Technical problems: Capturing all useful information stroke path, pressure, etc., in a natural manner Segmenting joined-up writing into individual letters Interpreting individual letters Coping with different styles of handwriting Used in PDAs and tablet computers leave the keyboards on the desk! 34 17

18 Handwriting Recognition (2) Solution to the segmentation problem Unistrokes One stroke per symbol User must learn the stroke alphabet Recognizer: simple, small, accurate Inventor: David Goldberg, 1993, Xerox PARC Commercial example: Graffiti on the Palm Pilot 35 Unistrokes Goal: Fast entry for experts 36 18

19 Graffiti Goal: Easy for novices Bksp 37 Demo: GraffitiExperiment 38 19

20 Demo: Graffiti on Android 39 Gestural Input Graffiti and Unistrokes are examples of gestural input Other examples 40 20

21 Marking Menus 1 1 Kurtenbach, G., & Buxton, W. (1993). The limits of expert performance using hierarchic marking menus. Proceedings of the INTERCHI '93, New York: ACM. 41 Marking Menus

22 EdgeWrite 1 1 Wobbrock, J. O., Myers, B. A., & Kembel, J. A. (2003). EdgeWrite: A stylus-based text entry method designed for high accuracy and stability of motion. Proceedings of UIST 2003, New York: ACM. 43 LURD-Writer 1 1 Felzer, T., & Nordmann, R. (2006). Alternative text entry using different input methods. Proceedings of ACCESS 2006, New York: ACM

23 H4-Writer 1 (based on Huffman codes) 1 MacKenzie, I. S., Soukoreff, R. W., & Helga, J. (2011). 1 thumb, 4 buttons, 20 words per minute: Design and evaluation of H4-Writer. Proc UIST 2011, New York: ACM. 45 EdgeWrite vs. H4-Writer 46 23

24 Demo Note: 0 = L 1 = U 2 = R 3 = D H UL E UU L DLL L DLL O RL _ DD 47 H4-Writer Input Possibilities Relative Relative Relative Demo 48 24

25 H4-Writer on Android 49 Eye on the Message/Keyboard 1 1 MacKenzie, I. S., & Castellucci, S. J. (2013). Eye on the message: Reducing attention demand for touch-based text entry. International Journal of Virtual Worlds and Human- Computer Interaction, 1,

26 H4-Writer Feedback RDRLLUULULDDLUUUDLLURLUDLULLUULUDDR h e l r «l o _ w o r l d Legend Click sound Typewriter key sound Erase sound (under discussion) Typewriter return sound Speaking of feedback 51 Reducing Visual Demand (Revisited) Auditory sense Tactile sense Auditory sense Vision Vision Tactile sense 52 26

27 Speech Recognition Improving rapidly But this has always been the claim for speech recognition Most successful when: User-specific Initial training and learns peculiarities Limited vocabulary systems Problems with External noise interfering Imprecision of pronunciation Large vocabularies Different speakers 53 Numeric Keypads For entering numbers quickly Calculator, PC keyboard, telephone 54 27

28 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 55 Positioning, Pointing, and Drawing Mouse Touchpad Trackballs Joysticks Touch screens Tablets Eye gaze Cursors 56 28

29 Input devices are not interchangeable! Device must be appropriate for usage Demonstration Joystick (Skedoodle) vs. rotary controls (Etch-a-sketch) Question: Which is better? Answer 1: Joystick (OK, draw a square) Answer 2: Rotary controls (OK, draw a circle) 57 The First Mouse Doug Engelbart, circa

30 The Mouse Handheld pointing device Very common Easy to use Two characteristics Planar movement Buttons Usually from 1 to 3 buttons on top, used for selecting or indicating an option or to initiate drawing, etc. 59 The Mouse (2) Mouse located on desktop Requires physical space No arm fatigue Relative movement only is detectable Movement of mouse moves screen cursor Screen cursor oriented in x,y plane, but Mouse movement in x,z plane An indirect manipulation device Device itself doesn t obscure screen, is accurate and fast Hand-eye coordination problems for novice users 60 30

31 How Does it Work? Two methods for detecting motion Mechanical Ball on underside of mouse turns as mouse is moved Rotates orthogonal potentiometers Can be used on almost any flat surface Optical Light emitting diodes (camera!) on underside of mouse May use special grid-like pad or just on desk Less susceptible to dust and dirt Detects fluctuating alterations in reflected light intensity to calculate relative motion in x,z plane 61 FootMouse Some experiments with the footmouse Controlling mouse movements with feet Not very common But foot controls are common elsewhere Car pedals Sewing machine speed control Organ and piano pedals 62 31

32 HeadMouse / MarkerMouse 1 1 Javanovic, R., & MacKenzie, I. S. (2010). MarkerMouse: Mouse cursor control using a head-mounted marker. Proc ICCHP 2010, Berlin: Springer. 63 Touchpad Small touch sensitive tablets stroke to move mouse pointer Used mainly in laptop computers Good accelerator settings important Fast stroke Lots of pixels per inch moved Initial movement to the target Slow stroke Less pixels per inch For accurate positioning 64 32

33 Trackball and Thumbwheels Trackball Ball is rotated inside static housing Like an upside-down mouse Relative motion moves cursor Indirect device, fairly accurate Separate buttons for picking Very fast for gaming Used in some portable and notebook computers Thumbwheels For accurate CAD two dials for x,y cursor position For fast scrolling single dial on mouse (next slide) 65 Wheel(s) on a Mouse (prior to Microsoft Intellimouse) ProAgio by Mouse Systems, Inc. Venolia, D., Facile 3D manipulation, Proceedings of the INTERCHI '93 Conference on Human Factors in Computing Systems, (New York: ACM, 1993),

34 Joystick and the Trackpoint Joystick Indirect pointing device (like a mouse) Two types (next slide) Isometric: senses pressure on stick Isotonic: senses movement of stick Mappings (typical, but see two slides over) Isometric: pressure on stick velocity/rate of cursor Isotonic: displacement of stick displacement/position of cursor Buttons for selection On top or on front (like a trigger) TrackPoint For laptop computers Introduced by IBM on the ThinkPad Isometric joystick 67 Isotonic and Isometric Joysticks From

35 Mappings Consider the possibilities Cursor Property Controlled Velocity Position Joystick Property Sensed Stick Displacement Stick Pressure? Typical Typical? 69 Touch-sensitive Screen Detect the presence of finger or stylus on screen Works by interrupting matrix of light beams, capacitive change or ultrasonic reflections Direct input pointing device Advantages Fast and requires no specialized pointing Good for menu selection Suitable for use in hostile environments: clean and safe from damage Disadvantages Finger can mark screen Imprecise ( fat finger problem) Difficult to select small regions or perform accurate drawing Lifting arm can be tiring Parallax 70 35

36 Stylus and Light Pen Stylus Small pen-like pointer to draw directly on screen May use touch sensitive surface or magnetic detection Used in PDAs, tablet PCs, and drawing tables Light pen Now rarely used Uses light from screen and timing of raster to detect location Both Very direct and obvious to user But can obscure screen 71 Digitizing Tablet Mouse-like device with cross hairs Used on special surface Rather like a stylus Very accurate Used for digitizing maps (for example) 72 36

37 Eye Gaze Control interface by eye gaze direction E.g., look at a menu item to select it Uses laser or infra-red beam reflected off retina a very low power (if laser)! Mainly used for evaluation (e.g., reading behaviour) Potential for hands-free control High accuracy requires headset Cheaper and lower accuracy devices available Sit under the screen like a small web cam 73 Cursor Keys Four keys (up, down, left, right) on keyboard Very, very cheap, but slow Useful for not much more than basic motion for text editing tasks No standardized layout, but inverted T is most common 74 37

38 Discrete Positioning Controls In phones, TV controls, etc. Cursor pads or mini-joysticks Discrete left-right, up-down Mainly for menu selection 75 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 76 38

39 Display Devices Bitmapped screens (CRT & LCD) Large and situated displays Digital paper 77 Bitmapped Displays Screen is a vast array of coloured dots called pixels (picture elements) 78 39

40 Resolution and Colour Depth Resolution used (inconsistently) for Number of pixels on screen (width and height) E.g., SVGA 1024 x 768 Density of pixels (in pixels or dots per inch dpi) Typically between 72 and 96 dpi Aspect ratio Ratio between width and height 4:3 for most screens Colour depth Number of different colours for each pixel Black/white or greys only 256 from a pallet 8 bits each for red/green/blue = millions of colours 79 Anti-aliasing Jaggies Diagonal lines that have discontinuities due to horizontal raster scan patter Anti-aliasing Softens edges by using shades of line colour Also used for text 80 40

41 Cathode-Ray Tube (CRT) Stream of electrons emitted from electron gun, focused and directed by magnetic fields, hit phosphor-coated screen which glows Used in (old) TVs and (old) computer monitors 81 Liquid Crystal Displays Smaller, lighter, and no radiation problems Found on PDAs, portables and notebooks, and increasingly on desktop and home TVs Also in dedicated displays: digital watches, mobile phones How it works Top plate transparent and polarized, bottom plate reflecting Light passes through top plate and crystal, and reflects back to eye Voltage applied to crystal changes polarization and hence colour Note: light reflected not emitted less eye strain Irrelevant (from an HCI perspective) 82 41

42 Large Displays Used for meetings, lectures, etc. Back-projected 83 Situated Displays Displays in public places Large or small Very public or for small groups 84 42

43 New Displays Curved surfaces Vertegaal, R. (2011, January-February). Interactions, pp Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 86 43

44 Virtual reality and 3D Interaction Positioning in 3D Moving and grasping See 3D (helmets and caves) 87 Positioning in 3D Space Cockpit and virtual controls Steering wheels, knobs and dials just like real! The 3D mouse Six degrees-of-freedom: x, y, z + roll, pitch, yaw Data glove Fibre optics used to detect finger position VR helmets Detect head motion and possibly eye gaze Whole body tracking Accelerometers strapped to limbs or reflective dots and video processing 88 44

45 Pitch, Roll, and Yaw 89 3D Displays Desktop VR or volumetric displays Grossman, T., Wigdor, D., and Balakrishnan, R., Multi-finger gestural interaction with 3d volumetric displays, Proc UIST 2004, (New York: ACM, 2004),

46 VR Headsets Small TV screen for each eye Slightly different angles 3D effect 91 VR Motion Sickness Time delay Move head lag display moves Conflict: head movements vs. eyes Depth perception Headset gives different stereo distance But all focused in same plane Conflict: eye angle vs. focus Conflicting cues sickness Helps motivate improvements in technology 92 46

47 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 93 Environment and Bio-sensing Sensors all around us Ultrasound detectors security, washbasins RFID security tags in shops Iris scanners, body temperature, heart rate, galvanic skin response, blink rate, brain signals OCZ's Neural Impulse Actuator (NIA) (senses muscle impulses and alpha/beta signals from the brain) 94 47

48 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 95 Paper: Printing and Scanning Print technology Fonts, page description, WYSIWYG Scanning, OCR 96 48

49 Printing Image made from small Allows any character set or graphic to be printed Critical features Resolution Size and spacing of the dots Measured in dots per inch (dpi) Speed Usually measured in pages per minute Cost!! 97 Fonts Font the particular style of text Size of font measured in points (1 pt about 1/72 ) vaguely related to its height 98 49

50 Fonts (continued) Pitch Fixed pitch every character has the same width; e.g., Courier Variable-pitch some characters wider; e.g., Times Roman (compare the i and m ) Serif or sans-serif Sans-serif square-ended strokes; e.g., Arial Serif with splayed ends; e.g., Times Roman or Palatino 99 Readability of Text lowercase Easy to read shape of words UPPERCASE Better for individual letters and non-words; e.g., flight numbers: BA793 vs ba793 Serif fonts Helps your eye on long lines of printed text But sans serif often better on screen

51 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks 101 Memory Short term and long term Speed, capacity, compression Formats, access

52 Short-term Memory RAM Random access memory (RAM) On silicon chips 100 nano-second access time Usually volatile (lose information if power turned off) Data transferred at around 100 Mbytes/second Same non-volatile RAM used to store basic setup information Typical desktop computers: 64 to 256 Mbytes RAM 103 Long-term Memory Disks Magnetic disks Floppy disks store around 1.4 Mbytes Hard disks typically 40 Gbytes to hundreds of Gbytes Access time ~ 10 ms, transfer rate ~ 100 kbytes/s Optical disks Use lasers to read and sometimes write More robust than magnetic media CD-ROM Same technology as home audio, ~ 600 Gbytes DVD for AV applications, or very large files

53 Moore s Law Computers get faster and faster 1965 Gordon Moore, co-founder of Intel, noticed a pattern Processor speed doubles every 18 months PC 1987 = 1.5 MHz, 2002 = 1.5 GHz Similar pattern for memory But doubles every 12 months Hard disk 1991 = 20 Mbytes, 2002 = 30 Gbytes Baby born today Record all sound and vision By 70, all life s memories stored in a grain of dust! 105 Topics Text entry devices Positioning, pointing, drawing Display devices Virtual reality and 3D interaction Physical controls, sensors, etc. Paper: printing and scanning Memory Networks

54 Networked Computing Networks allow access to Large memory and processing Other people (groupware, , Facebook, Twitter) Shared resource especially the web HCI Issues Network delays slow feedback Conflicts many people updating data Unpredictability 107 Thank You