INTERACTION IN VISUALIZATION. Petra Isenberg

Transcription

1 INTERACTION IN VISUALIZATION Petra Isenberg

2 RECAP Interaction is fundamental to the definition of visual exploration You have already seen examples for graphs for time series for multi-dimensional data

3 RECAP Visual exploration is more than just looking So far we focused mostly on output but we have already used limited input earlier in the course Today: input for steering visual output 3

4 WHY INTERACT?

5 DEFINITION OF INTERACTION Static content many infographics Dynamic content Animated content Changes independently from the user Interactive content Changes as a result of user actions 5

6 PERCEPTION REQUIRES ACTION LEDERMAN AND KLATZKY, 1987

7

8 THERE IS TOO MUCH DATA TO SHOW THERE ARE MANY WAYS TO SHOW IT LET THE USER DYNAMICALLY CONTROL WHAT TO SHOW AND HOW

9 NOT TOO LONG AGO 1970s ml

10 TAXONOMIES OF INTERACTION What? What is the user doing? Why? Why is the user doing it? How? How is the user doing it? 10

11 11 THE VISUALIZATION PIPELINE

12 THE VISUALIZATION PIPELINE Raw Data Selection Representation Presentation Interaction The Visualization Pipeline From [Spence, 2000]

13 THE VISUALIZATION PIPELINE Data Analytics Abstraction Spatial Layout Presentation View Data Transformation Spatial Mapping Transformation Presentation Transformation View Transformation From [Card et al., Readings in Information Visualization]

14 THE VISUALIZATION PIPELINE [Card, Mackinlay, Shneiderman, Readings in Information Visualization: Using Vision to Think, 1999

15 THE VISUALIZATION PIPELINE Interaction From Ed CHI Illustration by J. Heer

16 Jansen and Dragicevic 2013 (

17 Jansen and Dragicevic 2013 (

18 Jansen and Dragicevic 2013 (

19 Jansen and Dragicevic 2013 (

20 Jansen and Dragicevic 2013 (

21 Jansen and Dragicevic 2013 (

22 Jansen and Dragicevic 2013 (

23 Jansen and Dragicevic 2013 (

24 Jansen and Dragicevic 2013 ( (view level) (visual level) (data level)

25 TAXONOMIES OF INTERACTION What? What is the user doing? Why? Why is the user doing it? Tasks How? How is the user doing it? 25

26 INTERACTION AS TASKS

27 ANALYTIC TASKS BEN SHNEIDERMAN OVERVIEW: GAIN AN OVERVIEW OF THE ENTIRE COLLECTION 2. ZOOM: ZOOM IN ON ITEMS OF INTEREST 3. FILTER: FILTER OUT UNINTERESTING ITEMS 4. DETAILS-ON-DEMAND: SELECT AN ITEM OR GROUP AND GET DETAILS WHEN NEEDED 5. RELATE: VIEW RELATIONSHIPS AMONG ITEMS 6. HISTORY: KEEP PAST ACTIONS FOR UNDO, REPLAY, AND PROGRESSIVE REFINEMENT 7. EXTRACT: ALLOW EXTRACTION OF SUB-COLLECTIONS AND QUERY PARAMETERS.

28 STEPHEN FEW, 2006 (LINK) SOFTWARE: TIMESEARCHER 2 ANALYTICAL 1. OVERVIEW TASKS

29 ANALYTICAL 2. ZOOM TASKS

30 ANALYTICAL 3. FILTER TASKS

31 ANALYTICAL 4. DETAILS ON DEMAND TASKS

32 VISUAL INFORMATION SEEKING MANTRA OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND OVERVIEW FIRST, ZOOM AND FILTER, THEN DETAILS ON DEMAND

33 INTERACTION AS INTENTS

34 SEVEN CATEGORIES OF INTERACTION BY INTENT SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT Y I E T A L. 2007

35 SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT MARK SOMETHING AS INTERESTING

36 BASIC POINTING Point Selection Mouse METHODS Hover / Click Touch / Tap

37

38

39 BASIC POINTING Point Selection Mouse METHODS Hover / Click Touch / Tap Select Nearby Element (e.g., Bubble Cursor)

40

41

42

43 BASIC POINTING Point Selection Mouse METHODS Hover / Click Touch / Tap Select Nearby Element (e.g., Bubble Cursor) Region Selection Rubber-band or Lasso Area Cursors ( Brushes )

44 RANGE SELECTION GENERALIZED SELECTION HEER ET AL. 2008

45 SELECTION TOOLS BRUSHES L A S S O S

46 WILLS SELECTION TAXONOMY:

47 WILLS SELECTION TAXONOMY SELECTION MEMORY M E M O R Y 2 5 < A G E < 3 5 M E M O R Y L E S S 2 5 < A G E < 3 5 C O U N T R Y = C A N A D A E D U L E V E L = P O S T S E C O N D A R Y C O U N T R Y = C A N A D A E D U L E V E L = P O S T S E C O N D A R Y WILLS 2000

48 SELECTION OPERATIONS R E P L A C E I N T E R S E C T A D D S U B T R A C T T O G G L E

49 RECOMMENDED SELECTION OPERATIONS 1 3 & T O G G L E A D D S U B T R A C T 2 & 4 & R E P L A C E T O G G L E A D D I N T E R S E C T 5 & & & & WILLS 2000 R E P L A C E T O G G L E A D D S U B T R A C T I N T E R S E C T

50 HIGHLIGHTING S E L E C T I O N + C H A N G E I N A P P E A R A N C E

51 SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT SHOW ME SOMETHING ELSE

52 PROBLEM Where am I? 52

53 NAVIGATION PANNING ZOOMING

54 NAVIGATION PANNING ZOOMING

55 H A R R Y P O T T E R FA M I LY T R E E AV I Z. F R / G E N E A Q U I LT S

56 SELECT EXPLORE FILTER SHOW ME SOMETHING CONDITIONALLY RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT

57 FILTERING REPLACING A QUERY WITH DYNAMIC QUERY WIDGETS > SELECT house-address FROM realty-db WHERE price >= 200,000 AND price <= 400,000 AND bathrooms >= 3 AND garage == 2 AND bedrooms >= 4; Q? A! HOMEFINDER WILLIAMSON AND SCHNEIDERMAN 1992

58

59 DIRECT MANIPULATION 1. Visual representation of objects and actions 2. Rapid, incremental and reversible actions 3. Selection by pointing (not typing) 4. Immediate and continuous display of results

60 WILLIAMSON AND SHNEIDERMAN, 1992

61 CRIMESPOTTING STAMEN DESIGN

62 ZIPDECODE [FRY 04] BEN FRY 1999

63 BABY NAME VOYAGER MARTIN WATTENBERG 2005

64 SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT SHOW ME A DIFFERENT ARRANGEMENT

65 RE-ARRANGING DATA INTERACTIVE STACKED HISTOGRAMS [DIX & ELLIS, 1998]

66 MATRIX

67 MATRIX

68 KINETICA [RZESZORTARKSI & KITTUR 2013]

69 SELECT EXPLORE FILTER RECONFIGURE ENCODE SHOW ME A DIFFERENT REPRESENTATION ABSTRACT/ELABORATE CONNECT

70

71 SELECT EXPLORE FILTER RECONFIGURE ENCODE SHOW ME MORE OR LESS DETAIL ABSTRACT/ELABORATE CONNECT

72 CHANGING LEVELS OF ABSTRACTION POWERS OF TEN RAY & CHARLES EAMES 1977

73

74 SEMANTIC ZOOMING PAD++ BEDERSON AND HOLLAN 1994

75 LiveRAC ENCODINGS CHANGE COLORED BOX SPARKLINE SIMPLE LINE CHART FULL CHART: AXES AND TICKMARKS MCLACHLAN ET AL. 2008

76 OVERVIEW + DETAIL Panning a large graph 76

77 OVERVIEW + DETAIL Panning a line chart 77

78 OVERVIEW + DETAIL Browsing Multiple Views 78

79 OVERVIEW + DETAIL Browsing Multiple Views 79 Jansen et al, 2013

80 FOCUS + CONTEXT Space Distortion 1) Fisheye Views of Graphs 80 Sarkar and Brown, 1992

81 FOCUS + CONTEXT Space Distortion 2) Fisheye Menus 81 Bederson, 2000

82 FOCUS + Space Distortion 3) Perspective Wall CONTEXT 82 Mackinlay, Roberston and Card, 1991

83 FOCUS + CONTEXT Space Distortion 4) Melange 83 Elmqvist et al, 2010

84

85 FOCUS + Space Distorsion CONTEXT Melange 85 Brosz, Carpendale and Nacenta, 2011

86

87 FOCUS + CONTEXT Table Lens 87 Rao and Card, 1994

88 FOCUS + CONTEXT Generalized Fisheye Views 88 Furnas, 1986 Generalized Fisheye Views

89 FOCUS + CONTEXT Generalized Fisheye Views 89 Furnas, 2010 A Fisheye Follow-Up: Further Reflections on Focus + Context

90 MAGIC LENSES 90 Bier et al, 1993

91 MAGIC LENSES Movable filters for dynamic queries 91 Fishkin and Stone, 1995

92 MAGIC LENSES Exentric Labeling 92 Fekete and Plaisant, 1999

93 MAGIC LENSES Edge lenses 93 Wong, Carpendale and Greenberg,

94 SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT SHOW ME RELATED ITEMS

95 BRUSHING AND LINKING SELECT ( BRUSH ) A SUBSET OF DATA SEE SELECTED DATA IN OTHER VIEWS THE COMPONENTS MUST BE LINKED BY TUPLE (MATCHING DATA POINTS), OR BY QUERY (MATCHING RANGE OR VALUES)

96 BRUSHING AND LINKING BRUSHING SCATTERPLOTS BECKER & CLEVELAND 1982

97 EXAMPLE 3: BRUSHING Beker and Cleveland, 1987

98 BRUSHING & LINKING HOW LONG IN MAJORS SALARIES ASSISTS VS PUTOUTS (FIELDING ABILITY) HOME RUNS VS HITS (BATTING ABILITY) DISTRIBUTION OF POSITIONS PLAYED BASEBALL STATISTICS [FROM WILLS 95]

99 BRUSHING & LINKING HOW LONG IN MAJORS SALARIES ASSISTS VS PUTOUTS (FIELDING ABILITY) HOME RUNS VS HITS (BATTING ABILITY) DISTRIBUTION OF POSITIONS PLAYED BASEBALL STATISTICS [FROM WILLS 95]

100

101 SEVEN CATEGORIES OF INTERACTION BY INTENT SELECT EXPLORE FILTER RECONFIGURE ENCODE ABSTRACT/ELABORATE CONNECT Y I E T A L. 2007

102 TAXONOMIES OF INTERACTION What? What is the user doing? Why? Why is the user doing it? How? How is the user doing it? 102

103 HOW? INTERACTION TECHNIQUE An interaction technique is the fusion of input and output, consisting of all software and hardware elements, that provides a way for the user to accomplish a task (Tucker, 2004) TYPES OF INTERACTION TECHNIQUES Input: mouse, touch, keyboard, speech,... Shneiderman: Command-line interfaces vs. Direct manipulation interfaces 103

104 HOW? INTERACTION TECHNIQUE An interaction technique is the fusion of input and output, consisting of all software and hardware elements, that provides a way for the user to accomplish a task (Tucker, 2004) TYPES OF INTERACTION TECHNIQUES Input: mouse, touch, keyboard, speech,... Shneiderman: Command-line interfaces vs. Direct manipulation interfaces Beaudouin-Lafon: Instruments with different degrees of directness 104

105 PITFALLS #1 - Interaction has a cost 105

106 PITFALLS #2 - Controls take screen real-estate 106

107 PITFALLS #3 - Few techniques are self-explanatory 107

108 GOING BEYOND THE DESKTOP 108

109 TOUCH DEVICES Sadana and Stasko,

110 TABLETOP DEVICES Isenberg and Carpendale,

111 WALL-SIZED DISPLAYS 111

112 112 [Jansen et al., Tangible Remote Controller for Wall-sized Displays. CHI 12]

113 Jansen and Dragicevic 2013 ( (view level) (visual level) (data level)

114 Jansen and Dragicevic 2013 ( (view level) (visual level) (data level)

115 115 [Isenberg et al., Hybrid Images for Large Viewing Environments, InfoVis 13]

116 [Isenberg et al., Hybrid Images for Large Viewing Environments, InfoVis 13]

117 INTERACTION WITH THE PHYSICAL WORLD

118 PHYSICAL VISUALIZATIONS 118

119 tinyurl.com/physvis [Mark Wilson. How GM is saving cash using legos as a data viz tool. April 2012] 119

120 [Kruszynski & van Liere, Tangible Props for Scientific Visualization, Virtual Reality 13 (4) 2009]

121 [Stefaner & Hemmert, emoto data sculpture, 121

122 122 [PARM: Projected Augmented Relief Models, University of Nottingham, 2012]

123 123 Relief (Leithinger et al, 2009)

124

125 ACKNOWLEDGEMENTS Slides in were inspired and adapted from slides by Wesley Willett (University of Calgary) Pierre Dragicevic (Inria)