Project II. argument/reasoning based on the dataset)

Transcription

1 Project II Hive: Simple queries (join, aggregation, group by) Hive: Advanced queries (text extraction, link prediction and graph analysis) Tableau: Visualizations (mutidimensional, interactive, support an argument/reasoning based on the dataset) 60

2 Formalizing Design Choosing Visual Encoding Expressiveness Effectiveness 61

3 Choosing visual encodings Challenge: large design space Assume 8 visual encodings and n data attributes. We would like to pick the best encoding among a combinatorial set of possibilities with size (n+1) 8 Principle of Consistency: The properties of the image (visual variables) should match the properties of the data. Principle of Importance Ordering: Encode the most important information in the most effective way 62

4 Design Criteria 1 Expressiveness A set of facts is expressible in a visual language if the sentences (i.e. the visualizations) in the language express all the facts in the set of data, and only the facts in the data. 63

5 Cannot express the facts Color/Shape -> student count? Blood pressure? A one-to-many (1 N) relation cannot be expressed in a single horizontal dot plot because multiple tuples are mapped to the same position 64

6 Expresses facts not in the data A length is interpreted as a quantitative value; So, length of bar says something untrue about N data 65

7 Design Criteria 2 Effectiveness A visualization is more effective than another visualization if the information conveyed by one visualization is more readily perceived than the information in the other visualization. 66

8 Mackinlay s Ranking Conjectured effectiveness of the encoding 67

9 Mackinlay s Visualization Design Algorithm User formally specifies data model and type Additional input: ordered list of data variables to show Algorithm searches over design space Tests expressiveness of each visual encoding Generates specification for encodings that pass test Tests perceptual effectiveness of resulting image Outputs the most effective visualization 68

10 Limitations Does not cover many visualization techniques Bertin and others discuss networks, maps, diagrams Does not consider 3D, animation, illustration, photography, Does not model interaction Does not consider semantic data types / conventions 69

11 Tableau 70

12 Tableau 71

13 Tableau: Interactive Mashups 72

14 Image Model Visual language is a sign system Images perceived as a set of signs Sender encodes information in signs Receiver decodes information from signs Sémiologie Graphique,

15 Design Considerations 1 Expressiveness Prioritizes important information / Avoids false inferences Consistent visual mappings (e.g., respect color mappings) Make encodings meaningful rather than arbitrary Effectiveness Facilitates accurate decoding / Minimizes cognitive overhead Highlight elements of primary interest Grouping / Sorting Data Transformation Non-Data Elements Descriptive: Title, Label, Caption, Data Source, Annotations Reference: Gridlines, Legend 74

16 Design Considerations 2 Don t distract: faint gridlines, pastel highlights/fills The elimination diet approach start minimal Support comparison and pattern perception Between elements, to a reference line, or to totals Reduce cognitive overhead Minimize visual search, minimize ambiguity -> Avoid legend lookups if direct labeling works -> Avoid color mappings with indiscernible colors 75

17 Bertin s Antibiotic Data Set Effectiveness of three antibiotics on sixteen strains of bacteria. antibiotic dataset Genus of Bacteria: String Species of Bacteria: String Antibiotic Applied: String Gram-Staining: Pos / Neg Min. Inhibitory Concent. (g): Number Collected prior to Different Visualizations: Bar Chart, Line Chart, Dot Plots, Tables/Heat Maps, Others, e.g., Pie chart 76

18 Bertin s Antibiotic Data Set MIC is the lowest concentration of an antimicrobial that will inhibit the visible growth of a microorganism after overnight incubation. 77

19 Different Types of Visualizations Bar Charts MIC for 3 medicine 16 Bacteria 78

20 Different Types of Visualizations Bar Chart 79

21 Different Types of Visualizations Bar Chart 80

22 Different Types of Visualizations Bar Chart 81

23 Different Types of Visualizations Bar Chart 82

24 Different Types of Visualizations Bar Chart 83

25 Different Types of Visualizations Line Chart 84

26 Different Types of Visualizations Dot Plot 85

27 Different Types of Visualizations Table/Heap Map 86

28 Data Quality and Usability Hurdles Missing Data: no measurements Erroneous Values: misspelling, outliers Type Conversion: e.g., zip code to latlon Entity Resolution: diff. values for the same thing? Data Integration: effort/errors when combining data LESSON: Anticipate problems with your data. 87

29 Data Wrangling One often needs to manipulate data prior to analysis. Tasks include reformatting, cleaning, quality assessment, and integration. Some approaches include: Writing custom scripts Manual manipulation in spreadsheets Data Wrangler: Google Refine: 88

30 Exploratory Data Analysis Exploratory Process Construct graphics to address questions Inspect answer and assess new questions Repeat! Transform the data appropriately (e.g., invert, log) Show data variation, not design variation -Tufte 89

31 Exploratory Data analysis How do the drugs compare? Critique? 90

32 Exploratory Data Analysis How do the bacteria group with respect to antibiotic resistance? 91

33 Exploratory Data Analysis How do the bacteria group w.r.t. resistance? Do different drugs correlate? 92

34 Incorporating Model Hypothesis testing: What is the probability that the pattern might have arisen by chance? Prediction: How well do one (or more) data variables predict another? Abstract description: With what parameters does the data best fit a given function? What is the goodness of fit? Scientific theory: Which model explains reality? 93

35 Summary Value of Visualization Data And Image Models Visualization Design Exploratory Data Analysis Interactive Visualization 94