Polaris. Aditya Parameswaran

Transcription

1 Polaris Aditya Parameswaran

2 Key ideas Relational like-language to translate between visualization specifications and database queries The key idea behind Tableau Richer than existing tools in terms of space of possible visualizations Better than existing tools in that the connection to the underlying queries is more evident and directly captured 2

3 Data Visualizations: Why? Analyze Discover trends Stock price is going up/down Develop hypotheses House prices are down due to the downturn Check hypotheses Detect errors Null values in a column Share, record & communicate 3

4 At a high level Table-based displays Why are table-based displays good? Different dimensions can be captured in the structure of the table Naturally comparative Common and easy to understand

5 "#$"% %" &$'( )#$&*+!( &!,!"%-.#* /0%*,1 &2&$,!+!1 &23 4+!0&$+5&"+#2 #. -0$"+3+-%2!+#2&$ *%$&"+#2&$ 3&"&6&!%! 7.89' :' ";< &C?>GADA D?H><IH?A<J J8AK>?GA J?D? HG E8<>JA D;< J?D?H?A< AF;<L? =CD= ;<>M<A D;< J8AK>?G' & 98M<C =E E8<>JA =C A;<>M<A 8A F?>><J? M8AB?> AK<F8E8F?D8=C' ";< AK<F8E8F?D8=C BC?LH89B=BA>G J<E8C<A D;< 5 C?>GA8A?CJ M8AB?>8N?D8=C =K<@?D8=CA D= H< K<@E=@L<J HG D;< AGAD<L D= 9<C<@?D< D;< J8AK>?G'

6 How do we get to Table-based displays?!"#$"% %" &$'( )#$&*+!( &!,!"%-.#* /0%*,1 &2&$,!+!1 &23 4+!0&$+5&"+#2 #. -0$"+3+-%2!+#2&$ *%$&"+#2&$ 3&"&6&!%! Via a visual specification Arrange attributes on shelves X shelf Y shelf Layer (Z) shelf Different datasets Different attributes.89' :' ";< )=>?@8A BA<@ 8CD<@E?F<' &C?>GADA F=CAD@BFD D?H><IH?A<J J8AK>?GA J?D? HG J@?998C9 E8<>JA E@=L D;< J?D?H?A< AF;<L? =CD A;<>M<A D;@=B9;=BD D;< J8AK>?G' & 98M<C F=CE89B@?D8=C =E E8<>JA =C A;<>M<A 8A F?>><J? M8AB?> AK<F8E8F?D8=C' ";< AK<F8E8F?D8=C BC?LH89B=BA>G J<E8C<A D;?C?>GA8A?CJ M8AB?>8N?D8=C =K<@?D8=CA D= H< K<@E=@L<J HG D;< AGAD<L D= 9<C<@?D< D;< J8AK>?G'!"## $"%&#'()*+&$#, -"$.#(, %(), -"%*)$"+)$ +/! 7*6%#%*0, :(2#*=2($*- -"$.#(,$ 3(8* () *;'*)$"8

7 The result of a visual specification!"#$"% %" &$'( )#$&*+!( &!,!"%-.#* /0%*,1 &2&$,!+!1 &23 4+!0&$+5&"+#2 #. -0$"+3+-%2!+#2&$ *%$&"+#2&$ 3&"&6&!%! 7 Selection of dataset sources è layers Selection of attributes è shelves Plus relative order, number Grouping and aggregation, sorting Type of graphic shown on each pane Mapping of data to marks in the graphic.89' :' ";< )=>?@8A BA<@ 8CD<@E?F<' &C?>GADA F=CAD@BFD D?H><IH?A<J J8AK>?GA J?D? HG J@?998C9 E8<>JA E@=L D;< J?D?H?A< AF;<L? =CD= A;<>M<A D;@=B9;=BD D;< J8AK>?G' & 98M<C F=CE89B@?D8=C =E E8<>JA =C A;<>M<A 8A F?>><J? M8AB?> AK<F8E8F?D8=C' ";< AK<F8E8F?D8=C BC?LH89B=BA>G J<E8C<A D;<?C?>GA8A?CJ M8AB?>8N?D8=C =K<@?D8=CA D= H< K<@E=@L<J HG D;< AGAD<L D= 9<C<@?D< D;< J8AK>?G'!"## $"%&#'()*+&$#, -"$.#(, %(), -"%*)$"+)$ +/ #(01* $&2$*'$ +/ '3* -('(4! 7*6%#%*0, :(2#*=2($*- -"$.#(,$ 3(8* () *;'*)$"8* (0* (66&$'+%*- '+ &$")1 '(2&#(0

8 Visual Specification: High Level Three components Table configuration specification Via a table algebra Type of graphic Visual encodings

9 Types of Data Nominal =, Ordinal =,, <, > Quantitative Interval Airlines, Genre MPAA Rating, Batteries Year, Location =,, <, >, Arbitrary zero Quantitative Ratio =,, <, >,, % Sales, Profit Temperature Physical quantities 9

10 Conversion of Data Types Nominal =, Ordinal =,, <, > Quantitative Interval =,, <, >, Arbitrary zero Quantitative Ratio =,, <, >,, % Physical quantities Hot, warm, cold Temperature Well, so-so, badly Grade Score 10

11 Conversion of Data Types Nominal =, Ordinal =,, <, > Quantitative Interval =,, <, >, This paper moves nominal è ordinal and QI è QR Arbitrary zero Quantitative Ratio =,, <, >,, % Physical quantities 11

12 Table Algebra: Basic Operands Ordinal fields: interpret domain as a set that partitions table into rows and columns: Quarter = {(Qtr1),(Qtr2),(Qtr3),(Qtr4)} à Quantitative fields: treat domain as single element set and encode spatially as axes: Profit = {(Profit[-410,650])} à

13 Concatenation (+) operator Ordered union of set interpretations: Quarter + ProductType = {(Qtr1),(Qtr2),(Qtr3),(Qtr4)} + {(Coffee), (Espresso)} = {(Qtr1),(Qtr2),(Qtr3),(Qtr4),(Coffee),(Espresso)} Profit + Sales = {(Profit[-310,620]),(Sales[0,1000])}

14 Cross (x) operator Cross-product of set interpretations: Quarter x ProductType = {(Qtr1,Coffee), (Qtr1, Tea), (Qtr2, Coffee), (Qtr2, Tea), (Qtr3, Coffee), (Qtr3, Tea), (Qtr4, Coffee), (Qtr4,Tea)} ProductType x Profit =

15 Quarter x Month Nest (/) operator would create entry twelve entries for each quarter. i.e., (Qtr1, December) Quarter / Month would only create three entries per quarter based on tuples in database not semantics can be expensive to compute

16 Type of Graphic Option 1: Let the user decide Option 2: Polaris picks the appropriate one Types O-O O-Q Q-Q 16

17 Ordinal-Ordinal Graphics Typically axes variables are independent of each other and the focus is on studying some external function:! "### $%&'(&)$"*'( *' +"(,&-".&$"*' &'/ F(O1, O2) à R 17

18 /#,('-)%#,*#& $" )0( -*)/%)*$#4 % )%;+( $" -.%))('2+$)- -0$7*#& )0( '(+%)*$#-0*2 ;()7((# 5%'N()*#&.$-)- %#, 2'$"*).%)(&$'*F(, ;6 2'$,/.) )62( %#, 5%'N() IJ*&C O%LC P")(' -)/,6*#& )0( &'%20*.-4 #$)*.(- %# *#)('(-)*#& 2'$,/.)- 0%1( 0*&0 5%'N()*#&.$-)- 7*)0 +*))+( $' #$ '()/'# *# )0( "$'5 $" 2'$"*)C H$ "/')0(' *#1(-)*&%)(4 )7$ +*#N(,,*-2+%6-8 % )%;+( $" -.%))('2+$)- -0$7*#& 2'$"*) %#, 5%'N()3 *#&.$-)- "$' (%.0 -)%)( %#, % )(:) )%;+( *)(5*F*#& 2'$"*) ;6 2'$,/.) %#, -)%)( IJ*&C O;LC H0( )7$ 1*(7- %'( +*#N(, ;6 )0( -)%)( "*(+,8!" '(.$',- %'( -(+(.)(, *# (*)0(',*-2+%64 )0(# %++ '(.$',- 7*)0 )0( -%5( -)%)( 1%+/( %- )0( -(+(.)(, '(.$',- %'( 0*&0+*&0)(,C *- %;+( )$ /-( )0(-( +*#N(, 1*(7- )$,()('5*#( )0%)4 *# Q(7 R$'N4 -(1('%+ 2'$,/.)- %'( $""('*#& 1('6 +*))+( '()/'#,(-2*)( 0*&0 (:2(#,*)/'(-C )0(#.'(%)(- % )0*',,*-2+%6 IJ*&C O.L8 % -() $" ;%'.0%')- -0$7*#& 2'$"*)4 -%+(-4 %#, 5%'N()*#& "$' (%.0 2'$,/.) -$+, *# Q(7 R$'N4 ;'$N(#,$7# ;6 5$#)0C!# )0*$%&'(&)$"*'( *' +"(,&-".&$"*' &'/ )*01,$#% 2%&13")(4 +*-5 64 '*5 74 8&',&%9:0&%)3 ;<<; 1*(74 -(( "### T$.0%U2'$"*).89' :' &; <=>?@A< BC<;>D8E F<?E;BGD>G8;9 GH< C>@>I8A8G8<B EJ )EA> 5%'&*#,$(- #$) M/-)*"6 *)- 5%'N()*#& (:2(#-(-C D*)0 )0*- <=@AED>GEDK >;>AKB8B EJ?LAG8F8?<;B8E;>A F>G>I>B<B' "H< F>G> F8B@,%)%4 )0(.$52%#6U- 5%'N()*#& %#, 8B JED > HK@EGH<G8C>A CEJJ<< BGED< CH>8; >;F GH< >;>AKBG 8B B<>DCH8; M>KB GE D<FLC< GH< CE?@>;KNB <=@<;B<B' -%+(- -)'%)(&*(- *# )0*- -)%)(C Ordinal-Quantitative Graphics Bar chart (plain, clustered, stack) Dot plot (typically for dependencies F(O) à Q) Gantt chart!(; $*+,-./0 ;1 203<=6+. $: ,-4:8/8 (typically independent) P) V)%#"$',4 '(-(%'.0('-for,(1(+$2*#& P'&/- =BA?4 % 2%'%++(+ &'%20*.- +*;'%'64 "$/#, )0%) *)- 2('"$'5%#.( 0%, +*#(%' -2((,/2 70(# /-*#& /2 )$ WA 2'$.(--$'-4 %")(' 70*.0 *)2('"$'5%#.(,*5*#*-0(, '%2*,+6C X-*#& 9$+%'*-4 7( '(.'(%)( )0( %#%+6-*- )0(6 2('"$'5(, /-*#& %./-)$53;/*+) 1*-/%+*F%)*$# )$$+ =Y?C!#*)*%++64 )0(,(1(+$2('- 062$)0(-*F(, )0%) )0(,*5*#3 *-0*#& 2('"$'5%#.( 7%- % '(-/+) $" )$$ 5%#6 '(5$)( 1*-/%+*F%)*$#.$#-)'/.)(, )$,*-2+%6 )0*-,%)%C H0( 1* +*F%)*$# *-.$52$-(, $" )7$ +*#N(, 9$+%'*- *#-)%#.(-C,*-2+%6- % 0*-)$&'%5 $".%.0( 5*--(- ;6 1*')/%+ %,, )0( $)0(',*-2+%6- -$/'.(3.$,(4 7*)0 (%.0 +*#(U(#.$,*#& )0( #/5;(' $".%.0( 5*--(- -/""('(, ;6 +*#(C X2$# -((*#& )0(-(,*-2+%6-4 )0(6.$/+, )(++ 5(5$'6 7%- #$) *# "%.) )0( 2'$;+(5C H0(,(1(+$2('- #(:) 062$)0(-*F(, )0%) +$.N 18.$#)(# 5*&0) ;( % 2'$;+(54 -$ )0(6 '('%# P'&/- %#,.$++(

19 Ordinal-Quantitative Graphics 19

20 Quantitative Quantitative Graphics Scatterplots Typically for studying causal relationships Choropleths / Maps 20

21 Quantitative-Quantitative Graphics Bar chart (plain, clustered, stack) Dot plot (typically for dependencies F(O) à Q) Gantt chart (typically for independent) 21

22 Visual Mappings Mappings from records to marks Uses visual properties of the mark to encode differences: Size, shape, orientation, color 22

23 Recap Visual Specification Table spec Type of graphic Encoding of marks Now.. Additional manipulations needed for visualization composition 23

24 Data Transformations Why do we need this? Generation of new fields Simple aggregation Applied to a quant field Distinct count for an ord field Discrete partitioning Binning (discussed next) Does not change the graph type (still quant) Partitioning à Arbitrary binning Ad-hoc grouping Specific values can be grouped together Thresholding Formation of an other category 24

25 Data Transformations Binning Dealing with Large Cardinalities Per Hour Per Day Per Week Per Month Bin by hour Bin by day Bin by week Bin by month 25

26 Data Transformations: Aggregation Cumulative vs. Aggregate Normalization Logarithm Power Aid comparisons, reduce random variations 26

27 Data Transformation: Subselection, Ordering Filtering Sorting Brushing/linking Tooltips 27

28 Data Flow 28

29 Data Flow Step 1: SELECT * WHERE {filters} 29

30 Data Flow Step 2: SELECT * WHERE ROW(i) AND COLUMN(j) AND LAYER(k) 30

31 Data Flow Step 3: SELECT dim, agg GROUP BY g HAVING f ORDER BY S 31

32 Data Flow Does it make sense to split up the computation in this way? 32

33 Data Flow Other Directions for Speeding Up Materialized views Progressive refinement Sampling Caching and prefetching 33

34 OLAP Terminology Star Schema: for simplicity, consider as a single relation, with many dimensions and measures Relation of transactions Measures: values that can be measured and aggregated Sales, Profit Dimensions: independent variables Location, Category of Sale, Year 34

35 Aggregation Operators The measure attributes will be aggregated Standard SQL aggregations: COUNT, SUM, AVG, MAX/MIN, STD DEV 35

36 Canonical OLAP Query SELECT AGG(M), D FROM R WHERE GROUP BY D SELECT SUM(Sales), Category FROM R GROUP BY Category 36

37 OLAP = Data Cubes Each cell summarizes all measures for those dimension values Each cube dimension corresponds to a dimension in the relation

38 Types of Charts Bar Charts Line Charts Scatter Plot Choropleth 38

39 Bar Charts 39

40 Bar Charts Plotting a Q-R vs. either a N, O, Q-I, or Q-R Emphasize more the difference in height than the distances in the x axis Most fundamental chart 40

41 Line Charts 41

42 Line Charts Plotting a Q-R vs. a Q-I, or Q-R Mainly makes sense when the x-axis is ordered in some way Want to be able to see trends Assumption of interpolation, dependence 42

43 Scatter Plot 43

44 Scatterplot Plotting a Q-R vs. a Q-R Unlike line graphs, no assumption of interpolation Care more about density, understanding of correlation 44

45 Choropleths 45

46 Choropleth Overlaid on a map Q-R vs. Two-dimensional Q-I variable 46

47 Data Visualization Software d3 ggplot2 Google Charts Gnuplot Easier to Customizable use & learn Also, analytics software: Tableau, Spotfire 47

48 Data Visualization Software Almost all the tools described will be able to plot something like this 48

49 Data Visualization Software: d3 49

50 Data Analytics Tools: Tableau 50

51 What could the paper have improved? 51

52 What could the paper have improved? Some experiments or evaluation of approach Some notion of completeness or coverage Still none of that stopped them from publishing or even better, starting a publicly traded company! v=z5kqr71yjpe 52

53 Future Work What ways can we improve the system from a data processing standpoint? 53

54 Future Work What ways can we improve the system from a data processing standpoint? Redesign the backend database to optimize for queries of the form described Use user traces to speculate, optimize, prefetch 54