Visual Analytics. Visualizing multivariate data:

Transcription

1 Visual Analytics 1 Visualizing multivariate data: High density time-series plots Scatterplot matrices Parallel coordinate plots Temporal and spectral correlation plots Box plots Wavelets Radar and /or polar plots Others Demonstrations of DVAtool and Matlab examples

2 Visualizing Multivariate Data 2 Many statistical analyses involve only two variables: a predictor variable and a response variable. Such data are easy to visualize using 2D scatter plots, bivariate histograms, boxplots, etc. It's also possible to visualize trivariate data with 3D scatter plots, or 2D scatter plots with a third variable encoded with, for example color. However, many datasets involve a larger number of variables, making direct visualization more difficult. This demo explores some of the ways to visualize highdimensional data in MATLAB, using the Statistics Toolbox. Contents High Density plots Scatterplot Matrices Parallel Coordinates Plots In this demo, we'll use the carbig dataset, a dataset that contains various measured variables for about 400 automobiles from the 1970's and 1980's. We'll illustrate multivariate visualization using the values for fuel efficiency (in miles per gallon, MPG), acceleration (time from 0-60MPH in sec), engine displacement (in cubic inches), weight, and horsepower. We'll use the number of cylinders to group observations. >>load carbig >>X = [MPG,Acceleration,Displacement,Weight,Horsepower]; >>varnames = {'MPG'; 'Acceleration'; 'Displacement'; 'Weight'; 'Horsepower'}; Can do a simple graphical plot as follows: gplotmatrix(x); But this plot is not as revealing or informative as the next plot using the same gplotmatrix function. Scatterplot Matrices

3 3 Viewing slices through lower dimensional subspaces is one way to partially work around the limitation of two or three dimensions. For example, we can use the gplotmatrix function to display an array of all the bivariate scatterplots between our five variables, along with a univariate histogram for each variable. >>figure >>gplotmatrix(x,[],cylinders,['c' 'b' 'm' 'g' 'r'],[],[],false); >>text([ ], repmat(-.1,1,5), varnames, 'FontSize',11); >>text(repmat(-.12,1,5), [ ], varnames, 'FontSize',11, 'Rotation',90); The points in each scatterplot are color-coded by the number of cylinders: blue for 4 cylinders, green for 6, and red for 8. There is also a handful of 5 cylinder cars, and rotary-engined cars are listed as having 3 cylinders. This array of plots makes it easy to pick out patterns in the relationships between pairs of variables. However, there may be important patterns in higher dimensions, and those are not easy to recognize in this plot. Parallel Coordinates Plots The scatterplot matrix only displays bivariate relationships. However, there are other alternatives that display all the variables together, allowing you to investigate higher-dimensional relationships among variables. The most straight-forward multivariate plot is the parallel coordinates plot. In this plot, the coordinate axes are all laid out horizontally, instead of using orthogonal axes as in the usual Cartesian graph. Each observation is represented in the plot as a series of connected line

4 segments. For example, we can make a plot of all the cars with 4, 6, or 8 cylinders, and color observations by group. 4 >>Cyl468 = ismember(cylinders,[4 6 8]); >>parallelcoords(x(cyl468,:), 'group',cylinders(cyl468),... 'standardize','on', 'labels',varnames) The horizontal direction in this plot represents the coordinate axes, and the vertical direction represents the data. Each observation consists of measurements on five variables, and each measurement is represented as the height at which the corresponding line crosses each coordinate axis. Because the five variables have widely different ranges, this plot was made with standardized values, where each variable has been standardized to have zero mean and unit variance. With the color coding, the graph shows, for example, that 8 cylinder cars typically have low values for MPG and acceleration, and high values for displacement, weight, and horsepower. Even with color coding by group, a parallel coordinates plot with a large number of observations can be difficult to read. We can also make a parallel coordinates plot where only the median and quartiles (25% and 75% points) for each group are shown. This makes the typical differences and similarities among groups easier to distinguish. On the other hand, it may be the outliers for each group that are most interesting, and this plot does not show them at all. >>parallelcoords(x(cyl468,:), 'group',cylinders(cyl468), 'standardize','on',

5 5 'labels',varnames, 'quantile',.25) Here are examples of High density and correlation colour plots. Such plots allow one to visualize multivariate relationships at work. These can also be experienced using hands-on software tools such as the DVAtool developed by the U of Alberta group.

6 Tag names Time Trends Samples The above plot has a scroll feature so the relationships (or correlation or cause and effect analysis) can be explored. According to Shewhart, data should never be buried, it should be plotted and examined visually. The above data is from a refinery that had experienced plant-wide oscillations. The root cause of the oscillations was difficult to diagnose. A simplified schematic of the refinery appears below: The colour-coded temporal correlation plot of the data appears below. It is important to remember that even if 2 variables are oscillating at the same

7 7 frequency, their correlation can be zero if they are phase shifted by 90 degrees, that is if they are orthogonal. Thus one has to be careful when looking at correlation analysis in the time domain. Ideally such analysis should be carried out on lagadjusted variables. The following plots illustrate this concept. Information overload?? Imagine looking at thousands of numerical values of data! Looking for a needle in a haystack? Without a proper data analysis tool, such an exercise will generate more heat than light! ARAMCO talk: 2014 Here is the temporal correlation map of this same data: 41 Variables Correlation Color Map Variables Even though many variables appear to be oscillating at the same frequency, the temporal correlation plot is only able to identify a high degree of correlation between a few variables. There should be more than 4 variables that cluster

8 8 together as evident from the parallel temporal and spectral analysis of the data as shown below. The corresponding colour-coded correlation plot with all highly correlated spectral shapes clustered together appears below:

9 Variables Power Spectral Correlation Map Variables The plot essentially shows that many variables have very similar or almost identical spectral shapes and are therefore clustered together Also explore the following graphics plus 3D plots in Matlab and other software. Distributions as a way to visualize descriptive statistics

10 10 The distribution of the univariate data string is always insightful. It gives a picture of where the data lies, whether the data is symmetric or not and also which metric to use to describe the gross behavior of the process: mean, mode or median? (Figure from Wikipedia) When the distribution is symmetric, the mean, median and the mode are identical. The mode corresponds to the most frequent observation. The median gives the location such that 50% of the observations will be above and 50% will be below this point. The mean is the arithmetic average of all the observations. In the same way the dispersion of the distributions should be carefully considered.

11 11 Box plots:

12 12 In descriptive statistics, a box plot or boxplot is a convenient way of graphically depicting groups of numerical data through their quartiles. Box plots may also have lines extending vertically from the boxes (whiskers) indicating variability outside the upper and lower quartiles, hence the terms box-and-whisker plot and box-andwhisker diagram. Outliers may be plotted as individual points. Box plots are nonparametric: they display variation in samples of a statistical population without making any assumptions of the underlying statistical distribution. The spacings between the different parts of the box indicate the degree of dispersion (spread) and skewness in the data, and show outliers. In addition to the points themselves, they allow one to visually estimate various L-estimators, notably the interquartile range, midhinge, range, mid-range, and trimean. Box plots can be drawn either horizontally or vertically.

13 13 Radar or polar plots: Tag1 Tag2 Tag3 Tag4 Tag5 Tag6 Tag7 Tag8 Tag9 Tag10 Tag11 Tag12 Tag13 Tag14 Tag15 Tag16 Tag17 Tag18 Tag19 Tag20 Tag21 Tag22 Tag23 Tag24 Tag25 Tag26 Tag27 Tag28 Tag29 Tag30 Tag31 Tag32 Tag33 Tag34 Tag35 Tag36 Tag37 Tag38 Tag39 Tag40 Tag41 Tag42 Tag43 Tag44 Tag45 Tag46 Tag47 Tag49 Tag48 Tag50 Tag51 Tag52 Tag53 Tag54 Tag55 Tag57 Tag58 Tag59 Tag56 Tag60 Tag61 Tag62 Tag63 Tag64 Tag65 Tag66 Tag67 Tag68 Tag69 Tag70 Tag71 Tag72 Tag73 Tag74 Tag75 Tag76 Tag77 Tag78 Tag80 Tag79 Tag81 Tag82 Tag83 Tag84 Tag85 Tag86 Tag87 Tag88 Tag89 Tag90 Tag91 Tag92 Tag93 Tag94 Tag95 Tag96 Tag97 Tag98 Tag99 Tag100 Alarm Count Tag1 Tag3 Tag2 Tag5 Tag6 Tag9 Tag10 Tag7 Tag8 Tag11 Tag15 Tag16 Tag17 Tag19 Tag20 Tag13 Tag12 Tag22 Tag21 Tag24 Tag26 Tag23 Tag27 Tag34 Tag30 Tag37 Tag38 Tag40 Tag43 Tag47 Tag35 Tag36 Tag48 Tag50 Tag49 Tag42 Tag32 Tag63 Tag39 Tag41 Tag68 Tag75 Tag80 Tag45 Tag74 Tag78 Tag83 Tag87 Tag88 Tag44 Tag76 Tag84 Tag85 Tag91 Tag94 Tag96 Tag98 Tag99 Tag93 Tag14 Tag79 Tag100 Tag29 Tag51 Tag58 Tag59 Tag70 Tag77 Tag95 Tag64 Tag65 Tag57 Tag62 Tag71 Tag86 Tag92 Tag60 Tag66 Tag97 Tag54 Tag90 Tag61 Tag53 Tag69 Tag73 Tag89 Tag81 Tag33 Tag55 Tag52 Tag72 Tag56 Tag28 Tag82 Tag18 Tag67 Tag31 Tag46 Tag4 Tag25 Alarm Count

14 14 Diagnosis of Plant-Wide Oscillations Time Trends Samples

15 15 SEA Refinery Data Set

16 16 Power Spectral Correlation Map

17 17 Tags Corresponding to the 1 st group

18 18 Continuous Wavelet Example Signal made of three frequencies (0.05, 0.2 and 0.1 Hz) each present in 128 consecutive samples.

19 19 Application of Wavelets