13. Geospatio-temporal Data Analytics. Jacobs University Visualization and Computer Graphics Lab

Transcription

1 13. Geospatio-temporal Data Analytics

2 Recall: Twitter Data Analytics 573

3 Recall: Twitter Data Analytics 574

4 13.1 Time Series Data Analytics

5 Introduction to Time Series Analysis A time-series is a set of observations on a quantitative variable collected over time. Examples: Dow Jones Industrial Averages Historical data on sales, inventory, customer counts, interest rates, costs, etc. Businesses are often very interested in forecasting time series variables. In time series analysis, we analyze the past behavior of a variable in order to predict its future behavior. Data Analytics 576

6 Methods used in forecasting Regression Analysis Time Series Analysis (TSA) A statistical technique that uses time-series data for explaining the past or forecasting future events. The prediction is a function of time (days, months, years, etc.) No causal variable; examine past behavior of a variable and and attempt to predict future behavior Data Analytics 577

7 Components of TSA Time Frame (How far can we predict?) short-term (1-2 periods) medium-term (5-10 periods) long-term (12+ periods) No line of demarcation Trend Gradual, long-term movement (up or down) of demand. Easiest to detect Data Analytics 578

8 Components of TSA Cycle An up-and-down repetitive movement in demand. repeats itself over a long period of time Seasonal Variation An up-and-down repetitive movement within a trend occurring periodically. Often weather related but could be daily or weekly occurrence Random Variations Erratic movements that are not predictable because they do not follow a pattern Data Analytics 579

9 Time Series Plot Actual Sales $3,000 $2,500 Sales (in $1,000s) $2,000 $1,500 $1,000 $500 $ Time Period Data Analytics 580

10 Components of TSA Difficult to forecast demand because... There are no causal variables The components (trend, seasonality, cycles, and random variation) cannot always be easily or accurately identified Data Analytics 581

11 Some Time Series Terms Stationary Data - a time series variable exhibiting no significant upward or downward trend over time. Nonstationary Data - a time series variable exhibiting a significant upward or downward trend over time. Seasonal Data - a time series variable exhibiting a repeating patterns at regular intervals over time. Data Analytics 582

12 Approaching Time Series Analysis There are many different time series techniques. It is usually impossible to know which technique will be best for a particular data set. It is customary to try out several different techniques and select the one that seems to work best. To be an effective time series modeler, you need to keep several time series techniques in your tool box. Data Analytics 583

13 Measuring Accuracy We need a way to compare different time series techniques for a given data set. Four common techniques are the: mean absolute deviation: mean absolute percent error: the mean square error: root mean square error: MAD = RMSE = n i= 1 n i= 1 Y i MSE Y$ n 100 n Yi Ŷi MAPE = n i= 1 Yi ( Y Y) i $ i MSE = n i 2 Data Analytics 584

14 Extrapolation Models Extrapolation models try to account for the past behavior of a time series variable in an effort to predict the future behavior of the variable. Y$ = f Y, Y, Y, ( ) t+ 1 t t 1 t 2 K We ll first talk about several extrapolation techniques that are appropriate for stationary data. Data Analytics 585

15 An Example Electra-City is a retail store that sells audio and video equipment for the home and car. Each month the manager of the store must order merchandise from a distant warehouse. Currently, the manager is trying to estimate how many VCRs the store is likely to sell in the next month. He has collected 24 months of data. Data Analytics 586

16 Moving averages 1 $Y = 1 1 t Y Y Y k + + k L k t t-1 t-k-1 No general method exists for determining k. We must try out several k values to see what works best. Data Analytics 587

17 Implementing the Model Data Analytics 588

18 Comment on Comparing MSE Values Care should be taken when comparing MSE values of two different forecasting techniques. The lowest MSE may result from a technique that fits older values very well but fits recent values poorly. It is sometimes wise to compute the MSE using only the most recent values. Data Analytics 589

19 Forecasting With The Moving Average Model Forecasts for time periods 25 and 26 at time period 24: Y$ Y24 + Y = = = $ $ Y25 + Y Y26 = = = Data Analytics 590

20 Weighted Moving Average The moving average technique assigns equal weight to all previous observations 1 $Y = 1 1 t Y Y Y k + + k L k t t-1 t-k-1 The weighted moving average technique allows for different weights to be assigned to previous observations. We must determine values for k and the weights w i $Y = wy + + w Y L w Y t t t-1 k t-k -1 where 0 w and i 1 wi = 1 Data Analytics 591

21 Implementing the Model Data Analytics 592

22 Optimal weights that minimize MSE Data Analytics 593

23 Forecasting with Weighted Moving Averaging Forecasts for time periods 25 and 26 at time period 24: Y$ 25 = w1y24 + w2y23 = = Ŷ = w1ŷ 25 + w2y = = Data Analytics 594

24 Exponential Smoothing Y $ $ ( $ t + 1 = Y t + α Y t Y t ) where 0 α 1 It can be shown that the above equation is equivalent to: $ 2 n Y = αy + α( 1 α) Y + α( 1 α) Y + L+ α( 1 α) Y + L t + 1 t t 1 t 2 t n Data Analytics 595

25 Examples of Exponential Smoothing Functions Units Sold Time Period Number of VCRs Sold Exp. Smoothing alpha=0.1 Exp. Smoothing alpha=0.9 Data Analytics 596

26 Implementing the Model Data Analytics 597

27 Forecasting With Exponential Smoothing Forecasts for time periods 25 and 26 at time period 24: Y$ Y$ ( Y Y$ 25 = 24 + α 24 24) = ( ) = Y$ = Y$ + α( Y Y$ ) Y$ + α( Y $ Y $ ) = Y$ = Note that, Y $ = 3581., for t = 25, 26, 27, K t Data Analytics 598

28 Seasonality Seasonality is a regular, repeating pattern in time series data. May be additive or multiplicative in nature... Data Analytics 599

29 Stationary Seasonal Effects Additive Seasonal Effects Time Period Multiplicative Seasonal Effects Time Period Data Analytics 600

30 Ŷ Stationary Data with Additive Seasonal Effects t+ n Et + where E S t t = S t+ n p = α(yt -St-p) + (1-α)Et 1 = β (Yt - Et ) + (1- β )S 0 α 1 0 β 1 t p p represents the number of seasonal periods E t is the expected level at time period t. S t is the seasonal factor for time period t. Data Analytics 601

31 Implementing the Model Data Analytics 602

32 Forecasting with Additive Seasonal Effects Model Forecasts for time periods at time period 24: Ŷ Ŷ n = E24 + S24+ n 4 = E + S = = 25 Ŷ = E + S = = 26 Ŷ = E + S = = 27 Ŷ = E + S = = Data Analytics 603

33 Stationary Data with Multiplicative Seasonal Effects Ŷ t+ n Et where E S 0 0 t t = S t+ n p = α(y t/st-p) + (1-α)Et 1 = β (Y /E ) + (1- β )S α 1 β 1 t t t p p represents the number of seasonal periods E t is the expected level at time period t. S t is the seasonal factor for time period t. Data Analytics 604

34 Implementing the Model Data Analytics 605

35 Forecasting With Multiplicative Seasonal Effects Forecasts for time periods at time period 24: Ŷ 24+ n = E24 S24+ n 4 Ŷ = E S = = 25 Ŷ = E S = = 26 Ŷ = E S = = 27 Ŷ = E S = = Data Analytics 606

36 Trend Models Trend is the long-term sweep or general direction of movement in a time series. We ll now consider some nonstationary time series techniques that are appropriate for data exhibiting upward or downward trends. Data Analytics 607

37 Example WaterCraft Inc. is a manufacturer of personal water crafts (also known as jet skis). The company has enjoyed a fairly steady growth in sales of its products. The officers of the company are preparing sales and manufacturing plans for the coming year. Forecasts are needed of the level of sales that the company expects to achieve each quarter. Data Analytics 608

38 Double Exponential Smoothing (Holt s Method) Ŷ = E + t+ n where t nt t E t = αy t + (1-α)(E t-1 + T t-1 ) T t = β(e t E t-1 ) + (1-β) T t-1 0 α 1and 0 β 1 E t is the expected base level at time period t. T t is the expected trend at time period t. n is the period-ahead forecast made in period t. Data Analytics 609

39 Implementing the Model Data Analytics 610

40 Optimal values for α and ß that minimizes MSE Data Analytics 611

41 Forecasting With Holt s Model Forecasts for time periods 21 through 24 at time period 20: Ŷ 20+ n = E20 + nt20 Y$ 21 = E20 + 1T20 = = Y$ 22 = E20 + 2T20 = = Y$ 23 = E20 + 3T20 = = Y$ = E + 4T = = Data Analytics 612

42 Ŷ Holt-Winter s Method for Additive Seasonal Effects t+ n = Et + ntt + St+ n p S t where α ( Y S ) + (1- α )(E T ) E t = t t p t 1 + t 1 T ( ) t = β Et Et 1 + (1- β )Tt 1 ( Y ) t Et + (1-γ t p = γ )S 0 α 1 0 β 1 0 γ 1 Data Analytics 613

43 Implementing the Model Data Analytics 614

44 The Linear Trend Model $Y = b + bx t 0 1 1t where X 1t = t For example: X = 1, X = 2, X = 3, K Data Analytics 615

45 Implementing the Model Data Analytics 616

46 Forecasting With The Linear Trend Model Forecasts for time periods 21 through 24 at time period 20: Y$ 21 = b0 + b1x1 21 = = Y $ X = b0 + b = = Y$ 23 = b0 + b1x1 23 = = Y $ X = b0 + b = = Data Analytics 617

47 The Quadratic Trend Model $Y = b + bx + b X t where X = tand X = t 1 2 t t t 2 t Data Analytics 618

48 Implementing the Model Data Analytics 619

49 Forecasting With Quadratic Trend Model Forecasts for time periods 21 through 24 at time period 20: Ŷ Ŷ 2 = b0 + b1x 1 + b2x 2 = = = b0 + b1x 1 + b2x 2 = = Ŷ Ŷ 2 = b0 + b1x 1 + b2x 2 = = = b0 + b1x 1 + b2x 2 = = Data Analytics 620

50 Computing Multiplicative Seasonal Indices We can compute multiplicative seasonal adjustment indices for period p as follows: S p = Y i Y $ i, for all i occuring in season i n p p The final forecast for period i is then Y $ adjusted = Y$ i i S p, for any ioccuring in season p Data Analytics 621

51 Implementing the Model Data Analytics 622

52 Implementing the Model Data Analytics 623

53 Forecasting with Seasonal Adjustments Applied to Quadratic Trend Model Forecasts for time periods 21 through 24 at time period 20: Ŷ 1 = ( b0 + b1x 1 + b2x 2 ) S = % = Ŷ 2 = ( b0 + b1x 1 + b2x 2 ) S = % = Ŷ 3 = ( b0 + b1x 1 + b2x 2 ) S = % = Ŷ 4 = ( b0 + b1x 1 + b2x 2 ) S = % = Data Analytics 624

54 Summary of the Calculation and Use of Seasonal Indices 1. Create a trend model and calculate the estimated value for each observation in the sample. 2. For each observation, calculate the ratio of the actual value to the predicted trend value: Yt / Ŷ t. (For additive effects, compute the difference: Yt Ŷ t ). 3. For each season, compute the average of the ratios calculated in Step 2. These are the seasonal indices. 4. Multiply any forecast produced by the trend model by the appropriate seasonal index calculated in Step 3. (For additive seasonal effects, add the appropriate factor to the forecast.) Ŷ t Data Analytics 625

55 Seasonal Regression Models Indicator variables may also be used in regression models to represent seasonal effects. If there are p seasons, we need p-1 indicator variables. Our example problem involves quarterly data, so p=4 and we define the following 3 indicator variables: X X 3 4 t t 1 =, 0, 1 =, 0, if Y is an observation from quarter 1 t otherwise if Y is an observation from quarter 2 t otherwise X 5 t 1 if Y t is an observation from quarter 3 =, 0, otherwise Data Analytics 626

56 Implementing the Model The regression function is: $Yt = b + bx + b X + b X + b X + b X where X = tand X = t 1 2 t t t t t t t 2 Data Analytics 627

57 Implementing the Model Data Analytics 628

58 Implementing the Model Data Analytics 629

59 Implementing the Model Data Analytics 630

60 Forecasting With Seasonal Regression Model Forecasts for time periods 21 through 24 at time period 20: 2 Ŷ21 = (21) (21) (1) (0) (0) = 2 Ŷ22 = (22) (22) (0) (1) (0) = 2 Ŷ23 = (23) (23) (0) (0) (1) = Ŷ24 = (24) (24) (0) (0) (0) = Data Analytics 631

61 Combining Forecasts It is also possible to combine forecasts to create a composite forecast. Suppose we used three different forecasting methods on a given data set. Denote the predicted value of time period t using each method as follows: F, F, and F t t t We could create a composite forecast as follows: $Yt = b + bf + b F + b F t t t Data Analytics 632

62 Visual representation The shown plots are one way to visualize a time series. There are other options. In particular, when multiple time series are to be considered simultaneously. Data Analytics 633

63 Bar charts Data Analytics 634

64 Recall: Visualization examples Data Analytics 635

65 Time lines Data Analytics 636

66 Recall: Playfair s plot of trade between England and Denmark/Norway from 1786 Data Analytics 637

67 Multiresolution time lines Data Analytics 638

68 Recall: Visualization examples Data Analytics 639

69 Stacked graphs Data Analytics 640

70 Recall: Visualization examples Data Analytics 641

71 Sparklines Data Analytics 642

72 Radial layout: spiral Data Analytics 643

73 Multivariate time series data Data Analytics 644

74 Time series in documents Data Analytics 645

75 13.2 Geospatial Data Analytics

76 Recall: Snow s map of cholera in London 1663 Data Analytics 647

77 Recall: Edge bundling Geospatial: Data Analytics 648

78 Recall: Edge bundling Geospatial: Data Analytics 649

79 The geospatial perspective A distinct perspective on the world Domain of geospatial analysis Dealing with what happens where Human computer interface Data Analytics 650

80 Basic primitives: Place Terrestrial scale - shape and size Naming of places Dynamics Need for rigor coordinate systems and data 2D vs 3D perspectives on geospatial issues Data Analytics 651

81 Basic primitives: Attributes Characteristics associated with place Nominal, ordinal, interval, etc. Attribute tables Data Analytics 652

82 Basic primitives: Objects Points Lines Areas Representative points Polylines Polygons Data Analytics 653

83 Basic primitives: Maps Communicating with maps Traditional maps Digital mapping Alternative visualisations 3D and dynamic Data Analytics 654

84 Basic primitives: Multiple properties Multiple properties of place Concept of layers Combining layers Resolving modelling and geocoding differences Data Analytics 655

85 Basic primitives: Fields Discrete-object view Continuous-field view Data modelling as a form of representation Conversion between model views Data Analytics 656

86 Basic primitives: Density Density as object count (n) divided by area (a), so d=n/a Density as link between discrete and continuous model views Sensitivity to definition of both n and a Data Analytics 657

87 Basic primitives: Spatial weights Adjacency models Proximity distance based Spatial weights matrices, W Data Analytics 658

88 Basic primitives: Networks Basic components of a network Forms of network Concept of network topology Applications of network analysis Data Analytics 659

89 Basic primitives: Topology General concepts of topology Topological dimension Adjacency Connectivity Containment Data Analytics 660

90 Basic primitives: detail, resolution, scale Intrinsic complexity of the real world Spatial resolution Representative fraction Resolution and digital datasets Temporal resolution Data Analytics 661

91 Spatial relationships: Distance & direction Location and distance/direction calculations Representative points Direction and circular measure Spatial weights matrices Determination of weights: binary and real-valued Reconstruction and MDS Data Analytics 662

92 Spatial relationships: Context Spatial context and proximity Neighbourhoods Uniform distance Zonal Weighted Heterogeneity Data Analytics 663

93 Spatial relationships: Dependence Spatial dependence Spatial autocorrelation Correlograms Geostatistics Data Analytics 664

94 Spatial relationships: Sampling & interpolation Point sampling Systematic Random Stratified Zonal sampling and statistics Fields and interpolation Data Analytics 665

95 Spatial relationships: Smoothing & sharpening Convolution Pattern and process Filtering Data Analytics 666

96 Spatial statistics: Probability & uncertainty Probability in a spatial context Marginal vs Joint probability Probability fields Probability and uncertainty Probability density Uncertainty and error propagation Uncertainty and accuracy Data Analytics 667

97 Spatial statistics: Inference Statistical inference and significance: Deduce properties of underlying distribution Confidence Controlled experiments Natural experiments Sampling & independence Samples vs populations Data Analytics 668

98 Visual representations Visualizations are often based on maps Attributes are visually encoded within such a map Data Analytics 669

99 Encoding by bars Data Analytics 670

100 Encoding by color Data Analytics 671

101 Encoding by size Data Analytics 672

102 Encoding by glyphs Data Analytics 673

103 Recall: Visualization examples Data Analytics 674

104 13.3 Spatio-temporal Patterns

105 Combination of time and space Data Analytics 676

106 Clustering Data Analytics 677

107 Similarity measure Considered points: Distance between any two points on trajectories Closest distance between points at same/similar time Symmetry: One-sided distance Two-sided distance Accumulated distance Average over trajectory Minimum over trajectory Maximum over trajectory Data Analytics 678

108 Visual analytics Visual analytics approaches may combine spatial and temporal representations in one encoding. However, typically multiple views are used. Interactions are used for correlating. Data Analytics 679

109 Recall: Minard s map of Napoleon s march on Moscow drawn in 1869 Data Analytics 680

110 Embedding time series in geospatial layout Data Analytics 681

111 Recall: Visualization examples Data Analytics 682

112 Recall: Visualization examples Data Analytics 683

113 Recall: Visualization examples Data Analytics 684

114 Recall: Visualization examples Data Analytics 685

115 Recall: Visualization examples Data Analytics 686

116 13.4 Assignment

117 Assignment 11 Consider the taxi service trajectory data set provided here: +Prediction+Challenge,+ECML+PKDD+2015 Aggregate the time stamps by the hour and by the day and create two time series for all taxi trips and for trips of types A, B, and C, respectively. You should have created 8 time series. Visualize them. For each time series, try the different TSA models presented in class for prediction (consider linear, quadratic, or no trend, additive, multiplicative, or no seasons, regression, and combinations thereof). Compute the MSE to check, which predictions worked best for which time series. What can you conclude from your findings? p.t.o. Data Analytics 688

118 Assignment 11 Check whether you are able to detect day types A and B with your analysis (of course, without using that information). How would you do that? Bonus: Visualize a density map for each of the time series that shows the spatial distribution of the taxi trajectories. Allow for filtering the density maps by the day or the hour and its repetitions, i.e., only showing the trajectories that started on day x, x+7, x+14, for x = 1,,7, or only showing the trajectories that started at hour x, x+24, x+48, for x = 1,,24. Data Analytics 689