Intro to ARMA models. FISH 507 Applied Time Series Analysis. Mark Scheuerell 15 Jan 2019

Size: px

Start display at page:

Download "Intro to ARMA models. FISH 507 Applied Time Series Analysis. Mark Scheuerell 15 Jan 2019"

Kelly Fox
5 years ago
Views:

1 Intro to ARMA models FISH 507 Applied Time Series Analysis Mark Scheuerell 15 Jan 2019

2 Topics for today Review White noise Random walks Autoregressive (AR) models Moving average (MA) models Autoregressive moving average (ARMA) models Using ACF & PACF for model ID 2/70

3 White noise (WN) A time series is discrete white noise if its values are 1. independent 2. identically distributed with a mean of zero The distributional form for is flexible 3/70

4 White noise (WN) 4/70

5 Gaussian white noise We often assume so-called Gaussian white noise, whereby and the following apply as well autocovariance: autocorrelation: 5/70

6 Gaussian white noise 6/70

7 Random walk (RW) A time series is a random walk if is white noise 7/70

8 Random walk (RW) 8/70

9 Biased random walk A biased random walk (or random walk with drift) is written as where is the bias (drift) per time step and is white noise 9/70

10 Biased random walk 10/70

11 Differencing a biased random walk First-differencing a biased random walk yields a constant mean (level) plus white noise 11/70

12 Differencing a biased random walk 12/70

13 LINEAR STATIONARY MODELS 13/70

14 Linear stationary models We saw last week that linear filters are a useful way of modeling time series Here we extend those ideas to a general class of models call autoregressive moving average (ARMA) models 14/70

15 Autoregressive (AR) models Autoregressive models are widely used in ecology to treat a current state of nature as a function its past state(s) 15/70

16 Autoregressive (AR) models An autoregressive model of order p, or AR(p), is defined as where we assume 1. is white noise 2. for an order-p process 16/70

17 Examples of AR(p) models AR(1) AR(1) with (random walk) AR(2) 17/70

18 Examples of AR(p) models 18/70

19 Stationary AR(p) models Recall that stationary processes have the following properties 1. no systematic change in the mean or variance 2. no systematic trend 3. no periodic variations or seasonality We seek a means for identifying whether our AR(p) models are also stationary 19/70

20 Stationary AR(p) models We can write out an AR(p) model using the backshift operator 20/70

21 Stationary AR(p) models If we treat as a number (or numbers), we can out write the characteristic equation as To be stationary, all roots of the characteristic equation must exceed 1 in absolute value 21/70

22 Stationary AR(p) models For example, consider this AR(1) model from earlier 22/70

23 Stationary AR(p) models For example, consider this AR(1) model from earlier This model is indeed stationary because 23/70

24 Stationary AR(p) models What about this AR(2) model from earlier? 24/70

25 Stationary AR(p) models What about this AR(2) model from earlier? This model is not stationary because only one 25/70

26 What about random walks? Consider our random walk model 26/70

27 What about random walks? Consider our random walk model Random walks are not stationary because 27/70

28 Stationary AR(1) models We can define a space over which all AR(1) models are stationary 28/70

29 Stationary AR(1) models For, we have 29/70

30 Stationary AR(1) models For, we have For, we have 30/70

31 Stationary AR(1) models Thus, AR(1) models are stationary if and only if 31/70

32 Coefficients of AR(1) models Same value, but different sign 32/70

33 Coefficients of AR(1) models Both positive, but different magnitude 33/70

34 Autocorrelation function (ACF) Recall that the autocorrelation function ( ) measures the correlation between and a shifted version of itself 34/70

35 ACF for AR(1) models ACF oscillates for model with 35/70

36 ACF for AR(1) models For model with large, ACF has longer tail 36/70

37 Partial autocorrelation funcion (PACF) Recall that the partial autocorrelation function ( ) measures the correlation between and a shifted version of itself, with the linear dependence of removed 37/70

38 ACF & PACF for AR(p) models 38/70

39 PACF for AR(p) models Do you see the link between the order p and lag k? 39/70

40 Using ACF & PACF for model ID Model ACF PACF AR(p) Tails off slowly Cuts off after lag p 40/70

41 Moving average (MA) models Moving average models are most commonly used for forecasting a future state 41/70

42 Moving average (MA) models A moving average model of order q, or MA(q), is defined as where is white noise Each of the is a sum of the most recent error terms 42/70

43 Moving average (MA) models A moving average model of order q, or MA(q), is defined as where is white noise Each of the is a sum of the most recent error terms Thus, all MA processes are stationary because they are finite sums of stationary WN processes 43/70

44 Examples of MA(q) models 44/70

45 Examples of MA(q) models 45/70

46 AR(p) model as an MA( ) model It is possible to write an AR(p) model as an MA( ) model 46/70

47 AR(1) model as an MA( ) model For example, consider an AR(1) model 47/70

48 AR(1) model as an MA( ) model If our AR(1) model is stationary, then so 48/70

49 Invertible MA(q) models An MA(q) process is invertible if it can be written as a stationary autoregressive process of infinite order without an error term 49/70

50 Invertible MA(1) model For example, consider an MA(1) model 50/70

51 Invertible MA(1) model If we constrain, then and 51/70

52 Invertible MA(q) models Q: Why do we care if an MA(q) model is invertible? A: It helps us identify the model's parameters 52/70

53 Invertible MA(q) models For example, these MA(1) models are equivalent 53/70

54 ACF & PACF for MA(q) models 54/70

55 ACF for MA(q) models Do you see the link between the order q and lag k? 55/70

56 Using ACF & PACF for model ID Model ACF PACF AR(p) Tails off slowly Cuts off after lag p MA(q) Cuts off after lag q Tails off slowly 56/70

57 Using ACF & PACF for model ID 57/70

58 Autoregressive moving average models An autoregressive moving average, or ARMA(p,q), model is written as 58/70

59 Autoregressive moving average models We can write an ARMA(p,q) model using the backshift operator ARMA models are stationary if all roots of ARMA models are invertible if all roots of 59/70

60 Examples of ARMA(p,q) models 60/70

61 ACF for ARMA(p,q) models 61/70

62 PACF for ARMA(p,q) models 62/70

63 Using ACF & PACF for model ID Model ACF PACF AR(p) Tails off slowly Cuts off after lag p MA(q) Cuts off after lag q Tails off slowly ARMA(p,q) Tails off slowly Tails off slowly 63/70

64 NONSTATIONARY MODELS 64/70

65 Autoregressive integrated moving average (ARIMA) models If the data do not appear stationary, differencing can help This leads to the class of autoregressive integrated moving average (ARIMA) models ARIMA models are indexed with orders (p,d,q) where d indicates the order of differencing 65/70

66 ARIMA(p,d,q) models For, is an ARIMA(p,d,q) process if is an ARMA(p,q) process 66/70

67 ARIMA(p,d,q) models For, is an ARIMA(p,d,q) process if is an ARMA(p,q) process For example, if is an ARIMA(1,1,0) process then is an ARMA(1,0) = AR(1) process 67/70

68 ARIMA(p,d,q) models 68/70

69 ARIMA(p,d,q) models 69/70

70 Topics for today Review White noise Random walks Autoregressive (AR) models Moving average (MA) models Autoregressive moving average (ARMA) models Using ACF & PACF for model ID 70/70

Aaron Daniel Chia Huang Licai Huang Medhavi Sikaria Signal Processing: Forecasting and Modeling

Aaron Daniel Chia Huang Licai Huang Medhavi Sikaria Signal Processing: Forecasting and Modeling Abstract Forecasting future events and statistics is problematic because the data set is a stochastic, rather