Time Series Analysis by State Space Methods

Size: px
Start display at page:

Download "Time Series Analysis by State Space Methods"

Transcription

1 Time Series Analysis by State Space Methods Second Edition J. Durbin London School of Economics and Political Science and University College London S. J. Koopman Vrije Universiteit Amsterdam OXFORD UNIVERSITY PRESS

2 Contents Introduction Basic ideas of state space analysis Linear models Non-Gaussian and nonlinear models Prior knowledge Notation Other books on state space methods Website for the book 6 PARTI THE LINEAR STATE SPACE MODEL 2. Local level model Introduction Filtering The Kalman filter Regression lemma Bayesian treatment Minimum variance linear unbiased treatment Illustration Forecast errors Cholesky decomposition Error recursions State smoothing Smoothed state Smoothed state variance Illustration Disturbance smoothing Smoothed observation disturbances Smoothed state disturbances Illustration Cholesky decomposition and smoothing Simulation Illustration Missing observations Illustration 30

3 xiv Contents 2.8 Forecasting Illustration Initialisation Parameter estimation Loglikelihood evaluation Concentration of loglikelihood Illustration Steady state Diagnostic checking Diagnostic tests for forecast errors Detection of outliers and structural breaks Illustration Exercises Linear state space models Introduction Univariate structural time series models Trend component Seasonal component Basic structural time series model Cycle component Explanatory variables and intervention effects STAMP Multivariate structural time series models Homogeneous models Common levels Latent risk model ARMA models and ARIMA models Exponential smoothing Regression models Regression with time-varying coefficients Regression with ARMA errors Dynamic factor models State space models in continuous time Local level model Local linear trend model Spline smoothing Spline smoothing in discrete time Spline smoothing in continuous time Further comments on state space analysis State space versus Box-Jenkins approaches Benchmarking Simultaneous modelling of series from different sources Exercises 74

4 Contents xv 4. Filtering, smoothing and forecasting Introduction Basic results in multivariate regression theory Filtering Derivation of the Kalman filter Kalman filter recursion Kalman filter for models with mean adjustments Steady state State estimation errors and forecast errors State smoothing Introduction Smoothed state vector Smoothed state variance matrix State smoothing recursion Updating smoothed estimates Fixed-point and fixed-lag smoothers Disturbance smoothing Smoothed disturbances Smoothed disturbance variance matrices Disturbance smoothing recursion Other state smoothing algorithms Classical state smoothing Fast state smoothing The Whittle relation between smoothed estimates Two filter formula for smoothing Covariance matrices of smoothed estimators Weight functions Introduction Filtering weights Smoothing weights Simulation smoothing Simulation smoothing by mean corrections Simulation smoothing for the state vector de Jong-Shephard method for simulation of disturbances Missing observations Forecasting Dimensionality of observational vector Matrix formulations of basic results State space model in matrix form Matrix expression for densities Filtering in matrix form: Cholesky decomposition Smoothing in matrix form Matrix expressions for signal Simulation smoothing Exercises 121

5 xvi Contents 5. Initialisation of filter and smoother Introduction The exact initial Kalman filter ' The basic recursions Transition to the usual Kalman filter A convenient representation Exact initial state smoothing Smoothed mean of state vector Smoothed variance of state vector Exact initial disturbance smoothing Exact initial simulation smoothing Modifications for diffuse initial conditions Exact initial simulation smoothing Examples of initial conditions for some models Structural time series models Stationary ARMA models Nonstationary ARIMA models Regression model with ARMA errors Spline smoothing Augmented Kalman filter and smoother Introduction Augmented Kalman filter Filtering based on the augmented Kalman filter Illustration: the local linear trend model Comparisons of computational efficiency Smoothing based on the augmented Kalman filter Further computational aspects Introduction Regression estimation Introduction Inclusion of coefficient vector in state vector Regression estimation by augmentation Least squares and recursive residuals Square root filter and smoother Introduction Square root form of variance updating Givens rotations Square root smoothing Square root filtering and initialisation Illustration: local linear trend model Univariate treatment of multivariate series Introduction Details of univariate treatment 155

6 Contents xvii Correlation between observation equations Computational efficiency Illustration: vector splines Collapsing large observation vectors Introduction Collapse by transformation A generalisation of collapsing by transformation Computational efficiency Filtering and smoothing under linear restrictions Computer packages for state space methods Introduction SsfPack The basic SsfPack functions The extended SsfPack functions 166, Illustration: spline smoothing 167 Maximum likelihood estimation of parameters Introduction Likelihood evaluation Loglikelihood when initial conditions are known Diffuse loglikelihood Diffuse loglikelihood via augmented Kalman filter Likelihood when elements of initial state vector are fixed but unknown Likelihood when a univariate treatment of multivariate series is employed Likelihood when the model contains regression effects Likelihood when large observation vector is collapsed Parameter estimation Introduction Numerical maximisation algorithms The score vector The EM algorithm Estimation when dealing with diffuse initial conditions Large sample distribution of estimates Effect of errors in parameter estimation Goodness of fit Diagnostic checking 188 Illustrations of the use of the linear model Introduction Structural time series models Bivariate structural time series analysis Box-Jenkins analysis Spline smoothing Dynamic factor analysis 202

7 xviii Contents PART II NON-GAUSSIAN AND NONLINEAR STATE SPACE MODELS 9. Special cases of nonlinear and non-gaussian models Introduction Models with a linear Gaussian signal Exponential family models Poisson density Binary density Binomial density Negative binomial density Multinomial density Multivariate extensions Heavy-tailed distributions t-distribution Mixture of normals General error distribution Stochastic volatility models Multiple volatility factors Regression and fixed effects Heavy-tailed disturbances Additive noise Leverage effects Stochastic volatility in mean Multivariate SV models Generalised autoregressive conditional heteroscedasticity Other financial models Durations: exponential distribution Trade frequencies: Poisson distribution Credit risk models Nonlinear models Approximate filtering and smoothing Introduction The extended Kalman filter A multiplicative trend-cycle decomposition Power growth model The unscented Kalman filter The unscented transformation Derivation of the unscented Kalman filter Further developments of the unscented transform Comparisons between EKF and UKF Nonlinear smoothing Extended smoothing Unscented smoothing 237

8 Contents xix 10.5 Approximation via data transformation Partly multiplicative decompositions Stochastic volatility model Approximation via mode estimation _ Mode estimation for the linear Gaussian model Mode estimation for model with linear Gaussian signal Mode estimation by linearisation Mode estimation for exponential family models Mode estimation for stochastic volatility model Further advances in mode estimation Linearisation based on the state vector Linearisation for linear state equations Linearisation for nonlinear models Linearisation for multiplicative models An optimal property for the mode Treatments for heavy-tailed distributions Mode estimation for models with heavy-tailed densities Mode estimation for state errors with ^-distribution A simulation treatment for ^-distribution model A simulation treatment for mixture of normals model Importance sampling for smoothing Introduction Basic ideas of importance sampling Choice of an importance density Implementation details of importance sampling Introduction Practical implementation of importance sampling Antithetic variables Diffuse initialisation Estimating functions of the state vector Estimating mean functions Estimating variance functions Estimating conditional densities Estimating conditional distribution functions Forecasting and estimating with missing observations Estimating loglikelihood and parameters Estimation of likelihood Maximisation of loglikelihood Variance matrix of maximum likelihood estimate Effect of errors in parameter estimation Mean square error matrix due to simulation Importance sampling weights and diagnostics 275

9 xx Contents 12. Particle filtering Introduction Filtering by importance sampling Sequential importance sampling Introduction Recursions for particle filtering Degeneracy and resampling Algorithm for sequential importance sampling The bootstrap particle filter Introduction The bootstrap filter Algorithm for bootstrap filter Illustration: local level model for Nile data The auxiliary particle filter Algorithm for auxiliary filter Illustration: local level model for Nile data Other implementations of particle filtering Importance density from extended or unscented filter The local regression filter The mode equalisation filter Rao-Blackwellisation Introduction The Rao-Blackwellisation technique Bayesian estimation of parameters Introduction Posterior analysis for linear Gaussian model Posterior analysis based on importance sampling Non-informative priors Posterior analysis for a nonlinear non-gaussian model Posterior analysis of functions of the state vector Computational aspects of Bayesian analysis Posterior analysis of parameter vector Markov chain Monte Carlo methods Non-Gaussian and nonlinear illustrations Introduction Nonlinear decomposition: UK visits abroad Poisson density: van drivers killed in Great Britain Heavy-tailed density: outlier in gas consumption 317

10 Contents xxi 14.5 Volatility: pound/dollar daily exchange rates Data transformation analysis Estimation via importance sampling Particle filtering illustration Binary density: Oxford-Cambridge boat race 324 References 326 Author Index 340 Subject Index 343

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer Ludwig Fahrmeir Gerhard Tute Statistical odelling Based on Generalized Linear Model íecond Edition. Springer Preface to the Second Edition Preface to the First Edition List of Examples List of Figures

More information

Handbook of Statistical Modeling for the Social and Behavioral Sciences

Handbook of Statistical Modeling for the Social and Behavioral Sciences Handbook of Statistical Modeling for the Social and Behavioral Sciences Edited by Gerhard Arminger Bergische Universität Wuppertal Wuppertal, Germany Clifford С. Clogg Late of Pennsylvania State University

More information

Random Number Generation and Monte Carlo Methods

Random Number Generation and Monte Carlo Methods James E. Gentle Random Number Generation and Monte Carlo Methods With 30 Illustrations Springer Contents Preface vii 1 Simulating Random Numbers from a Uniform Distribution 1 1.1 Linear Congruential Generators

More information

STATISTICS (STAT) Statistics (STAT) 1

STATISTICS (STAT) Statistics (STAT) 1 Statistics (STAT) 1 STATISTICS (STAT) STAT 2013 Elementary Statistics (A) Prerequisites: MATH 1483 or MATH 1513, each with a grade of "C" or better; or an acceptable placement score (see placement.okstate.edu).

More information

Stochastic Simulation: Algorithms and Analysis

Stochastic Simulation: Algorithms and Analysis Soren Asmussen Peter W. Glynn Stochastic Simulation: Algorithms and Analysis et Springer Contents Preface Notation v xii I What This Book Is About 1 1 An Illustrative Example: The Single-Server Queue 1

More information

Tracking Algorithms. Lecture16: Visual Tracking I. Probabilistic Tracking. Joint Probability and Graphical Model. Deterministic methods

Tracking Algorithms. Lecture16: Visual Tracking I. Probabilistic Tracking. Joint Probability and Graphical Model. Deterministic methods Tracking Algorithms CSED441:Introduction to Computer Vision (2017F) Lecture16: Visual Tracking I Bohyung Han CSE, POSTECH bhhan@postech.ac.kr Deterministic methods Given input video and current state,

More information

STAMP 5.0: A Review. Francis X. Diebold, Lorenzo Giorgianni, and Atsushi Inoue

STAMP 5.0: A Review. Francis X. Diebold, Lorenzo Giorgianni, and Atsushi Inoue STAMP 5.0: A Review Francis X. Diebold, Lorenzo Giorgianni, and Atsushi Inoue Department of Economics University of Pennsylvania 3718 Locust Walk Philadelphia, PA 19104 February 1996 Acknowledgments: We

More information

Analysis of Panel Data. Third Edition. Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS

Analysis of Panel Data. Third Edition. Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS Analysis of Panel Data Third Edition Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS Contents Preface to the ThirdEdition Preface to the Second Edition Preface to the First Edition

More information

Dynamic Models with R

Dynamic Models with R Dynamic Models with R Book Summary State space models serve as a good entry point in to modern Bayesian inference for time series. The authors in the preface mention that State space models provide a very

More information

Analysis of Incomplete Multivariate Data

Analysis of Incomplete Multivariate Data Analysis of Incomplete Multivariate Data J. L. Schafer Department of Statistics The Pennsylvania State University USA CHAPMAN & HALL/CRC A CR.C Press Company Boca Raton London New York Washington, D.C.

More information

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1 Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches

More information

A Beginner's Guide to. Randall E. Schumacker. The University of Alabama. Richard G. Lomax. The Ohio State University. Routledge

A Beginner's Guide to. Randall E. Schumacker. The University of Alabama. Richard G. Lomax. The Ohio State University. Routledge A Beginner's Guide to Randall E. Schumacker The University of Alabama Richard G. Lomax The Ohio State University Routledge Taylor & Francis Group New York London About the Authors Preface xv xvii 1 Introduction

More information

JMP Book Descriptions

JMP Book Descriptions JMP Book Descriptions The collection of JMP documentation is available in the JMP Help > Books menu. This document describes each title to help you decide which book to explore. Each book title is linked

More information

Statistics (STAT) Statistics (STAT) 1. Prerequisites: grade in C- or higher in STAT 1200 or STAT 1300 or STAT 1400

Statistics (STAT) Statistics (STAT) 1. Prerequisites: grade in C- or higher in STAT 1200 or STAT 1300 or STAT 1400 Statistics (STAT) 1 Statistics (STAT) STAT 1200: Introductory Statistical Reasoning Statistical concepts for critically evaluation quantitative information. Descriptive statistics, probability, estimation,

More information

Generalized Additive Models

Generalized Additive Models :p Texts in Statistical Science Generalized Additive Models An Introduction with R Simon N. Wood Contents Preface XV 1 Linear Models 1 1.1 A simple linear model 2 Simple least squares estimation 3 1.1.1

More information

COPYRIGHTED MATERIAL CONTENTS

COPYRIGHTED MATERIAL CONTENTS PREFACE ACKNOWLEDGMENTS LIST OF TABLES xi xv xvii 1 INTRODUCTION 1 1.1 Historical Background 1 1.2 Definition and Relationship to the Delta Method and Other Resampling Methods 3 1.2.1 Jackknife 6 1.2.2

More information

CHAPTER 1 INTRODUCTION

CHAPTER 1 INTRODUCTION Introduction CHAPTER 1 INTRODUCTION Mplus is a statistical modeling program that provides researchers with a flexible tool to analyze their data. Mplus offers researchers a wide choice of models, estimators,

More information

Probabilistic Robotics

Probabilistic Robotics Probabilistic Robotics Sebastian Thrun Wolfram Burgard Dieter Fox The MIT Press Cambridge, Massachusetts London, England Preface xvii Acknowledgments xix I Basics 1 1 Introduction 3 1.1 Uncertainty in

More information

Introduction to Mobile Robotics

Introduction to Mobile Robotics Introduction to Mobile Robotics Gaussian Processes Wolfram Burgard Cyrill Stachniss Giorgio Grisetti Maren Bennewitz Christian Plagemann SS08, University of Freiburg, Department for Computer Science Announcement

More information

book 2014/5/6 15:21 page v #3 List of figures List of tables Preface to the second edition Preface to the first edition

book 2014/5/6 15:21 page v #3 List of figures List of tables Preface to the second edition Preface to the first edition book 2014/5/6 15:21 page v #3 Contents List of figures List of tables Preface to the second edition Preface to the first edition xvii xix xxi xxiii 1 Data input and output 1 1.1 Input........................................

More information

Modelling and Quantitative Methods in Fisheries

Modelling and Quantitative Methods in Fisheries SUB Hamburg A/553843 Modelling and Quantitative Methods in Fisheries Second Edition Malcolm Haddon ( r oc) CRC Press \ y* J Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of

More information

STATISTICS (STAT) 200 Level Courses. 300 Level Courses. Statistics (STAT) 1

STATISTICS (STAT) 200 Level Courses. 300 Level Courses. Statistics (STAT) 1 Statistics (STAT) 1 STATISTICS (STAT) 200 Level Courses STAT 250: Introductory Statistics I. 3 credits. Elementary introduction to statistics. Topics include descriptive statistics, probability, and estimation

More information

Institute for Statics und Dynamics of Structures Fuzzy Time Series

Institute for Statics und Dynamics of Structures Fuzzy Time Series Institute for Statics und Dynamics of Structures Fuzzy Time Series Bernd Möller 1 Description of fuzzy time series 2 3 4 5 Conclusions Folie 2 von slide422 1 Description of fuzzy time series 2 3 4 5 Conclusions

More information

Sequential Monte Carlo Methods in Practice

Sequential Monte Carlo Methods in Practice Arnaud Doucet Nando de Freitas Neil Gordon Editors Sequential Monte Carlo Methods in Practice Foreword by Adrian Smith With 168 Illustrations Springer Contents Foreword Acknowledgments Contributors v vii

More information

Real Coded Genetic Algorithm Particle Filter for Improved Performance

Real Coded Genetic Algorithm Particle Filter for Improved Performance Proceedings of 2012 4th International Conference on Machine Learning and Computing IPCSIT vol. 25 (2012) (2012) IACSIT Press, Singapore Real Coded Genetic Algorithm Particle Filter for Improved Performance

More information

Intro to ARMA models. FISH 507 Applied Time Series Analysis. Mark Scheuerell 15 Jan 2019

Intro to ARMA models. FISH 507 Applied Time Series Analysis. Mark Scheuerell 15 Jan 2019 Intro to ARMA models FISH 507 Applied Time Series Analysis Mark Scheuerell 15 Jan 2019 Topics for today Review White noise Random walks Autoregressive (AR) models Moving average (MA) models Autoregressive

More information

Contents. Preface to the Second Edition

Contents. Preface to the Second Edition Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................

More information

PATTERN CLASSIFICATION AND SCENE ANALYSIS

PATTERN CLASSIFICATION AND SCENE ANALYSIS PATTERN CLASSIFICATION AND SCENE ANALYSIS RICHARD O. DUDA PETER E. HART Stanford Research Institute, Menlo Park, California A WILEY-INTERSCIENCE PUBLICATION JOHN WILEY & SONS New York Chichester Brisbane

More information

Overview. EECS 124, UC Berkeley, Spring 2008 Lecture 23: Localization and Mapping. Statistical Models

Overview. EECS 124, UC Berkeley, Spring 2008 Lecture 23: Localization and Mapping. Statistical Models Introduction ti to Embedded dsystems EECS 124, UC Berkeley, Spring 2008 Lecture 23: Localization and Mapping Gabe Hoffmann Ph.D. Candidate, Aero/Astro Engineering Stanford University Statistical Models

More information

L10. PARTICLE FILTERING CONTINUED. NA568 Mobile Robotics: Methods & Algorithms

L10. PARTICLE FILTERING CONTINUED. NA568 Mobile Robotics: Methods & Algorithms L10. PARTICLE FILTERING CONTINUED NA568 Mobile Robotics: Methods & Algorithms Gaussian Filters The Kalman filter and its variants can only model (unimodal) Gaussian distributions Courtesy: K. Arras Motivation

More information

MODERN FACTOR ANALYSIS

MODERN FACTOR ANALYSIS MODERN FACTOR ANALYSIS Harry H. Harman «ö THE pigj UNIVERSITY OF CHICAGO PRESS Contents LIST OF ILLUSTRATIONS GUIDE TO NOTATION xv xvi Parti Foundations of Factor Analysis 1. INTRODUCTION 3 1.1. Brief

More information

Assignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018

Assignment 2. Unsupervised & Probabilistic Learning. Maneesh Sahani Due: Monday Nov 5, 2018 Assignment 2 Unsupervised & Probabilistic Learning Maneesh Sahani Due: Monday Nov 5, 2018 Note: Assignments are due at 11:00 AM (the start of lecture) on the date above. he usual College late assignments

More information

STATISTICS (STAT) 200 Level Courses Registration Restrictions: STAT 250: Required Prerequisites: not Schedule Type: Mason Core: STAT 346:

STATISTICS (STAT) 200 Level Courses Registration Restrictions: STAT 250: Required Prerequisites: not Schedule Type: Mason Core: STAT 346: Statistics (STAT) 1 STATISTICS (STAT) 200 Level Courses STAT 250: Introductory Statistics I. 3 credits. Elementary introduction to statistics. Topics include descriptive statistics, probability, and estimation

More information

Title. Description. time series Introduction to time-series commands

Title. Description. time series Introduction to time-series commands Title time series Introduction to time-series commands Description The Time-Series Reference Manual organizes the commands alphabetically, making it easy to find individual command entries if you know

More information

Real-Time Monitoring of Complex Industrial Processes with Particle Filters

Real-Time Monitoring of Complex Industrial Processes with Particle Filters Real-Time Monitoring of Complex Industrial Processes with Particle Filters Rubén Morales-Menéndez Dept. of Mechatronics and Automation ITESM campus Monterrey Monterrey, NL México rmm@itesm.mx Nando de

More information

Missing Data Analysis for the Employee Dataset

Missing Data Analysis for the Employee Dataset Missing Data Analysis for the Employee Dataset 67% of the observations have missing values! Modeling Setup Random Variables: Y i =(Y i1,...,y ip ) 0 =(Y i,obs, Y i,miss ) 0 R i =(R i1,...,r ip ) 0 ( 1

More information

CHAPTER 7 EXAMPLES: MIXTURE MODELING WITH CROSS- SECTIONAL DATA

CHAPTER 7 EXAMPLES: MIXTURE MODELING WITH CROSS- SECTIONAL DATA Examples: Mixture Modeling With Cross-Sectional Data CHAPTER 7 EXAMPLES: MIXTURE MODELING WITH CROSS- SECTIONAL DATA Mixture modeling refers to modeling with categorical latent variables that represent

More information

MS in Applied Statistics: Study Guide for the Data Science concentration Comprehensive Examination. 1. MAT 456 Applied Regression Analysis

MS in Applied Statistics: Study Guide for the Data Science concentration Comprehensive Examination. 1. MAT 456 Applied Regression Analysis MS in Applied Statistics: Study Guide for the Data Science concentration Comprehensive Examination. The Part II comprehensive examination is a three-hour closed-book exam that is offered on the second

More information

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)

More information

w KLUWER ACADEMIC PUBLISHERS Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms Roman G. Strongin Yaroslav D.

w KLUWER ACADEMIC PUBLISHERS Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms Roman G. Strongin Yaroslav D. Global Optimization with Non-Convex Constraints Sequential and Parallel Algorithms by Roman G. Strongin Nizhni Novgorod State University, Nizhni Novgorod, Russia and Yaroslav D. Sergeyev Institute of Systems

More information

Optimum Array Processing

Optimum Array Processing Optimum Array Processing Part IV of Detection, Estimation, and Modulation Theory Harry L. Van Trees WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Preface xix 1 Introduction 1 1.1 Array Processing

More information

Applications of the k-nearest neighbor method for regression and resampling

Applications of the k-nearest neighbor method for regression and resampling Applications of the k-nearest neighbor method for regression and resampling Objectives Provide a structured approach to exploring a regression data set. Introduce and demonstrate the k-nearest neighbor

More information

MODEL SELECTION AND MODEL AVERAGING IN THE PRESENCE OF MISSING VALUES

MODEL SELECTION AND MODEL AVERAGING IN THE PRESENCE OF MISSING VALUES UNIVERSITY OF GLASGOW MODEL SELECTION AND MODEL AVERAGING IN THE PRESENCE OF MISSING VALUES by KHUNESWARI GOPAL PILLAY A thesis submitted in partial fulfillment for the degree of Doctor of Philosophy in

More information

Image Analysis, Classification and Change Detection in Remote Sensing

Image Analysis, Classification and Change Detection in Remote Sensing Image Analysis, Classification and Change Detection in Remote Sensing WITH ALGORITHMS FOR ENVI/IDL Morton J. Canty Taylor &. Francis Taylor & Francis Group Boca Raton London New York CRC is an imprint

More information

Monte Carlo Methods and Statistical Computing: My Personal E

Monte Carlo Methods and Statistical Computing: My Personal E Monte Carlo Methods and Statistical Computing: My Personal Experience Department of Mathematics & Statistics Indian Institute of Technology Kanpur November 29, 2014 Outline Preface 1 Preface 2 3 4 5 6

More information

Assessing the Quality of the Natural Cubic Spline Approximation

Assessing the Quality of the Natural Cubic Spline Approximation Assessing the Quality of the Natural Cubic Spline Approximation AHMET SEZER ANADOLU UNIVERSITY Department of Statisticss Yunus Emre Kampusu Eskisehir TURKEY ahsst12@yahoo.com Abstract: In large samples,

More information

SAS High-Performance Analytics Products

SAS High-Performance Analytics Products Fact Sheet What do SAS High-Performance Analytics products do? With high-performance analytics products from SAS, you can develop and process models that use huge amounts of diverse data. These products

More information

Generalized least squares (GLS) estimates of the level-2 coefficients,

Generalized least squares (GLS) estimates of the level-2 coefficients, Contents 1 Conceptual and Statistical Background for Two-Level Models...7 1.1 The general two-level model... 7 1.1.1 Level-1 model... 8 1.1.2 Level-2 model... 8 1.2 Parameter estimation... 9 1.3 Empirical

More information

Practical Course WS12/13 Introduction to Monte Carlo Localization

Practical Course WS12/13 Introduction to Monte Carlo Localization Practical Course WS12/13 Introduction to Monte Carlo Localization Cyrill Stachniss and Luciano Spinello 1 State Estimation Estimate the state of a system given observations and controls Goal: 2 Bayes Filter

More information

(W: 12:05-1:50, 50-N202)

(W: 12:05-1:50, 50-N202) 2016 School of Information Technology and Electrical Engineering at the University of Queensland Schedule of Events Week Date Lecture (W: 12:05-1:50, 50-N202) 1 27-Jul Introduction 2 Representing Position

More information

Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013

Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013 Your Name: Your student id: Solution Sketches Midterm Exam COSC 6342 Machine Learning March 20, 2013 Problem 1 [5+?]: Hypothesis Classes Problem 2 [8]: Losses and Risks Problem 3 [11]: Model Generation

More information

IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING

IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING SECOND EDITION IMAGE ANALYSIS, CLASSIFICATION, and CHANGE DETECTION in REMOTE SENSING ith Algorithms for ENVI/IDL Morton J. Canty с*' Q\ CRC Press Taylor &. Francis Group Boca Raton London New York CRC

More information

ESTIMATING DENSITY DEPENDENCE, PROCESS NOISE, AND OBSERVATION ERROR

ESTIMATING DENSITY DEPENDENCE, PROCESS NOISE, AND OBSERVATION ERROR ESTIMATING DENSITY DEPENDENCE, PROCESS NOISE, AND OBSERVATION ERROR Coinvestigators: José Ponciano, University of Idaho Subhash Lele, University of Alberta Mark Taper, Montana State University David Staples,

More information

DEPARTMENT OF STATISTICS

DEPARTMENT OF STATISTICS Department of Statistics 1 DEPARTMENT OF STATISTICS Office in Statistics Building, Room 102 (970) 491-5269 or (970) 491-6546 stat.colostate.edu (http://www.stat.colostate.edu) Don Estep, Department Chair

More information

1 Methods for Posterior Simulation

1 Methods for Posterior Simulation 1 Methods for Posterior Simulation Let p(θ y) be the posterior. simulation. Koop presents four methods for (posterior) 1. Monte Carlo integration: draw from p(θ y). 2. Gibbs sampler: sequentially drawing

More information

Fundamentals of Digital Image Processing

Fundamentals of Digital Image Processing \L\.6 Gw.i Fundamentals of Digital Image Processing A Practical Approach with Examples in Matlab Chris Solomon School of Physical Sciences, University of Kent, Canterbury, UK Toby Breckon School of Engineering,

More information

INLA: an introduction

INLA: an introduction INLA: an introduction Håvard Rue 1 Norwegian University of Science and Technology Trondheim, Norway May 2009 1 Joint work with S.Martino (Trondheim) and N.Chopin (Paris) Latent Gaussian models Background

More information

Conditional Volatility Estimation by. Conditional Quantile Autoregression

Conditional Volatility Estimation by. Conditional Quantile Autoregression International Journal of Mathematical Analysis Vol. 8, 2014, no. 41, 2033-2046 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ijma.2014.47210 Conditional Volatility Estimation by Conditional Quantile

More information

Journal of Statistical Software

Journal of Statistical Software JSS Journal of Statistical Software May 2011, Volume 41, Issue 9. http://www.jstatsoft.org/ State Space Methods in RATS Thomas Doan Estima Abstract This paper uses several examples to show how the econometrics

More information

Quantitative Biology II!

Quantitative Biology II! Quantitative Biology II! Lecture 3: Markov Chain Monte Carlo! March 9, 2015! 2! Plan for Today!! Introduction to Sampling!! Introduction to MCMC!! Metropolis Algorithm!! Metropolis-Hastings Algorithm!!

More information

HIERARCHICAL FORECASTING OF WEB SERVER WORKLOAD USING SEQUENTIAL MONTE CARLO TRAINING

HIERARCHICAL FORECASTING OF WEB SERVER WORKLOAD USING SEQUENTIAL MONTE CARLO TRAINING HIERARCHICAL FORECASTING OF WEB SERVER WORKLOAD USING SEQUENTIAL MONTE CARLO TRAINING Tom Vercauteren*, Pradeep Aggarwal Ý, Xiaodong Wang Ý, Ta-Hsin Li Þ * INRIA, Sophia Antipolis, France, Ý Department

More information

5. Key ingredients for programming a random walk

5. Key ingredients for programming a random walk 1. Random Walk = Brownian Motion = Markov Chain (with caveat that each of these is a class of different models, not all of which are exactly equal) 2. A random walk is a specific kind of time series process

More information

Introductory Combinatorics

Introductory Combinatorics Introductory Combinatorics Third Edition KENNETH P. BOGART Dartmouth College,. " A Harcourt Science and Technology Company San Diego San Francisco New York Boston London Toronto Sydney Tokyo xm CONTENTS

More information

Statistical Methods for the Analysis of Repeated Measurements

Statistical Methods for the Analysis of Repeated Measurements Charles S. Davis Statistical Methods for the Analysis of Repeated Measurements With 20 Illustrations #j Springer Contents Preface List of Tables List of Figures v xv xxiii 1 Introduction 1 1.1 Repeated

More information

Serial Correlation and Heteroscedasticity in Time series Regressions. Econometric (EC3090) - Week 11 Agustín Bénétrix

Serial Correlation and Heteroscedasticity in Time series Regressions. Econometric (EC3090) - Week 11 Agustín Bénétrix Serial Correlation and Heteroscedasticity in Time series Regressions Econometric (EC3090) - Week 11 Agustín Bénétrix 1 Properties of OLS with serially correlated errors OLS still unbiased and consistent

More information

Latent Curve Models. A Structural Equation Perspective WILEY- INTERSCIENΠKENNETH A. BOLLEN

Latent Curve Models. A Structural Equation Perspective WILEY- INTERSCIENΠKENNETH A. BOLLEN Latent Curve Models A Structural Equation Perspective KENNETH A. BOLLEN University of North Carolina Department of Sociology Chapel Hill, North Carolina PATRICK J. CURRAN University of North Carolina Department

More information

Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask

Machine Learning and Data Mining. Clustering (1): Basics. Kalev Kask Machine Learning and Data Mining Clustering (1): Basics Kalev Kask Unsupervised learning Supervised learning Predict target value ( y ) given features ( x ) Unsupervised learning Understand patterns of

More information

Package lgarch. September 15, 2015

Package lgarch. September 15, 2015 Type Package Package lgarch September 15, 2015 Title Simulation and Estimation of Log-GARCH Models Version 0.6-2 Depends R (>= 2.15.0), zoo Date 2015-09-14 Author Genaro Sucarrat Maintainer Genaro Sucarrat

More information

Computer vision: models, learning and inference. Chapter 10 Graphical Models

Computer vision: models, learning and inference. Chapter 10 Graphical Models Computer vision: models, learning and inference Chapter 10 Graphical Models Independence Two variables x 1 and x 2 are independent if their joint probability distribution factorizes as Pr(x 1, x 2 )=Pr(x

More information

AN IMPROVED PARTICLE FILTER ALGORITHM BASED ON NEURAL NETWORK FOR TARGET TRACKING

AN IMPROVED PARTICLE FILTER ALGORITHM BASED ON NEURAL NETWORK FOR TARGET TRACKING AN IMPROVED PARTICLE FILTER ALGORITHM BASED ON NEURAL NETWORK FOR TARGET TRACKING Qin Wen, Peng Qicoiig 140 Lab, Institution of Communication and Information Engineering, University of Electronic Science

More information

Chapter 1 Introduction. Chapter Contents

Chapter 1 Introduction. Chapter Contents Chapter 1 Chapter Contents OVERVIEW OF SAS HPF SOFTWARE... 5 Uses of SAS High-Performance Forecasting Software... 7 Contents of SAS High-Performance Forecasting Software... 7 ABOUT THIS BOOK... 9 Chapter

More information

Resampling Methods for Dependent Data

Resampling Methods for Dependent Data S.N. Lahiri Resampling Methods for Dependent Data With 25 Illustrations Springer Contents 1 Scope of Resampling Methods for Dependent Data 1 1.1 The Bootstrap Principle 1 1.2 Examples 7 1.3 Concluding

More information

Chad Fulton... Federal Reserve Board

Chad Fulton... Federal Reserve Board Estimating time series models by state space methods in Python - Statsmodels September 12, 2018 - Securities and Exchange Commission Chad Fulton......................................... Federal Reserve

More information

CLASSIFICATION AND CHANGE DETECTION

CLASSIFICATION AND CHANGE DETECTION IMAGE ANALYSIS, CLASSIFICATION AND CHANGE DETECTION IN REMOTE SENSING With Algorithms for ENVI/IDL and Python THIRD EDITION Morton J. Canty CRC Press Taylor & Francis Group Boca Raton London NewYork CRC

More information

Modeling and Reasoning with Bayesian Networks. Adnan Darwiche University of California Los Angeles, CA

Modeling and Reasoning with Bayesian Networks. Adnan Darwiche University of California Los Angeles, CA Modeling and Reasoning with Bayesian Networks Adnan Darwiche University of California Los Angeles, CA darwiche@cs.ucla.edu June 24, 2008 Contents Preface 1 1 Introduction 1 1.1 Automated Reasoning........................

More information

Minitab 18 Feature List

Minitab 18 Feature List Minitab 18 Feature List * New or Improved Assistant Measurement systems analysis * Capability analysis Graphical analysis Hypothesis tests Regression DOE Control charts * Graphics Scatterplots, matrix

More information

SYS 6021 Linear Statistical Models

SYS 6021 Linear Statistical Models SYS 6021 Linear Statistical Models Project 2 Spam Filters Jinghe Zhang Summary The spambase data and time indexed counts of spams and hams are studied to develop accurate spam filters. Static models are

More information

Machine Learning. Topic 4: Linear Regression Models

Machine Learning. Topic 4: Linear Regression Models Machine Learning Topic 4: Linear Regression Models (contains ideas and a few images from wikipedia and books by Alpaydin, Duda/Hart/ Stork, and Bishop. Updated Fall 205) Regression Learning Task There

More information

MCMC Diagnostics. Yingbo Li MATH Clemson University. Yingbo Li (Clemson) MCMC Diagnostics MATH / 24

MCMC Diagnostics. Yingbo Li MATH Clemson University. Yingbo Li (Clemson) MCMC Diagnostics MATH / 24 MCMC Diagnostics Yingbo Li Clemson University MATH 9810 Yingbo Li (Clemson) MCMC Diagnostics MATH 9810 1 / 24 Convergence to Posterior Distribution Theory proves that if a Gibbs sampler iterates enough,

More information

Recurrent Neural Network (RNN) Industrial AI Lab.

Recurrent Neural Network (RNN) Industrial AI Lab. Recurrent Neural Network (RNN) Industrial AI Lab. For example (Deterministic) Time Series Data Closed- form Linear difference equation (LDE) and initial condition High order LDEs 2 (Stochastic) Time Series

More information

Replication Note for A Bayesian Poisson Vector Autoregression Model

Replication Note for A Bayesian Poisson Vector Autoregression Model Replication Note for A Bayesian Poisson Vector Autoregression Model Patrick T. Brandt pbrandt@utdallas.edu Todd Sandler tsandler@utdallas.edu School of Economic, Political and Policy Sciences The University

More information

Within these three broad categories, similar commands have been grouped together. Declare data to be time-series data [TS] tsfill

Within these three broad categories, similar commands have been grouped together. Declare data to be time-series data [TS] tsfill Title time series Introduction to time-series commands Description The Time-Series Reference Manual organizes the commands alphabetically, making it easy to find individual command entries if you know

More information

Assignments Fill out this form to do the assignments or see your scores.

Assignments Fill out this form to do the assignments or see your scores. Assignments Assignment schedule General instructions for online assignments Troubleshooting technical problems Fill out this form to do the assignments or see your scores. Login Course: Statistics W21,

More information

Using the DATAMINE Program

Using the DATAMINE Program 6 Using the DATAMINE Program 304 Using the DATAMINE Program This chapter serves as a user s manual for the DATAMINE program, which demonstrates the algorithms presented in this book. Each menu selection

More information

Integrated Algebra 2 and Trigonometry. Quarter 1

Integrated Algebra 2 and Trigonometry. Quarter 1 Quarter 1 I: Functions: Composition I.1 (A.42) Composition of linear functions f(g(x)). f(x) + g(x). I.2 (A.42) Composition of linear and quadratic functions II: Functions: Quadratic II.1 Parabola The

More information

Recipes for State Space Models in R

Recipes for State Space Models in R Recipes for State Space Models in R Paul Teetor July 2015 Introduction This monograph is a collection of recipes for creating state-space models in R. I like the power of state-space models, and R had

More information

Principles of Robot Motion

Principles of Robot Motion Principles of Robot Motion Theory, Algorithms, and Implementation Howie Choset, Kevin Lynch, Seth Hutchinson, George Kantor, Wolfram Burgard, Lydia Kavraki, and Sebastian Thrun A Bradford Book The MIT

More information

This chapter explains two techniques which are frequently used throughout

This chapter explains two techniques which are frequently used throughout Chapter 2 Basic Techniques This chapter explains two techniques which are frequently used throughout this thesis. First, we will introduce the concept of particle filters. A particle filter is a recursive

More information

PROGRAMA DE CURSO. Robotics, Sensing and Autonomous Systems. SCT Auxiliar. Personal

PROGRAMA DE CURSO. Robotics, Sensing and Autonomous Systems. SCT Auxiliar. Personal PROGRAMA DE CURSO Código Nombre EL7031 Robotics, Sensing and Autonomous Systems Nombre en Inglés Robotics, Sensing and Autonomous Systems es Horas de Horas Docencia Horas de Trabajo SCT Docentes Cátedra

More information

Smooth Simultaneous Structural Graph Matching and Point-Set Registration

Smooth Simultaneous Structural Graph Matching and Point-Set Registration 8th IAPR - TC-15 Workshop on Graph-based Representations in Pattern Recognition Smooth Simultaneous Structural Graph Matching and Point-Set Registration Gerard Sanromà¹, René Alquézar² and Francesc Serratosa¹

More information

10-701/15-781, Fall 2006, Final

10-701/15-781, Fall 2006, Final -7/-78, Fall 6, Final Dec, :pm-8:pm There are 9 questions in this exam ( pages including this cover sheet). If you need more room to work out your answer to a question, use the back of the page and clearly

More information

Logistic growth model with bssm Jouni Helske 21 November 2017

Logistic growth model with bssm Jouni Helske 21 November 2017 Logistic growth model with bssm Jouni Helske 21 November 2017 Logistic growth model This vignette shows how to model general non-linear state space models with bssm. The general non-linear Gaussian model

More information

Expectation-Maximization Methods in Population Analysis. Robert J. Bauer, Ph.D. ICON plc.

Expectation-Maximization Methods in Population Analysis. Robert J. Bauer, Ph.D. ICON plc. Expectation-Maximization Methods in Population Analysis Robert J. Bauer, Ph.D. ICON plc. 1 Objective The objective of this tutorial is to briefly describe the statistical basis of Expectation-Maximization

More information

The Curse of Dimensionality

The Curse of Dimensionality The Curse of Dimensionality ACAS 2002 p1/66 Curse of Dimensionality The basic idea of the curse of dimensionality is that high dimensional data is difficult to work with for several reasons: Adding more

More information

Bayesian Background Estimation

Bayesian Background Estimation Bayesian Background Estimation mum knot spacing was determined by the width of the signal structure that one wishes to exclude from the background curve. This paper extends the earlier work in two important

More information

DATA ANALYSIS USING HIERARCHICAL GENERALIZED LINEAR MODELS WITH R

DATA ANALYSIS USING HIERARCHICAL GENERALIZED LINEAR MODELS WITH R DATA ANALYSIS USING HIERARCHICAL GENERALIZED LINEAR MODELS WITH R Lee, Rönnegård & Noh LRN@du.se Lee, Rönnegård & Noh HGLM book 1 / 24 Overview 1 Background to the book 2 Crack growth example 3 Contents

More information

Passive Multi Target Tracking with GM-PHD Filter

Passive Multi Target Tracking with GM-PHD Filter Passive Multi Target Tracking with GM-PHD Filter Dann Laneuville DCNS SNS Division Toulon, France dann.laneuville@dcnsgroup.com Jérémie Houssineau DCNS SNS Division Toulon, France jeremie.houssineau@dcnsgroup.com

More information

Probabilistic Graphical Models

Probabilistic Graphical Models Overview of Part One Probabilistic Graphical Models Part One: Graphs and Markov Properties Christopher M. Bishop Graphs and probabilities Directed graphs Markov properties Undirected graphs Examples Microsoft

More information

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Lincoln Laboratory ASAP-2001 Workshop Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Shawn Kraut and Jeffrey Krolik Duke University Department of Electrical and Computer

More information

Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg

Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg Discussion on Bayesian Model Selection and Parameter Estimation in Extragalactic Astronomy by Martin Weinberg Phil Gregory Physics and Astronomy Univ. of British Columbia Introduction Martin Weinberg reported

More information