Resampling Methods for Dependent Data

Size: px
Start display at page:

Download "Resampling Methods for Dependent Data"

Transcription

1 S.N. Lahiri Resampling Methods for Dependent Data With 25 Illustrations Springer

2 Contents 1 Scope of Resampling Methods for Dependent Data The Bootstrap Principle Examples Concluding Remarks Notation 13 2 Bootstrap Methods * Introduction IID Bootstrap Inadequacy of IID Bootstrap for Dependent Data Bootstrap Based on IID Innovations Moving Block Bootstrap Nonoverlapping Block Bootstrap Generalized Block Bootstrap Circular Block Bootstrap Stationary Block Bootstrap Subsampling Transformation-Based Bootstrap Sieve Bootstrap 41 3 Properties of Block Bootstrap Methods for the Sample Mean Introduction Consistency of MBB, NBB, CBB: Sample Mean 47

3 XII Contents Consistency of Bootstrap Variance Estimators Consistency of Distribution Function Estimators Consistency of the SB: Sample Mean Consistency of SB Variance Estimators Consistency of SB Distribution Function Estimators 63 4 Extensions and Examples Introduction Smooth Functions of Means M-Estimators Differentiate Functional Bootstrapping the Empirical Process Consistency of the MBB for Differentiable Statistical Functionals Examples 99 5 Comparison of Block Bootstrap Methods Introduction Empirical Comparisons The Theoretical Framework Expansions for the MSEs Theoretical Comparisons Asymptotic Efficiency Comparison at Optimal Block Lengths Concluding Remarks Proofs Proofs of Theorems for the MBB, the NBB, and the CBB Proofs of Theorems for the SB Second-Order Properties Introduction Edgeworth Expansions for the Mean Under Independence Edgeworth Expansions for the Mean Under Dependence Expansions for Functions of Sample Means Expansions Under the Smooth Function Model Under Independence Expansions for Normalized and Studentized Statistics Under Independence 163^ Expansions for Normalized Statistics Under Dependence Expansions for Studentized Statistics Under Dependence Second-Order Properties of Block Bootstrap Methods

4 Contents XIII 7 Empirical Choice of the Block Size Introduction Theoretical Optimal Block Lengths Optimal Block Lengths for Bias and Variance Estimation Optimal Block Lengths for Distribution Function Estimation A Method Based on Subsampling A Nonparametric Plug-in Method Motivation The Bias Estimator The JAB Variance Estimator The Optimal Block Length Estimator Model-Based Bootstrap Introduction Bootstrapping Stationary Autoregressive Processes Bootstrapping Explosive Autoregressive Processes Bootstrapping Unstable Autoregressive Processes Bootstrapping a Stationary ARMA Process Frequency Domain Bootstrap Introduction Bootstrapping Ratio Statistics Spectral Means and Ratio Statistics Frequency Domain Bootstrap for Ratio Statistics Second-Order Correctness of the FDB Bootstrapping Spectral Density Estimators Frequency Domain Bootstrap for Spectral Density Estimation Consistency of the FDB Distribution Function Estimator Bandwidth Selection A Modified FDB Motivation The Autoregressive-Aided FDB A Long-Range Dependence Introduction A Class of Long-Range Dependent Processes Properties of the MBB Method Main Results Proofs Properties of the Subsampling Method Results on the Normalized Sample Mean 252

5 XIV Contents Results on the Studentized Sample Mean Proofs., Numerical Results Bootstrapping Heavy-Tailed Data and Extremes Introduction Heavy-Tailed Distributions Consistency of the MBB Invalidity of the MBB Extremes of Stationary Random Variables Results on Bootstrapping Extremes Bootstrapping Extremes With Estimated Constants Resampling Methods for Spatial Data Introduction Spatial Asymptotic Frameworks Block Bootstrap for Spatial Data on a Regular Grid Description of the Block Bootstrap Method Numerical Examples Consistency of Bootstrap Variance Estimators Results on the Empirical Distribution Function Differentiable Functional Estimation of Spatial Covariance Parameters The Variogram Least Squares Variogram Estimation The RGLS Method Properties of the RGLS Estimators Numerical Examples Bootstrap for Irregularly Spaced Spatial Data A Class of Spatial Stochastic Designs 319" Asymptotic Distribution of M-Estimators A Spatial Block Bootstrap Method Properties of the Spatial Bootstrap Method Resampling Methods for Spatial Prediction Prediction of Integrals Prediction of Point Values - % A 339 B 345 References 349 Author Index 367 Subject Index 371

COPYRIGHTED MATERIAL CONTENTS

COPYRIGHTED MATERIAL CONTENTS PREFACE ACKNOWLEDGMENTS LIST OF TABLES xi xv xvii 1 INTRODUCTION 1 1.1 Historical Background 1 1.2 Definition and Relationship to the Delta Method and Other Resampling Methods 3 1.2.1 Jackknife 6 1.2.2

More information

Stochastic Simulation: Algorithms and Analysis

Stochastic Simulation: Algorithms and Analysis Soren Asmussen Peter W. Glynn Stochastic Simulation: Algorithms and Analysis et Springer Contents Preface Notation v xii I What This Book Is About 1 1 An Illustrative Example: The Single-Server Queue 1

More information

Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221. Yixiao Sun Department of Economics, University of California, San Diego

Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221. Yixiao Sun Department of Economics, University of California, San Diego Nonparametric and Semiparametric Econometrics Lecture Notes for Econ 221 Yixiao Sun Department of Economics, University of California, San Diego Winter 2007 Contents Preface ix 1 Kernel Smoothing: Density

More information

PATTERN CLASSIFICATION AND SCENE ANALYSIS

PATTERN CLASSIFICATION AND SCENE ANALYSIS PATTERN CLASSIFICATION AND SCENE ANALYSIS RICHARD O. DUDA PETER E. HART Stanford Research Institute, Menlo Park, California A WILEY-INTERSCIENCE PUBLICATION JOHN WILEY & SONS New York Chichester Brisbane

More information

An Introduction to the Bootstrap

An Introduction to the Bootstrap An Introduction to the Bootstrap Bradley Efron Department of Statistics Stanford University and Robert J. Tibshirani Department of Preventative Medicine and Biostatistics and Department of Statistics,

More information

Random Number Generation and Monte Carlo Methods

Random Number Generation and Monte Carlo Methods James E. Gentle Random Number Generation and Monte Carlo Methods With 30 Illustrations Springer Contents Preface vii 1 Simulating Random Numbers from a Uniform Distribution 1 1.1 Linear Congruential Generators

More information

Chapter 3. Bootstrap. 3.1 Introduction. 3.2 The general idea

Chapter 3. Bootstrap. 3.1 Introduction. 3.2 The general idea Chapter 3 Bootstrap 3.1 Introduction The estimation of parameters in probability distributions is a basic problem in statistics that one tends to encounter already during the very first course on the subject.

More information

Time Series Analysis by State Space Methods

Time Series Analysis by State Space Methods Time Series Analysis by State Space Methods Second Edition J. Durbin London School of Economics and Political Science and University College London S. J. Koopman Vrije Universiteit Amsterdam OXFORD UNIVERSITY

More information

A selective bibliography on the Bootstrap

A selective bibliography on the Bootstrap A selective bibliography on the Bootstrap P. Bertail CREST-Laboratoire de Statistiques 1 Books. The jackknife Gray, H., Schucany, W.(1972). The Generalized Jackknife Statistic. Marcel Dekker, New-York.

More information

Analysis of Panel Data. Third Edition. Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS

Analysis of Panel Data. Third Edition. Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS Analysis of Panel Data Third Edition Cheng Hsiao University of Southern California CAMBRIDGE UNIVERSITY PRESS Contents Preface to the ThirdEdition Preface to the Second Edition Preface to the First Edition

More information

Statistics (STAT) Statistics (STAT) 1. Prerequisites: grade in C- or higher in STAT 1200 or STAT 1300 or STAT 1400

Statistics (STAT) Statistics (STAT) 1. Prerequisites: grade in C- or higher in STAT 1200 or STAT 1300 or STAT 1400 Statistics (STAT) 1 Statistics (STAT) STAT 1200: Introductory Statistical Reasoning Statistical concepts for critically evaluation quantitative information. Descriptive statistics, probability, estimation,

More information

Institute for Statics und Dynamics of Structures Fuzzy Time Series

Institute for Statics und Dynamics of Structures Fuzzy Time Series Institute for Statics und Dynamics of Structures Fuzzy Time Series Bernd Möller 1 Description of fuzzy time series 2 3 4 5 Conclusions Folie 2 von slide422 1 Description of fuzzy time series 2 3 4 5 Conclusions

More information

The Bootstrap and Jackknife

The Bootstrap and Jackknife The Bootstrap and Jackknife Summer 2017 Summer Institutes 249 Bootstrap & Jackknife Motivation In scientific research Interest often focuses upon the estimation of some unknown parameter, θ. The parameter

More information

On the Test and Estimation of Fractional Parameter. in ARFIMA Model: Bootstrap Approach

On the Test and Estimation of Fractional Parameter. in ARFIMA Model: Bootstrap Approach Applied Mathematical Sciences, Vol. 8, 2014, no. 96, 4783-4792 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ams.2014.46498 On the Test and Estimation of Fractional Parameter in ARFIMA Model:

More information

Nonparametric Estimation of Distribution Function using Bezier Curve

Nonparametric Estimation of Distribution Function using Bezier Curve Communications for Statistical Applications and Methods 2014, Vol. 21, No. 1, 105 114 DOI: http://dx.doi.org/10.5351/csam.2014.21.1.105 ISSN 2287-7843 Nonparametric Estimation of Distribution Function

More information

Optimum Array Processing

Optimum Array Processing Optimum Array Processing Part IV of Detection, Estimation, and Modulation Theory Harry L. Van Trees WILEY- INTERSCIENCE A JOHN WILEY & SONS, INC., PUBLICATION Preface xix 1 Introduction 1 1.1 Array Processing

More information

Curve and Surface Fitting with Splines. PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium

Curve and Surface Fitting with Splines. PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium Curve and Surface Fitting with Splines PAUL DIERCKX Professor, Computer Science Department, Katholieke Universiteit Leuven, Belgium CLARENDON PRESS OXFORD 1995 - Preface List of Figures List of Tables

More information

MS&E 226: Small Data

MS&E 226: Small Data MS&E 226: Small Data Lecture 13: The bootstrap (v3) Ramesh Johari ramesh.johari@stanford.edu 1 / 30 Resampling 2 / 30 Sampling distribution of a statistic For this lecture: There is a population model

More information

DEPARTMENT OF STATISTICS

DEPARTMENT OF STATISTICS Department of Statistics 1 DEPARTMENT OF STATISTICS Office in Statistics Building, Room 102 (970) 491-5269 or (970) 491-6546 stat.colostate.edu (http://www.stat.colostate.edu) Don Estep, Department Chair

More information

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer

Ludwig Fahrmeir Gerhard Tute. Statistical odelling Based on Generalized Linear Model. íecond Edition. . Springer Ludwig Fahrmeir Gerhard Tute Statistical odelling Based on Generalized Linear Model íecond Edition. Springer Preface to the Second Edition Preface to the First Edition List of Examples List of Figures

More information

Conditional Volatility Estimation by. Conditional Quantile Autoregression

Conditional Volatility Estimation by. Conditional Quantile Autoregression International Journal of Mathematical Analysis Vol. 8, 2014, no. 41, 2033-2046 HIKARI Ltd, www.m-hikari.com http://dx.doi.org/10.12988/ijma.2014.47210 Conditional Volatility Estimation by Conditional Quantile

More information

Overview. Spectral Processing of Point- Sampled Geometry. Introduction. Introduction. Fourier Transform. Fourier Transform

Overview. Spectral Processing of Point- Sampled Geometry. Introduction. Introduction. Fourier Transform. Fourier Transform Overview Spectral Processing of Point- Sampled Geometry Introduction Fourier transform Spectral processing pipeline Spectral filtering Adaptive subsampling Summary Point-Based Computer Graphics Markus

More information

Locally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling

Locally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling Locally Weighted Least Squares Regression for Image Denoising, Reconstruction and Up-sampling Moritz Baecher May 15, 29 1 Introduction Edge-preserving smoothing and super-resolution are classic and important

More information

Big Data Methods. Chapter 5: Machine learning. Big Data Methods, Chapter 5, Slide 1

Big Data Methods. Chapter 5: Machine learning. Big Data Methods, Chapter 5, Slide 1 Big Data Methods Chapter 5: Machine learning Big Data Methods, Chapter 5, Slide 1 5.1 Introduction to machine learning What is machine learning? Concerned with the study and development of algorithms that

More information

Adaptive System Identification and Signal Processing Algorithms

Adaptive System Identification and Signal Processing Algorithms Adaptive System Identification and Signal Processing Algorithms edited by N. Kalouptsidis University of Athens S. Theodoridis University of Patras Prentice Hall New York London Toronto Sydney Tokyo Singapore

More information

Section 4 Matching Estimator

Section 4 Matching Estimator Section 4 Matching Estimator Matching Estimators Key Idea: The matching method compares the outcomes of program participants with those of matched nonparticipants, where matches are chosen on the basis

More information

Stat 510, Lab April 27, 2007

Stat 510, Lab April 27, 2007 Stat 510, Lab 10.5 April 27, 2007 1 In Class 1.1 Smoothing Let s look at the Australian labour data which was seen in previous labs. We would like to estimate the spectral density of the data. We may generate

More information

Overview of various smoothers

Overview of various smoothers Chapter 2 Overview of various smoothers A scatter plot smoother is a tool for finding structure in a scatter plot: Figure 2.1: CD4 cell count since seroconversion for HIV infected men. CD4 counts vs Time

More information

MOBILITY MODELS FOR NEXT WIRELESS NETWORKS GENERATION WILEY AD HOC, VEHICULAR AND MESH NETWORKS. Paolo Santi

MOBILITY MODELS FOR NEXT WIRELESS NETWORKS GENERATION WILEY AD HOC, VEHICULAR AND MESH NETWORKS. Paolo Santi MOBILITY MODELS FOR NEXT GENERATION WIRELESS NETWORKS AD HOC, VEHICULAR AND MESH NETWORKS Paolo Santi Istituto di Informatica e Telematica del CNR, Italy WILEY A John Wiley & Sons, Ltd., Publication Contents

More information

Optimization and Simulation

Optimization and Simulation Optimization and Simulation Statistical analysis and bootstrapping Michel Bierlaire Transport and Mobility Laboratory School of Architecture, Civil and Environmental Engineering Ecole Polytechnique Fédérale

More information

On Kernel Density Estimation with Univariate Application. SILOKO, Israel Uzuazor

On Kernel Density Estimation with Univariate Application. SILOKO, Israel Uzuazor On Kernel Density Estimation with Univariate Application BY SILOKO, Israel Uzuazor Department of Mathematics/ICT, Edo University Iyamho, Edo State, Nigeria. A Seminar Presented at Faculty of Science, Edo

More information

Finite Sample Criteria for Autoregressive Order Selection

Finite Sample Criteria for Autoregressive Order Selection 3550 IEEE TRANSACTIONS ON SIGNAL PROCESSING, VOL. 48, NO. 12, DECEMBER 2000 Finite Sample Criteria for Autoregressive Order Selection Piet M. T. Broersen Abstract The quality of selected AR models depends

More information

Optimization in the Big Data Regime 5: Parallelization? Sham M. Kakade

Optimization in the Big Data Regime 5: Parallelization? Sham M. Kakade Optimization in the Big Data Regime 5: Parallelization? Sham M. Kakade Machine Learning for Big Data CSE547/STAT548 University of Washington S. M. Kakade (UW) Optimization for Big data 1 / 21 Announcements...

More information

Modelling and Quantitative Methods in Fisheries

Modelling and Quantitative Methods in Fisheries SUB Hamburg A/553843 Modelling and Quantitative Methods in Fisheries Second Edition Malcolm Haddon ( r oc) CRC Press \ y* J Taylor & Francis Croup Boca Raton London New York CRC Press is an imprint of

More information

Spatial Interpolation & Geostatistics

Spatial Interpolation & Geostatistics (Z i Z j ) 2 / 2 Spatial Interpolation & Geostatistics Lag Lag Mean Distance between pairs of points 1 Tobler s Law All places are related, but nearby places are related more than distant places Corollary:

More information

Statistical Methods for the Analysis of Repeated Measurements

Statistical Methods for the Analysis of Repeated Measurements Charles S. Davis Statistical Methods for the Analysis of Repeated Measurements With 20 Illustrations #j Springer Contents Preface List of Tables List of Figures v xv xxiii 1 Introduction 1 1.1 Repeated

More information

Last time... Coryn Bailer-Jones. check and if appropriate remove outliers, errors etc. linear regression

Last time... Coryn Bailer-Jones. check and if appropriate remove outliers, errors etc. linear regression Machine learning, pattern recognition and statistical data modelling Lecture 3. Linear Methods (part 1) Coryn Bailer-Jones Last time... curse of dimensionality local methods quickly become nonlocal as

More information

Contents. Preface to the Second Edition

Contents. Preface to the Second Edition Preface to the Second Edition v 1 Introduction 1 1.1 What Is Data Mining?....................... 4 1.2 Motivating Challenges....................... 5 1.3 The Origins of Data Mining....................

More information

Slides credited from Dr. David Silver & Hung-Yi Lee

Slides credited from Dr. David Silver & Hung-Yi Lee Slides credited from Dr. David Silver & Hung-Yi Lee Review Reinforcement Learning 2 Reinforcement Learning RL is a general purpose framework for decision making RL is for an agent with the capacity to

More information

STATISTICS (STAT) Statistics (STAT) 1

STATISTICS (STAT) Statistics (STAT) 1 Statistics (STAT) 1 STATISTICS (STAT) STAT 2013 Elementary Statistics (A) Prerequisites: MATH 1483 or MATH 1513, each with a grade of "C" or better; or an acceptable placement score (see placement.okstate.edu).

More information

Spatial Interpolation - Geostatistics 4/3/2018

Spatial Interpolation - Geostatistics 4/3/2018 Spatial Interpolation - Geostatistics 4/3/201 (Z i Z j ) 2 / 2 Spatial Interpolation & Geostatistics Lag Distance between pairs of points Lag Mean Tobler s Law All places are related, but nearby places

More information

Assessing the Quality of the Natural Cubic Spline Approximation

Assessing the Quality of the Natural Cubic Spline Approximation Assessing the Quality of the Natural Cubic Spline Approximation AHMET SEZER ANADOLU UNIVERSITY Department of Statisticss Yunus Emre Kampusu Eskisehir TURKEY ahsst12@yahoo.com Abstract: In large samples,

More information

ADAPTIVE METROPOLIS-HASTINGS SAMPLING, OR MONTE CARLO KERNEL ESTIMATION

ADAPTIVE METROPOLIS-HASTINGS SAMPLING, OR MONTE CARLO KERNEL ESTIMATION ADAPTIVE METROPOLIS-HASTINGS SAMPLING, OR MONTE CARLO KERNEL ESTIMATION CHRISTOPHER A. SIMS Abstract. A new algorithm for sampling from an arbitrary pdf. 1. Introduction Consider the standard problem of

More information

STATISTICS (STAT) 200 Level Courses. 300 Level Courses. Statistics (STAT) 1

STATISTICS (STAT) 200 Level Courses. 300 Level Courses. Statistics (STAT) 1 Statistics (STAT) 1 STATISTICS (STAT) 200 Level Courses STAT 250: Introductory Statistics I. 3 credits. Elementary introduction to statistics. Topics include descriptive statistics, probability, and estimation

More information

Flexible Lag Definition for Experimental Variogram Calculation

Flexible Lag Definition for Experimental Variogram Calculation Flexible Lag Definition for Experimental Variogram Calculation Yupeng Li and Miguel Cuba The inference of the experimental variogram in geostatistics commonly relies on the method of moments approach.

More information

Statistical Modeling with Spline Functions Methodology and Theory

Statistical Modeling with Spline Functions Methodology and Theory This is page 1 Printer: Opaque this Statistical Modeling with Spline Functions Methodology and Theory Mark H. Hansen University of California at Los Angeles Jianhua Z. Huang University of Pennsylvania

More information

Model combination. Resampling techniques p.1/34

Model combination. Resampling techniques p.1/34 Model combination The winner-takes-all approach is intuitively the approach which should work the best. However recent results in machine learning show that the performance of the final model can be improved

More information

Progressive photon mapping can be interpreted as an estimator that constructs the full paths from camera to light by constantly connecting two fresh

Progressive photon mapping can be interpreted as an estimator that constructs the full paths from camera to light by constantly connecting two fresh 1 Progressive photon mapping can be interpreted as an estimator that constructs the full paths from camera to light by constantly connecting two fresh subpaths. The estimation is a sum, where W_n is the

More information

Petrel TIPS&TRICKS from SCM

Petrel TIPS&TRICKS from SCM Petrel TIPS&TRICKS from SCM Knowledge Worth Sharing Petrel 2D Grid Algorithms and the Data Types they are Commonly Used With Specific algorithms in Petrel s Make/Edit Surfaces process are more appropriate

More information

Introduction to Objective Analysis

Introduction to Objective Analysis Chapter 4 Introduction to Objective Analysis Atmospheric data are routinely collected around the world but observation sites are located rather randomly from a spatial perspective. On the other hand, most

More information

Nonparametric Error Estimation Methods for Evaluating and Validating Artificial Neural Network Prediction Models

Nonparametric Error Estimation Methods for Evaluating and Validating Artificial Neural Network Prediction Models Nonparametric Error Estimation Methods for Evaluating and Validating Artificial Neural Network Prediction Models Janet M. Twomey and Alice E. Smith Department of Industrial Engineering University of Pittsburgh

More information

Further Simulation Results on Resampling Confidence Intervals for Empirical Variograms

Further Simulation Results on Resampling Confidence Intervals for Empirical Variograms University of Wollongong Research Online Centre for Statistical & Survey Methodology Working Paper Series Faculty of Engineering and Information Sciences 2010 Further Simulation Results on Resampling Confidence

More information

MODERN FACTOR ANALYSIS

MODERN FACTOR ANALYSIS MODERN FACTOR ANALYSIS Harry H. Harman «ö THE pigj UNIVERSITY OF CHICAGO PRESS Contents LIST OF ILLUSTRATIONS GUIDE TO NOTATION xv xvi Parti Foundations of Factor Analysis 1. INTRODUCTION 3 1.1. Brief

More information

An introduction to multi-armed bandits

An introduction to multi-armed bandits An introduction to multi-armed bandits Henry WJ Reeve (Manchester) (henry.reeve@manchester.ac.uk) A joint work with Joe Mellor (Edinburgh) & Professor Gavin Brown (Manchester) Plan 1. An introduction to

More information

Direct Sequential Co-simulation with Joint Probability Distributions

Direct Sequential Co-simulation with Joint Probability Distributions Math Geosci (2010) 42: 269 292 DOI 10.1007/s11004-010-9265-x Direct Sequential Co-simulation with Joint Probability Distributions Ana Horta Amílcar Soares Received: 13 May 2009 / Accepted: 3 January 2010

More information

Unified Methods for Censored Longitudinal Data and Causality

Unified Methods for Censored Longitudinal Data and Causality Mark J. van der Laan James M. Robins Unified Methods for Censored Longitudinal Data and Causality Springer Preface v Notation 1 1 Introduction 8 1.1 Motivation, Bibliographic History, and an Overview of

More information

Missing Data Analysis for the Employee Dataset

Missing Data Analysis for the Employee Dataset Missing Data Analysis for the Employee Dataset 67% of the observations have missing values! Modeling Setup Random Variables: Y i =(Y i1,...,y ip ) 0 =(Y i,obs, Y i,miss ) 0 R i =(R i1,...,r ip ) 0 ( 1

More information

Use of Extreme Value Statistics in Modeling Biometric Systems

Use of Extreme Value Statistics in Modeling Biometric Systems Use of Extreme Value Statistics in Modeling Biometric Systems Similarity Scores Two types of matching: Genuine sample Imposter sample Matching scores Enrolled sample 0.95 0.32 Probability Density Decision

More information

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu

FMA901F: Machine Learning Lecture 3: Linear Models for Regression. Cristian Sminchisescu FMA901F: Machine Learning Lecture 3: Linear Models for Regression Cristian Sminchisescu Machine Learning: Frequentist vs. Bayesian In the frequentist setting, we seek a fixed parameter (vector), with value(s)

More information

The Immersed Interface Method

The Immersed Interface Method The Immersed Interface Method Numerical Solutions of PDEs Involving Interfaces and Irregular Domains Zhiiin Li Kazufumi Ito North Carolina State University Raleigh, North Carolina Society for Industrial

More information

Performance Estimation and Regularization. Kasthuri Kannan, PhD. Machine Learning, Spring 2018

Performance Estimation and Regularization. Kasthuri Kannan, PhD. Machine Learning, Spring 2018 Performance Estimation and Regularization Kasthuri Kannan, PhD. Machine Learning, Spring 2018 Bias- Variance Tradeoff Fundamental to machine learning approaches Bias- Variance Tradeoff Error due to Bias:

More information

Pattern Fabric Defect Detection Using Nonparametric Regression

Pattern Fabric Defect Detection Using Nonparametric Regression , Int. J. Appl. Math. Stat.; Vol. 53; Issue No. 6; Year 2015, ISSN 0973-1377 (Print), ISSN 0973-7545 (Online) Copyright 2015 by CESER PUBLICATIONS Pattern Fabric Defect Detection Using Nonparametric Regression

More information

Chapter 1. Introduction

Chapter 1. Introduction Chapter 1 Introduction A Monte Carlo method is a compuational method that uses random numbers to compute (estimate) some quantity of interest. Very often the quantity we want to compute is the mean of

More information

Hyperparameters and Validation Sets. Sargur N. Srihari

Hyperparameters and Validation Sets. Sargur N. Srihari Hyperparameters and Validation Sets Sargur N. srihari@cedar.buffalo.edu 1 Topics in Machine Learning Basics 1. Learning Algorithms 2. Capacity, Overfitting and Underfitting 3. Hyperparameters and Validation

More information

Principles of Robot Motion

Principles of Robot Motion Principles of Robot Motion Theory, Algorithms, and Implementation Howie Choset, Kevin Lynch, Seth Hutchinson, George Kantor, Wolfram Burgard, Lydia Kavraki, and Sebastian Thrun A Bradford Book The MIT

More information

ELEC Dr Reji Mathew Electrical Engineering UNSW

ELEC Dr Reji Mathew Electrical Engineering UNSW ELEC 4622 Dr Reji Mathew Electrical Engineering UNSW Review of Motion Modelling and Estimation Introduction to Motion Modelling & Estimation Forward Motion Backward Motion Block Motion Estimation Motion

More information

Robert Collins CSE598G. Robert Collins CSE598G

Robert Collins CSE598G. Robert Collins CSE598G Recall: Kernel Density Estimation Given a set of data samples x i ; i=1...n Convolve with a kernel function H to generate a smooth function f(x) Equivalent to superposition of multiple kernels centered

More information

Structural Mechanics: Graph and Matrix Methods

Structural Mechanics: Graph and Matrix Methods Structural Mechanics: Graph and Matrix Methods A. Kaveh Department of Civil Engineering Technical University of Vienna Austria RESEARCH STUDIES PRESS LTD. Taunton, Somerset, England 0 JOHN WILEY & SONS

More information

Heteroskedasticity and Homoskedasticity, and Homoskedasticity-Only Standard Errors

Heteroskedasticity and Homoskedasticity, and Homoskedasticity-Only Standard Errors Heteroskedasticity and Homoskedasticity, and Homoskedasticity-Only Standard Errors (Section 5.4) What? Consequences of homoskedasticity Implication for computing standard errors What do these two terms

More information

Nonparametric Approaches to Regression

Nonparametric Approaches to Regression Nonparametric Approaches to Regression In traditional nonparametric regression, we assume very little about the functional form of the mean response function. In particular, we assume the model where m(xi)

More information

Topics in Machine Learning-EE 5359 Model Assessment and Selection

Topics in Machine Learning-EE 5359 Model Assessment and Selection Topics in Machine Learning-EE 5359 Model Assessment and Selection Ioannis D. Schizas Electrical Engineering Department University of Texas at Arlington 1 Training and Generalization Training stage: Utilizing

More information

Kernel Density Estimation

Kernel Density Estimation Kernel Density Estimation An Introduction Justus H. Piater, Université de Liège Overview 1. Densities and their Estimation 2. Basic Estimators for Univariate KDE 3. Remarks 4. Methods for Particular Domains

More information

Test Oracles and Randomness

Test Oracles and Randomness Test Oracles and Randomness UNIVERSITÄ T ULM DOCENDO CURANDO SCIENDO Ralph Guderlei and Johannes Mayer rjg@mathematik.uni-ulm.de, jmayer@mathematik.uni-ulm.de University of Ulm, Department of Stochastics

More information

TIME SERIES ANALYSIS FOR IRREGULARLY SAMPLED DATA. Piet M.T. Broersen. Department of Multi Scale Physics, Delft University of Technology

TIME SERIES ANALYSIS FOR IRREGULARLY SAMPLED DATA. Piet M.T. Broersen. Department of Multi Scale Physics, Delft University of Technology TIME SERIES ANALYSIS FOR IRREGULARLY SAMPLED DATA Piet M.T. Broersen Department of Multi Scale Physics, Delft University of Technology Abstract: Many spectral estimation methods for irregularly sampled

More information

Statistics 406 Exam November 17, 2005

Statistics 406 Exam November 17, 2005 Statistics 406 Exam November 17, 2005 1. For each of the following, what do you expect the value of A to be after executing the program? Briefly state your reasoning for each part. (a) X

More information

Aaron Daniel Chia Huang Licai Huang Medhavi Sikaria Signal Processing: Forecasting and Modeling

Aaron Daniel Chia Huang Licai Huang Medhavi Sikaria Signal Processing: Forecasting and Modeling Aaron Daniel Chia Huang Licai Huang Medhavi Sikaria Signal Processing: Forecasting and Modeling Abstract Forecasting future events and statistics is problematic because the data set is a stochastic, rather

More information

From Par(cle Filters to Malliavin Filtering with Applica(on to Target Tracking Sérgio Pequito Phd Student

From Par(cle Filters to Malliavin Filtering with Applica(on to Target Tracking Sérgio Pequito Phd Student From Par(cle Filters to Malliavin Filtering with Applica(on to Target Tracking Sérgio Pequito Phd Student Final project of Detection, Estimation and Filtering course July 7th, 2010 Guidelines Mo(va(on

More information

RESAMPLING METHODS. Chapter 05

RESAMPLING METHODS. Chapter 05 1 RESAMPLING METHODS Chapter 05 2 Outline Cross Validation The Validation Set Approach Leave-One-Out Cross Validation K-fold Cross Validation Bias-Variance Trade-off for k-fold Cross Validation Cross Validation

More information

1 Methods for Posterior Simulation

1 Methods for Posterior Simulation 1 Methods for Posterior Simulation Let p(θ y) be the posterior. simulation. Koop presents four methods for (posterior) 1. Monte Carlo integration: draw from p(θ y). 2. Gibbs sampler: sequentially drawing

More information

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1

Preface to the Second Edition. Preface to the First Edition. 1 Introduction 1 Preface to the Second Edition Preface to the First Edition vii xi 1 Introduction 1 2 Overview of Supervised Learning 9 2.1 Introduction... 9 2.2 Variable Types and Terminology... 9 2.3 Two Simple Approaches

More information

Lecture 27: Review. Reading: All chapters in ISLR. STATS 202: Data mining and analysis. December 6, 2017

Lecture 27: Review. Reading: All chapters in ISLR. STATS 202: Data mining and analysis. December 6, 2017 Lecture 27: Review Reading: All chapters in ISLR. STATS 202: Data mining and analysis December 6, 2017 1 / 16 Final exam: Announcements Tuesday, December 12, 8:30-11:30 am, in the following rooms: Last

More information

Three challenges in route choice modeling

Three challenges in route choice modeling Three challenges in route choice modeling Michel Bierlaire and Emma Frejinger transp-or.epfl.ch Transport and Mobility Laboratory, EPFL Three challenges in route choice modeling p.1/61 Route choice modeling

More information

LAB #2: SAMPLING, SAMPLING DISTRIBUTIONS, AND THE CLT

LAB #2: SAMPLING, SAMPLING DISTRIBUTIONS, AND THE CLT NAVAL POSTGRADUATE SCHOOL LAB #2: SAMPLING, SAMPLING DISTRIBUTIONS, AND THE CLT Statistics (OA3102) Lab #2: Sampling, Sampling Distributions, and the Central Limit Theorem Goal: Use R to demonstrate sampling

More information

Minima, Maxima, Saddle points

Minima, Maxima, Saddle points Minima, Maxima, Saddle points Levent Kandiller Industrial Engineering Department Çankaya University, Turkey Minima, Maxima, Saddle points p./9 Scalar Functions Let us remember the properties for maxima,

More information

The Bootstrap. Philip M. Dixon Iowa State University,

The Bootstrap. Philip M. Dixon Iowa State University, Statistics Preprints Statistics 12-2001 The Bootstrap Philip M. Dixon Iowa State University, pdixon@iastate.edu Follow this and additional works at: http://lib.dr.iastate.edu/stat_las_preprints Part of

More information

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources

Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Lincoln Laboratory ASAP-2001 Workshop Passive Differential Matched-field Depth Estimation of Moving Acoustic Sources Shawn Kraut and Jeffrey Krolik Duke University Department of Electrical and Computer

More information

Curves and Surfaces for Computer-Aided Geometric Design

Curves and Surfaces for Computer-Aided Geometric Design Curves and Surfaces for Computer-Aided Geometric Design A Practical Guide Fourth Edition Gerald Farin Department of Computer Science Arizona State University Tempe, Arizona /ACADEMIC PRESS I San Diego

More information

Analysis of Incomplete Multivariate Data

Analysis of Incomplete Multivariate Data Analysis of Incomplete Multivariate Data J. L. Schafer Department of Statistics The Pennsylvania State University USA CHAPMAN & HALL/CRC A CR.C Press Company Boca Raton London New York Washington, D.C.

More information

4.5 The smoothed bootstrap

4.5 The smoothed bootstrap 4.5. THE SMOOTHED BOOTSTRAP 47 F X i X Figure 4.1: Smoothing the empirical distribution function. 4.5 The smoothed bootstrap In the simple nonparametric bootstrap we have assumed that the empirical distribution

More information

Package npcp. August 23, 2017

Package npcp. August 23, 2017 Type Package Package np August 23, 2017 Title Some Nonparametric CUSUM Tests for Change-Point Detection in Possibly Multivariate Observations Version 0.1-9 Date 2017-08-23 Author Ivan Kojadinovic Maintainer

More information

Package iosmooth. R topics documented: February 20, 2015

Package iosmooth. R topics documented: February 20, 2015 Package iosmooth February 20, 2015 Type Package Title Functions for smoothing with infinite order flat-top kernels Version 0.91 Date 2014-08-19 Author, Dimitris N. Politis Maintainer

More information

Estimating and Sampling Graphs with Multidimensional Random Walks. Group 2: Mingyan Zhao, Chengle Zhang, Biao Yin, Yuchen Liu

Estimating and Sampling Graphs with Multidimensional Random Walks. Group 2: Mingyan Zhao, Chengle Zhang, Biao Yin, Yuchen Liu Estimating and Sampling Graphs with Multidimensional Random Walks Group 2: Mingyan Zhao, Chengle Zhang, Biao Yin, Yuchen Liu Complex Network Motivation Social Network Biological Network erence: www.forbe.com

More information

Generalized Additive Models

Generalized Additive Models :p Texts in Statistical Science Generalized Additive Models An Introduction with R Simon N. Wood Contents Preface XV 1 Linear Models 1 1.1 A simple linear model 2 Simple least squares estimation 3 1.1.1

More information

Nonparametric regression using kernel and spline methods

Nonparametric regression using kernel and spline methods Nonparametric regression using kernel and spline methods Jean D. Opsomer F. Jay Breidt March 3, 016 1 The statistical model When applying nonparametric regression methods, the researcher is interested

More information

Bootstrapping Methods

Bootstrapping Methods Bootstrapping Methods example of a Monte Carlo method these are one Monte Carlo statistical method some Bayesian statistical methods are Monte Carlo we can also simulate models using Monte Carlo methods

More information

The Cross-Entropy Method

The Cross-Entropy Method The Cross-Entropy Method Guy Weichenberg 7 September 2003 Introduction This report is a summary of the theory underlying the Cross-Entropy (CE) method, as discussed in the tutorial by de Boer, Kroese,

More information

Cross-validation. Cross-validation is a resampling method.

Cross-validation. Cross-validation is a resampling method. Cross-validation Cross-validation is a resampling method. It refits a model of interest to samples formed from the training set, in order to obtain additional information about the fitted model. For example,

More information

Statistical Matching using Fractional Imputation

Statistical Matching using Fractional Imputation Statistical Matching using Fractional Imputation Jae-Kwang Kim 1 Iowa State University 1 Joint work with Emily Berg and Taesung Park 1 Introduction 2 Classical Approaches 3 Proposed method 4 Application:

More information

Lab Session 1. Introduction to Eviews

Lab Session 1. Introduction to Eviews Albert-Ludwigs University Freiburg Department of Empirical Economics Time Series Analysis, Summer 2009 Dr. Sevtap Kestel To see the data of m1: 1 Lab Session 1 Introduction to Eviews We introduce the basic

More information

Nonparametric Importance Sampling for Big Data

Nonparametric Importance Sampling for Big Data Nonparametric Importance Sampling for Big Data Abigael C. Nachtsheim Research Training Group Spring 2018 Advisor: Dr. Stufken SCHOOL OF MATHEMATICAL AND STATISTICAL SCIENCES Motivation Goal: build a model

More information