Nonparametric Density Estimation

Transcription

1 Nonparametric Estimation Data: X 1,..., X n iid P where P is a distribution with density f(x). Aim: Estimation of density f(x) Parametric density estimation: Fit parametric model {f(x θ) θ Θ} to data parameter estimate ˆθ Estimate f(x) by f(x ˆθ) Problem: Choice of suitable model danger of misfits Complex models (eg mixtures) are difficult to fit Nonparametric density estimation: Few assumptions (eg density is smooth) Exploratory tool Example: Velocities of galaxies Velocities in km/sec of 82 galaxies from 6 well-separated conic sections of an unfilled survey of the Corona Borealis region. Multimodality is evidence for voids and superclusters in the far universe. 5 0 Kernel estimate (h=0.814) Kernel estimate (h=42) Normal mixture model (k=4) of galaxy (1000km/s) Kernel Estimation, May 20,

2 Histogram Histogram estimator For constants a 0 and h, let a k = a 0 + k h and H k = # { X i Xi (a k 1, a k ] } be the number of observations in the kth interval (a k 1, a k ]. Then ˆf hist (x) = 1 hn n H k 1 (ak 1,a k ](x) k=1 is the histogram estimator of f(x). Advantages: Easy to compute Disadvantages: Sensitive in choice of offset a 0 Nonsmooth estimator Five shifted histograms with bin width and the averaged histogram, for the duration of eruptions of the Old Faithful geyser. Kernel Estimation, May 20,

3 Centered Histogram Aim: Estimate density f(x) at point x Idea: Shift histogram to be centered on x ˆf rect (x) = 1 hn #{ X i X i (x h/2, x + h/2] } Advantages: Exact computation (and plot) of estimate for all x Only depends on one parameter: Bin width h Disadvantages: Can yield very noisy estimates Nonsmooth estimator 1 The centered histogram estimator can be rewritten as ˆf rect (x) = 1 n ( ) 1 n h K x Xi h where i=1 K(x) = 1 ( 1 2, 1 2 ] (x) is the indicator function for the interval ( 1 2, 1 2 ]. The function K is called a kernel or filter. use different (smooth) kernel functions K(x) Kernel Estimation, May 20,

4 Let K(x) be a function such that K(x) 0, K(x) dx = 1. Kernel Estimators Then the kernel density estimators with kernel K() and bandwidth h is given by ˆf K (x) = 1 hn n ( x Xi ) K. h i=1 Common kernel functions: Rectangular kernel Rectangular kernel Triangular kernel data Triangular kernel Normal kernel data Normal kernel data Kernel Estimation, May 20,

5 Statistical properties The expectation of ˆf K (x) is E( ˆf K (x)) = = 1 Kernel Estimators ( h K x y h ) f(y) dy K(z) f(x hz) dz = f(x) + O(h 2 ). The bias of ˆf K (x) decreases as h gets smaller. The variance of ˆf K (x) is var( ˆf K (x)) f(x) nh K(x) 2 dx. The variance of ˆf K (x) vanishes as nh. Conclusions: Restrictions on bandwidth: h 0 and nh as n. Theory suggests that h n 1 5, but the constant of proportionality depends on the unknown density. Trade-off between bias and variance: Undersmoothing If bandwidth is too small, the variance becomes large. Oversmoothing If bandwidth is too large, the bias becomes large. Kernel Estimation, May 20,

6 Kernel Estimators Examples: Old Faithful and Galaxies 1.0 h=3 5 h= h=6 0 h= h=2 h= h=4 5 h= h=8 2 h= h= h= Kernel Estimation, May 20,

7 Kernel Estimates How to do it in R? In R, kernel density estimates can be computed by the command density(): plot(density(y,bw=,method="gaussian"),type="l") As default h is chosen according to the following rule of thumb ĥ = 0.9 min(s, R/1.34) n 1 5 where s is the sample standard deviation and R is the interquartile range. Better methods for selecting h are due to eg Sheather and Jones (1991) and can be invoked by the command bw.sj: h<-bw.sj(y) plot(density(y,h)) h<-bw.sj(y,method="dpi") plot(density(y,h)) # solve-the-equation method # direct-plug-in method Rule of thumb h=35 Direct plug in h=65 Solve the equation h=4 1 Kernel Estimation, May 20,