Bayesian inference for sample surveys

Transcription

1 Bayesan nference for sample surveys Roderck Lttle Module 3: Bayesan models for complex sample desgns

2 Modelng sample selecton Role of sample desgn n model-based (Bayesan) nference Key to understandng the role s to nclude the sample selecton process as part of the model Modelng the sample selecton process Smple and stratfed random samplng Cluster samplng, other mechansms See Chapter 7 of Bayesan Data Analyss (Gelman, Carln, Stern and Rubn 1995) Models for complex sample desgns 2

3 Formal models that nclude data collecton Y ( y1,..., yn ) = populaton data; y may be a vector Z fully-observed covarates, desgn varables Q Q( Y, Z) = fnte populaton quantty I ( I1,..., I N ) = Sample Incluson Indcators R y I S 1, observed T 0, otherwse Y ( Ync, Yexc) Y ncluded part of Y, Y = excluded part of Y nc Notaton mples Stable Unt Treatment Value Assumpton (SUTVA): Values not affected by choce of ncluson vector I exc Models for complex sample desgns 3

4 Full model for Y and I py (, I Z,, ) p( Y Z, ) p( I Y, Z, ) Model for Populaton Observed data: Ync Z I Observed-data lkelhood: Posteror dstrbuton of parameters: Model for Incluson (,, ) (No mssng values) L(, Y, Z, I) p( Y, I Z,, ) p( Y, I Z,, ) dy nc nc exc p(, Y, Z, I) p(, Z) L(, Y, Z, I) nc nc Models for complex sample desgns 4

5 Ignorng the data collecton process The lkelhood gnorng the data-collecton process s based on the model for Y alone wth lkelhood: L( Y, Z) p( Y Z, ) p( Y Z, ) dy nc nc exc The correspondng posterors for and nc are: When the full posteror reduces to ths smpler posteror, the data collecton mechansm s called gnorable for Bayesan nference about. nc p( Y, Z) p( Z) L( Y, Z) Y exc p( Yexc Ync, Z) p( Yexc Ync, Z, ) p( Ync, Z) d Posteror predctve dstrbuton of,y exc Y exc Models for complex sample desgns 5

6 Condtons when data collecton mechansm can be gnored Two general and smple suffcent condtons for gnorng the data-collecton mechansm are: Selecton at Random (SAR): p( I Y, Z, ) p( I Ync, Z, ) for all Yexc. Bayesan Dstnctness: p(, Z) p( Z) p( Z) It s easy to show that these condtons together mply that: p(, Y Y, Z) p(, Y Y, Z, I) exc nc exc nc so the model for the data-collecton mechansm does not affect nferences about the parameter or fnte populaton quanttes Q. Models for complex sample desgns 6

7 Ex: smple random samplng For Smple Random Samplng, the samplng dstrbuton s: piy (, ) R S F T 1 HG NI N, I n n K J f ; 1 0, otherwse. Ths s clearly gnorable, wth Z null. Ths justfes gnorng the mechansm n Module 2 Models for complex sample desgns 7

8 Bayes nference for probablty samples In other probablty samplng desgns, selecton does not depend on values of Y and the mechansm s known, that s: p( I Y, Z, ) p( I Z) for all Y. Ths means that the data-collecton mechansm s gnorable for Bayesan nference (wth complete data) But the model needs to approprately account for relatonshp of survey outcomes Y wth the desgn varables Z. Consder how to do ths for (a) unequal probablty samples, and (b) clustered (multstage) samples Models for complex sample desgns 8

9 Models for unequal probablty samples Approprate analyss depends on how the varables leadng to the desgn weghts enter the model of substantve nterest (a) all are ncluded (b) some are ncluded, others aren t (c) none are ncluded Consder these dstnctons for (a) means and (b) regresson coeffcents Models for complex sample desgns 9

10 Desgn-based weghtng A pure form of desgn-based estmaton s to weght sampled unts by nverse of ncluson probabltes Sampled unt represents w 1/ unts n the populaton More generally, a common approach s: w w w ( w ) w ( w, w ) w w s n s p s n s n samplng weght ( w ) nonresponse weght s w ( w, w ) post-stratfcaton weght p s n Models for complex sample desgns 10

11 Weghtng and models The weghts can t generally be gnored from a modelng perspectve Ignores dfferent selecton effects that bas estmates Weghts are auxlary covarates from a modelng perspectve Desgn: weght the respondents One sze fts all Y varables Model: use weghts to help predct non-sampled and non-respondng values Weghtng adds nose for Y s unrelated to weghts The model perspectve s more flexble (but potentally more work) Models for complex sample desgns 11

12 Ex 1: stratfed random samplng Populaton dvded nto J strata Z s set of stratum ndcators: z 1, f unt s n stratum j; 0, otherwse. Sample Populaton Z Y Z Stratfed random samplng: smple random sample of n j unts selected from populaton of N j unts n stratum j. Ths desgn s gnorable provdng model for outcomes condtons on the stratum varables Z. Models for complex sample desgns 12

13 Inference for a mean from a stratfed sample Consder a model that ncludes stratum effects: 2 For smplcty assume j s known and the flat pror: Standard Bayesan calculatons lead to where: st J j1 p( Z) const. j [ Y Y, Z,{ }]~ N( y, ) 2 2 nc j st st y P y, P N / N, y sample mean n stratum j, 2 [ y z j]~ nd N( j, j) j j j j j J st Pj (1 f j) j / nj, f j nj / N j j1 Models for complex sample desgns 13

14 Bayes for stratfed normal model Bayes nference for ths model s equvalent to standard classcal nference for the populaton mean from a stratfed random sample The posteror mean weghts case by nverse of ncluson probablty: st J J 1 1 j j j j1 j1 : x j y N N y N y /, where n / N selecton probablty n stratum j. j j j Wth unknown varances, Bayes for ths model wth flat pror on log(varances) yelds useful t-lke correctons for small samples Models for complex sample desgns 14

15 Suppose we gnore stratum effects? Suppose we assume nstead that: 2 [ y z j]~ nd N(, ), the prevous model wth no stratum effects. Wth a flat pror on the mean, the posteror mean of Y s then the unweghted mean J 2 EY ( Ync, Z, ) y p jy j, pj nj / n j1 Ths s potentally a very based estmator f the selecton rates j n j / N j vary across the strata The problem s that results from ths model are hghly senstve volatons of the assumpton of no stratum effects and stratum effects are lkely n most realstc settngs. Hence prudence dctates a model that allows for stratum effects, such as the model n the prevous slde. Models for complex sample desgns 15

16 Desgn consstency Loosely speakng, an estmator s desgn-consstent f (rrespectve of the truth of the model) t converges to the true populaton quantty as the sample sze ncreases, holdng desgn features constant. For stratfed samplng, the posteror mean y st based on the stratfed normal model converges to Y, and hence s desgnconsstent For the normal model that gnores stratum effects, the posteror mean y converges to J J Y jn jyj / N j1 j 1 j j and hence s not desgn consstent unless j const. We generally advocate Bayesan models that yeld desgnconsstent estmates, to lmt effects of model msspecfcaton Models for complex sample desgns 16

17 Target and workng models I thnk t s helpful to dstngush between Target model: the model that determnes the target parameter/quantty of nterest Workng model: the model used to model the data (.e. to predct the non-sampled values n the populaton) In our smple settng, target model does not condton 2 on Z: [ y z j]~ N(, ) nd Target quantty, the overall populaton mean, results from fttng ths model to whole populaton Workng model needs to condton on Z y z j N 2 [ ]~ nd ( j, j) Models for complex sample desgns 17

18 Weghtng n regresson In multple lnear regresson, standard method of estmaton s ordnary least squares (OLS) Model-based: If resdual varance s not constant, weght by nverse of resdual varance y u u 2 Var( ) / weghted LS wth weght Desgn-based: OLS wrong, weght by nverse of probablty of selecton, w 1/ Whch s rght? Need to consder varables leadng to the samplng weght, and how they enter the regresson model Models for complex sample desgns 18

19 Regresson wth sample weghts Target model: T 2 y x ~ N( 0 x, / u), u known (constant for OLS) Target parameter: Correspondng fnte populaton parameter: B = result of fttng model to the entre populaton z desgn varables leadng to samplng weghts (stratum, sze n pps sample) Consder three cases: (a) z ncluded as part of x (b) z not a part of x (c) z ( z, z ), z a part of x, z not a part of x Models for complex sample desgns 19

20 Regresson wth sample weghts (a) z ncluded as part of x If workng model s correctly specfed, then regresson wth weght u s correct no need to nclude the sample weght Desgn-weghted regresson wth weght uw yelds a desgn-consstent estmate of the target populaton quantty B. If ths dffers markedly from model estmate wth weght u, ths suggests model s msspecfed, and assumptons need checkng. Models for complex sample desgns 20

21 Regresson wth sample weghts (b) z not a part of x u Workng model wth weght s subject to a known selecton bas arsng from the stratfed desgn only vald f ths selecton does not affect the target parameter estmate Prncpled modelng approach s to regress y on x and z and then average over the dstrbuton of z gven x; e.g. f E( y x, z ) x z then E( y x ) x E( z x, ), etc Bayes smulaton: mpute draws of the non-sampled values of Y based on regresson of Y on X, Z, and then ft regresson of Y on X to mputed populaton. Repeat to smulate posteror dstrbuton of Models for complex sample desgns 21

22 Regresson wth sample weghts (b) z not a part of x Pragmatc approach: desgn-based regresson of y on x wth weghts wu Model-based justfcaton: assume a workng model wth a dfferent regresson model for y on x wthn each stratum defned by Z. Regresson of y on x wth weght wu then approxmates the posteror mean of. (Lttle 2004, Example 11) Pragmatc approach B: compare regresson of y on x wth weghts wu wth regresson of y on x wth weghts u. If coeffcents of nterest are close, effects of selecton may be gnored, leadng to model-based soluton. Models for complex sample desgns 22

23 Regresson wth sample weghts (c) z ( z, z ), z a part of x, z not a part of x Prncpled modelng approach s to regress y on x and z and then average over the dstrbuton of z gven x; e.g. f E( y x, z ) x z then E( y x ) x E( z x, ), etc. Bayes smulaton: mpute draws of the non-sampled values of Y based on regresson of Y on X, Z, and then ft regresson of Y on X to mputed populaton. Repeat to smulate posteror dstrbuton of 2 Models for complex sample desgns 23

24 Regresson wth sample weghts (c) z ( z, z ), z a part of x, z not a part of x Pragmatc approach: desgn-based regresson of y on x wth weghts wu, where w s component of samplng weght 2 2 attrbutable to z (gven z ). 2 1 (Weghtng on wu s ok but neffcent) Pragmatc approach B: compare regresson of y on x wth weghts wu wth regresson of y on x wth weghts u. If coeffcents of nterest are close, effects of selecton may be gnored, leadng to model-based soluton. Models for complex sample desgns 24

25 y HT Ex 4. One contnuous (post)stratfer Z Consder PPS samplng, Z = measure of sze Standard desgn-based estmator s weghted Horvtz-Thompson estmate y y n 1 y / ; selecton prob (HT) N 1 model-based predcton estmate for HT 2 2 ~ Nor(, ) ("HT model") Sample Populaton Z Y Z When the relatonshp between Y and Z devates a lot from the HT model, HT estmate s neffcent and CI s can have poor coverage Models for complex sample desgns 25

26 Ex. Basu s neffcent elephants 1 50 y,..., y weghts of N 50 elephants Objectve: T y y... y. Only one elephant can be weghed! Crcus traner wants to choose average elephant (Sambo) Crcus statstcan requres scentfc prob. samplng: Select Sambo wth probablty 99/100 One of other elephants wth probablty 1/4900 Sambo gets selected! Traner: Statstcan requres unbased Horvtz-Thompson (1952) estmator: HT estmator s unbased on average but always crazy! HT model s clearly hopeless here Models for complex sample desgns 26

27 What went wrong? HT estmator optmal under an mplct HT model that have the same dstrbuton That s clearly a slly model gven ths desgn Whch s why the estmator s slly y / Models for complex sample desgns 27

28 Ex 4. One contnuous (post)stratfer Z y wt 1 N n 1 y / ; selecton prob (HT) Sample Populaton Z Y Z A modelng alternatve to the HT estmator s create predctons from a more robust model relatng Y to Z: n N 1 y ˆ ˆ mod = y y, y predctons from: N 1 n1 2 k ~ Nor( ( ), ); ( ) = penalzed s y S S (Zheng and Lttle 2003, 2005) plne of Y on Z Models for complex sample desgns 28

29 Smulaton: PPS samplng n 6 populatons Models for complex sample desgns 29

30 Estmated RMSE of four estmators for N=1000, n=100 Populaton model wt gr NULL Normal Lognormal LINUP Normal Lognormal LINDOWN Normal Lognormal SINE Normal Lognormal EXP Normal Lognormal Models for complex sample desgns 30

31 95% CI coverages: HT Populaton V1 V3 V4 V5 NULL LINUP LINDOWN SINE EXP ESS V1 Yates-Grundy, Hartley-Rao for jont ncluson probs. V3 Treatng sample as f t were drawn wth replacement V4 Parng consecutve strata V5 Estmaton usng consecutve dfferences Models for complex sample desgns 31

32 95% CI coverages: B-splne Populaton V1 V2 V3 NULL LINUP LINDOWN SINE EXP ESS Fxed wth more knots V1 Model-based (nformaton matrx) V2 Jackknfe V3 BRR Models for complex sample desgns 32

33 Why does model do better? Assumes smooth relatonshp HT weghts can bounce around Predctons use szes of the non-sampled cases HT estmator does not use these Often not provded to users (although they could be) Lttle & Zheng (2007) also show gans for model when szes of non-sampled unts are not known Predcted usng a Bayesan Bootstrap (BB) model BB s a form of stochastc weghtng Models for complex sample desgns 33

34 Z 1 Ex 3. One stratfer, one post-stratfer Desgn-based approaches (A) Standard weghtng s w w w ( w ) s p s Notes: (1) Z proportons are not matched! 1 * (2) why not w wpws( wp)? (B) Devlle and Sarndal (1992) modfes samplng weghts { ws} to adjusted weghts { w} that match poststratum margn, but are close to { ws} wth respect to a dstance measure dw ( s, w). Questons: What s the prncple for choosng the dstance measure? Should the { w} necessarly be close to { w }? Z 2 Sample Populaton Z1 Z2 Y Z1 Z2 s Models for complex sample desgns 34

35 Ex 3. One stratfer Z, one post-stratfer 1 Z 2 Model-based approach Saturated model: { n } ~ MNOM( n, ); y jk 2 jk ~Nor( jk, jk ) jk Sample Populaton Z1 Z2 Y Z1 Z2 y Pˆ y w n y / w n mod J K J K J K jk jk jk jk jk jk jk j1 k1 j1 k1 j1 k1 n sample count, y sample mean of Y jk jk Pˆ proporton from rakng (IPF) of { n } jk to known margns { P },{ P } j k w npˆ / n = model weght jk jk jk jk Z 1 Z 2 { n jk } { Pj } { P k } Models for complex sample desgns 35

36 Ex 3. One stratfer, one post-stratferz2 Model-based approach y Pˆ y w n y / w n st J K J K J K jk jk jk jk jk jk jk j1 k1 j1 k1 j1 k1 What to do when n s small? jk Z 1 Model: replace y by predcton from modfed model: jk 2 e.g. y jk ~ Nor( j k jk, jk ), J K 2 j k 0, jk ~ Nor(0, ) (Gelman 2007) j1 k1 Sample Populaton Z1 Z2 Y Z1 Z2 2 Settng = 0 yelds addtve model, otherwse shrnks towards addtve model Desgn: arbtrary collapsng, ad-hoc modfcaton of weght Models for complex sample desgns 36

37 Two stage samplng Most practcal sample desgns nvolve selectng a cluster of unts and measure a subset of unts wthn the selected cluster Two stage sample s very effcent and cost effectve But outcome on subjects wthn a cluster may be correlated (typcally, postvely). Models can easly ncorporate the correlaton among observatons Models for complex sample desgns 37

38 Sample desgn: Ex 4. Two-stage samples Stage 1: Sample c clusters from C clusters Stage 2: Sample k unts from the selected cluster =1,2,,c K N Populaton sze of cluster C K 1 Estmand of nterest: Populaton mean Q Infer about excluded clusters and excluded unts wthn the selected clusters Models for complex sample desgns 38

39 Models for two-stage samples Model for observables Y N C j K 2 j ~ (, ); 1,..., ; 1, 2,..., 2 ~ d N(, ) Assume and are known Pror dstrbuton ( ) 1 Models for complex sample desgns 39

40 Estmand of nterest and nference strategy The populaton mean can be decomposed as c NQ [ k y ( K k ) Y ] KY,exc 1 c1 Posteror mean gven Y nc c C ENQY (,, 1,2,..., c; ) [ ky( Kk) ] K nc 1 c1 c C ENQY ( ) [ ky( Kk) E( Y )] KE( Y ) where E( Y ) ˆ E( Y ) nc nc nc 1 c1 2 2 y( k / ) (1/ ) nc 2 2 k / 1/ y ˆ /( / k ) 2 2 nc 2 1/( 2 / k ) Models for complex sample desgns 40 C

41 Posteror Varance Posteror varance can be easly computed c C ( nc) ( )( ( ) ) ( ) 1 c1 Var NQ Y K k K k K K VarY ( Y ) EVarY [ ( Y, ) Y ] VarEY [ ( Y, ) Y ] K k,exc nc,exc nc nc,exc nc nc 2 2, 1,2,, c Var( Y Y ) E[ Var( Y Y, ) Y ] Var[ E( Y Y, ) Y ] nc nc nc nc nc 2 2 / K, c 1, c 2,, C Models for complex sample desgns 41

42 Inference wth unknown and For unknown and Opton 1: Plug n maxmum lkelhood estmates. These can be obtaned usng PROC MIXED n SAS. PROC MIXED actually gves estmates of and E( Y nc ) (Emprcal Bayes) Opton 2: Fully Bayes wth addtonal pror v 2 (,, ) exp b /(2 ) where b and v are small postve numbers Models for complex sample desgns 42

43 Extensons and Applcatons Relaxng equal varance assumpton 2 Yl ~ N(, ) (,log )~d BVN(, ) Incorporatng covarates (generalzaton of rato and regresson estmates) 2 Yl ~ N( xl, ) (,log )~d MVN(, ) Small Area estmaton. An applcaton of the herarchcal model. Here the quantty of nterest s EY ( Y ) ( ky ( K k ) E( Y Y )) / K nc,exc nc Models for complex sample desgns 43

44 Extensons Relaxng normal assumptons Y h x v v : a known functon ~ d MVN(, ) 2 l ~Glm( ( l ), ( )) Incorporate desgn features such as stratfcaton and weghtng by modelng explctly the samplng mechansm. Models for complex sample desgns 44

45 Summary Bayes nference for surveys must ncorporate desgn features approprately Stratfcaton and clusterng can be ncorporated n Bayes nference through desgn varables Unlke desgn-based nference, Bayes nference s not asymptotc, and delvers good frequentst propertes n small samples Models for complex sample desgns 45