Using Auxiliary Data for Adjustment In Longitudinal Research. Dirk Sikkel Joop Hox Edith de Leeuw

Transcription

1 Usng Auxlary Data for Adjustment In Longtudnal Research Drk Skkel Joop Hox Edth de Leeuw 1. Longtudnal Research The use of longtudnal research desgns, such as cohort or panel studes, has become more and more popular durng the last decade. Both cohort and panel desgns are based on measurng the same sample of ndvduals at dfferent ponts n tme, wth the goal to study change over tme. Cohort studes use subjects who have certan characterstcs n common, for example a brth cohort of subjects who were all born between 1980 and Panel studes are smply studes that use the same subjects repeatedly. Increasngly, access panels are used for cross sectonal research, because t s convenent to have a group of respondents that can be reached easly and be used as a samplng frame from whch sub groups can be selected. Ths makes t quck and convenent to approach specfc respondents for whch certan key questons are relevant. These access panels have become ncreasngly popular, as llustrated by the database of ESOMAR, the nternatonal organzaton of marketng research agences. In June 2004, 1035 organzatons were lsted that carry out panel research on a contnuous bass. In addton, there are the governmental, sem-governmental and academc agences that use panels, such as natonal statstcal offces and unverstes. Whereas most commercal panels are based on the Internet, most non-commercal panels are not. Famous, trend-settng face to face panels n the USA are PSID (Panel Study of Income Dynamcs), CPS (Current Populaton Survey, desgned to keep track of the labour market) and SIPP (Survey of Income and Program Partcpaton, one of ts man research topcs s the transfer of government money to ndvduals). In ths chapter, we use the term panel to refer n general to any desgn n whch ndvdual subjects are repeatedly used. Ths ncludes longtudnal research where subjects are followed for many years, but also access panels whose members agree to respond to a certan number of questonnares only when ther nformaton s needed. Ideally, a panel conssts of subjects that are a probablty sample of the populaton that the researcher has n mnd, and remans ntact durng the lfetme of the panel. In realty, many panels are a volunteer panel, whch makes t dffcult to assess whether t s representatve for a larger populaton, and all panels suffer from panel attrton: panel members who cease to cooperate wth the study. If the researcher has auxlary data on the panel members that can be compared to data from the general populaton, a number of remedes are possble. The purpose of ths chapter s to present and dscuss what can be done to prevent or repar the consequences of nonresponse n longtudnal studes. 2. Types of Samples n Longtudnal Research When a panel conssts of volunteers, the departure from a probablty sample of such a group s unknown. In hs typology of web surveys, Couper (2000) draws an mportant dstncton between probablty and non-probablty surveys, and dentfes three nterestng categores of 1

2 panel surveys. A volunteer opt-n panel refers to the stuaton where vstors of well-used web stes are asked to regster for a panel, and leave basc demographc nformaton that can be used to select respondents from the database n a later stage. Many non-nternet volunteer panels are constructed n a smlar way; subjects are approached, often usng a quota samplng procedure, and asked to partcpate n a research panel. Although the subsequent respondent selecton s generally based on probablty samplng, the ntal panel conssts entrely of volunteers. The stuaton s slghtly dfferent n the case of the two types of pre-recruted panels. Here the panelsts are recruted usng tradtonal samplng technques, for nstance n a random dgt dallng telephone survey. Durng the telephone ntervew bass background nformaton s collected, that s used to dentfy elgble respondents. In pre-recruted panels of the total populaton one recruts the panel from the full populaton. In pre-recruted panels of nternet users one s nterested n obtanng a probablty sample only of those people that have access to the Internet. Internet access panels can be any of the three types mentoned before; t n a pre-recruted panel of the total populaton a subject s drawn who does not have Internet access the researchers just provde that. Except for the volunteer opt-n panels, all forms of panel research have n common that there exsts the opportunty to obtan unbased populaton estmates for a large number of varables durng the recrutment process, provded ths process s such that the ncluson probabltes of the respondents are known. Ths creates the opportunty to use such auxlary data n a later stage to adjust for ntal nonresponse and panel attrton. In addton, durng the lfetme of the panel further nformaton can be collected, whch can also be used for the purpose of adjustment. 3. Mssng data Compared to the deal sample, wth known ncluson probabltes, data may be mssng due to many causes. The most crucal stage n longtudnal research s the recrutment stage, where canddates for panel membershp may fal to partcpate for a varety of reasons. An llustratve example s the CentERpanel, located at Tlburg Unversty. The recrutment procedure was based on a random sample of phone numbers. What happened after the frst call s descrbed n table 2. Table 1. Response rates n dfferent stages of recrutment n the CentERpanel % cumulatve % phone number usable partcpaton frst contact ntervew prepared to take part n follow up ntervews phone number correct n membershp ntervew partcpaton membershp ntervew respondent qualfes as a member prepared to become a member From all selected phone numbers, 98.2% was usable. From the potental respondents who were contacted, 38.4% refused the frst contact ntervew. The data of those who dd take part, and were prepared to take part n follow up ntervews, were stored n a data base and later retreved when the respondents were asked to take part n the panel. In the end, 10.4% of the ntal phone numbers became panel members. Most commercal research nsttutes do not keep track of the recrutment process n ths detal, but t seems plausble that ther response 2

3 rates are at best n the same order of magntude. Of course, n specalzed, non-commercal panels, response rates may be much hgher. Once respondents have become a panel member, new sources of mssng data arse. The panel members may fal to answer sngle questons (tem non response), skp panel waves (wave non response) or drop out of the panel (attrton). A specal case of attrton by purpose s a rotatng panel, n whch respondents are forced to leave the panel after a fxed amount of tme. g g g g g g g g t 1 t 2 t 3 t 4 t 5 t 6 t 7 t 8 t 9 t 10 t 11 Fgure 1. Scheme for a rotatng panel Fgure 1 shows a smple rotaton scheme. At t 1 measurements are based only on group g 1. At t 2 statstcs are estmated usng group g 1 and g 2. At t 4, four groups are beng used: g 1, g 2, g 3 and g 4. Then group g 1 leaves the panel. At t 5 groups g 2, g 3, g 4 and g 5 are used for estmaton. Note that an estmate for the dfference between t 1 and t 5 s based on a combnaton of the dependent estmates of the groups g 2, g 3, g 4 and the ndependent estmates of groups g 1 and g 5. There s a host of technques that can be used for dealng wth mssng data. Bascally there are three strateges: - preventon, by panel management. Specal attenton to members of the panel may prevent droppng out. Such measures may nclude small presents, but also specfc tokens of attenton, for nstance sendng a brthday card at the approprate tme. - damage repar, by mputaton. Ths can be acheved by fndng smlar cases n whch there are no mssng data or by estmatng a model y = f(x ) + ε, n whch y s a mssng value for ndvdual, x s a vector of known values for ndvdual, and ε s a random error term. The randomness may be gnored by mputng the expected value f(x ) or accounted for by addng a random term or by multple mputaton. Compared to cross sectonal research, n panel research there are usually many more varables avalable for the model f(x). The best predctor of y s often the correspondng value n the prevous panel wave. - damage control, by calbraton. Ths nvolve assgnng weghts w to the ndvdual cases such that known populaton parameters or unbased estmators of populaton parameters are reproduced by the estmaton procedure. So for a wave W for auxlary varable x k the weghts w are determned n such a way that 3

4 W w x k = t k where x k s the value of ndvdual on varable x k and t k s the populaton parameter determned by x k at tme, e.g. a means or a total. In the case when complete records are mssng, lke wave non response or attrton, calbraton s the most obvous way to handle mssng data, as there are no auxlary varables measured at tme, whch can be used to predct the ndvdual value y at tme. The calbraton procedure mples the assumpton that the mssng data are mssng at random (see Lttle and Rubn, 2002), condtonal on the varables whch determne the weghts. In general, longtudnal research offers golden opportuntes both for mputaton and for calbraton, because there are so many auxlary varables avalable. In ths chapter we concentrate mostly on calbraton, whch s more suted to use n automated routnes than mputaton, and therefore has better chances to be appled on a routne bass. 4. Calbraton There are dfferent ratonales behnd calbraton. Perhaps the smplest of them s the noton of representatves. An estmaton procedure Θ(.) s representatve wth respect to a matrx of auxlary varables X when Θ(X) produces a vector of known populaton parameters t. (Hajek, 1981, p. 40). Ths defnton apples to the routne weghtng procedures wth respect to categorcal varables lke age (categores), sex and regon, but also to totals and averages of numercal varables lke savngs and ncome. Bascally, weghtng s meant to reduce bas, by makng the estmaton procedure for known populaton totals unbased. Under certan condtons, e.g. n a random sample n whch the data satsfy a well specfed model, weghtng may also result n varance reducton. The clearest expresson of ths vew s the well known regresson estmator, whch s an mplct procedure to weght the data. Let 1 v = ( X'X) t, then v s the vector of weght components,.e. the vector of weghts w can be wrtten as or w = Xv, (1) w = xkvk, (2) k n other words, the weghts are lnear combnatons of the auxlary varables. Bethlehem and Keller (1987) show that n a smple random sample the varance of a target varable y after weghtng s equal to σ yreg 2 = (1-ρ 2 )σ y 2, (3) where ρ s the multple correlaton between x and y and σ y 2 s the unweghted varance of y. When data are mssng at random, gven X, the weghtng procedure yelds unbased estmates. Devlle and Särndal (1992) show that the regresson estmator s a specal case of a broad class of weghtng procedures whch s based on mnmzng a dstance measure Σ D(w,π -1 ) under the condton w X=t ; verbally ths means that the w have to stay as close 4

5 as possble to the recprocal ncluson probabltes π, under the condton that the populaton totals for the auxlary varables are reproduced. The choce of dstance measure D(.) determnes the the type of weghts w. For the regresson estmator the dstance measure D s 1 ( π ) 2 D( w, π w (4) 1 ) = π Devlle and Särndal show that the well known procedure of teratve proportonal fttng s also a specal case of ths class of weghtng procedures. They also show that the varance (3) s the asymptotc frst order approxmaton n the total class of weghtg procedures. In ths chapter we restrct ourselves to the regresson estmator, as t s the most flexble and easy to use procedure. It can be appled to both numercal and categorcal varables, n the latter case by usng dummy varables for each of the categores. The same procedures can also be appled to populaton averages nstead of totals. In that case the weghted auxlary varables add to t/n, and the dstance measure D s the functon D(w,(nπ ) -1 ), where N s the populaton sze and n the sample sze. Ths s, however, only one part of the story. When the target varable y s unrelated to X, the varance of the weghted totals ncreases compared to the unweghted case wth a factor 1+CV w 2, where CV w s the varaton coeffcent of the weghts w, se e.g. Ksh (1992). Ths s a ratonale for the practce of trmmng weghts: to change weghts whch have excessve values to smaller weghts. Ths can be done after ntal weghts have been computed (Potter, 1988, 1990, 1993) or by puttng restrctons on the weghts n the estmaton procedure. Such algorthms, agan, fall nto the class of weghtng procedures descrbed by Devlle and Särndal (1992). For practcal examples see also Kalton and Flores-Cervantes (2003). For research agences that are operatng pre-recruted panels, equaton (3) presents an excellent opportunty to calbrate panel waves or cross sectonal research projects. What s needed s to collect wthn the recrutment process data wth respect to a varety of varables that can be expected to have hgh correlatons wth key varables n future research projects. Ths recrutment process can be a face to face survey based on admnstratve populaton data or a survey by phone usng random dgt dallng. When the response levels n the recrutment process are hgh, the qualty of the estmators can also be hgh. Of course, the key varables n future research projects may be unknown n the recrutment stage. But usually a research organzaton works n a lmted number of felds, n whch t s qute clear what the key varables are. For a general data collecton organzaton lke CentERdata, whch has ts man focus on scentfc and polcy ssues, a number of varables whch are collected n the recrutment stage are lsted n table 3. The percentages measured n the recrutment stage are compared to those n a panel wave n may Table 3. Dstrbutons of some calbraton varables n the recrutment stage and n the CentERpanel, 1999 Recrutment % Panel % number of rooms n the house 1-3 rooms rooms rooms or more rooms travellng tme to work 5

6 Recrutment % Panel % >20 mnutes <20 mnutes satsfacton wth health (1-10) 1 thru , vsted the cnema last year yes no member of sport club yes no vctm burglary (ever) yes no The choce of the varables s such that they gve nformaton about housng, commutng behavour, health, cultural partcpaton and vctmzaton. Table 3 shows that members of sport clubs are severely underrepresented n the panel, whereas vctms of burglary are overrepresented. The choce of varables to be measured n the recrutment stage clearly has a large substantve component, but the fnal crteron s a statstcal one: equaton (3). 5. Calbratng multple waves When a research project conssts of more than one wave, a straghtforward way to proceed s to treat each wave separately and to construct weghts for each wave. For very smple forms of analyss ths may be a possble approach, but t precludes the smultaneous use of varables from dfferent waves as each would then have a dfferent set of weghts. So t s preferable when respondents whch contrbute to dfferent waves have one sngle weght n each of the waves. Only a slght modfcaton of the theory n secton 3 s requred to do ths trck. Let us assume we have waves W 1, W 2,, W q, wth n 1, n 2,, n q respondents. Panel members may have mssng values n a subset of the waves. The total number of respondents, who partcpate n at least one of the waves, s equal to n. For each wave W a populaton total t (or a populaton mean μ = t /N ) s gven of a vector of auxlary varables x, =1, 2, q. These auxlary varables may be constant over tme (lke year of brth), or may vary over tme (lke ncome). We formulate the problem n terms of populaton means Then the queston s: can we construct weghts n such a way that the correct expected samplng totals at tme are reproduced.e. w xk = W n μ. (5) k 6

7 The soluton to ths problem s to construct the matrx of observed auxlary varables. Let c =1 f respondent partcpates n wave, and c =0 f ths s not the case. Then let X be the matrx X = [c 1,c 2,, c q, X 1, X 2,,X q ] (6) Where c s the vector of c and X the matrx of auxlary varables at tme, =1, 2,, q. For respondents whch are mssng n wave, the auxlary varables are equal to zero. As a consequence, the expected overall means over all cases are p μ k wth p =n /n (these are the means that have to be specfed n a fle wth populaton parameters). The weghts are determned such that equaton (6) s satsfed, wth the frst q populaton means (correspondng to the c ) equal to n. For every wave W ths ensures that W w = n (7) and hence that every weghted total corresponds to the samplng total n wave W. An applcaton of ths theory s estmaton of the total value of the caravans whch are owned by Dutch households. Statstcs Netherlands provdes data on the percentage of housholds ownng a caravan, whch was 14.0, 13.0 and 12.0 n 2001, 2002 and 2003, respectvely. In the DNB household survey, whch was carred out usng the CentERpanel, these percentages are consderably lower, see table 4. Ths has consequences for the average value of a caravan per household (a household whch does not own a caravan has value zero). The multple correlatons between the X-values and the values of the caravans was on avarage Table 4. Weghtng usng the percentages households ownng a caravan. Source: DNB household survey N unweghted N weghted % households wth caravans unweghted % households wth caravans weghted average value of caravans unweghted ( ) average value of caravans weghted ( ) total value of caravans weghted (M ) Equaton (1) shows that calbraton usng the regresson estmator (and many other procedures) nvolves nverson of the X X matrx. As a consequence, X may not contan lnear dependences or, more n partcular, dentcal columns. Ths may be the case when two waves have dentcal respondents. Then the c vectors are dentcal. The obvous soluton s to delete one of these vectors from the matrx of auxlary varables. In that case there may exst another problem, when one of the auxlary varables n the data set s dentcal (e.g. sex), whereas the populaton percentages may have changed (e.g. a slght change n the proporton of men n the populaton). There s no other soluton than just to pck one of the values, as a weghted sum can only have one outcome. 6. Dfferences between waves 7

8 It was only 15 years ago when the man rason d être of a panel was the possblty of measurng dfferences between waves n an effcent way by vrtue of the correlaton between observatons of the same varable n dfferent waves. To llustrate ths n a not too complcated way, we restrct ourselves to the case of two waves W 1 and W 2. Let n 1. be the number of respondents wth nonmssng values n W 1 \W 2, n 2. the number of respondents n W 2 \W 1, and n 12 the number of respondents n W 1 W 2. We are nterested n a populaton parameter D = Y 2 -Y 1, the growth between tme 1 and tme 2. For ths parameter we have two estmators: y nd, based on the respondents whch are n W 1 or W 2 but not n both, and y dep, based on the respondents n the ntersecton of W 1 and W 2, for whch twe correlated observatons are avalable. The standard errors of y nd and y dep are σ nd and σ dep, respectvely. We can combne y nd and y dep to a composte estmator y c dep ( α ) y nd = αy + 1 (8) such that the varance var ( y ) α σ + ( 1 α ) σ c = (9) dep s mnmal. Ths varance s mnmzed when nd 2 σ nd α = (10) σ + σ 2 nd 2 dep Assume that y has equal varance σ y 2 n waves 1 and 2 and let ρ y be the correlaton of y n waves 1 and 2, then we have n the unweghted case and σ = + nd σ y (11) n1. n2. σ ρ y = (12) n dep σ y 12 Whch yelds α n + n = (13) 2n1. n2. (1 ρ y ) n1. + n2. + n12 When the data are unweghted,.e. each case has weght 1, applyng the composte estmator amounts to weghtng the cases by γα n W 1 W 2 and γ(1-α) n W 1 \W 2 and W 2 \W 1. When the weghts are requred to add up to n, γ s the constant n/{n 12 α+(n 1. +n 2. )(1-α)}. 8

9 Now assume that the cases are already weghted as descrbed n secton 4. Let us further assume that w = n and w = n12. It s easly verfed that ths can be acheved by W 1 W 2 W W 1 2 ncludng a constant column of 1 s n X. When the cases n both waves are mssng at random the varances can be expected to be proportonal to (11) and (12), respectvely. For optmal weghtng of y, we may apply the same procedure, usng an addtonal weght factor γα and γ(1-α), respectvely, wth α gven by (13). The drawback of ths procedure, of course, s that for every varable y a new weght has to be calculated as t depends on ρ y. Ths may be worthwhle f y s a key varable wth a hgh correlaton between waves, e.g. the change n employment status between 2 months. If there s no such key varable, a value of ρ may be used that s typcal for the data set. In practce, α may often be close to 1. In the data set descrbed n table 4 we found n 1. =115, n 2. =114, n 12 =771, ρ y =0.56, whch yelded a value of α of Multple Imputaton A dfferent way to cope wth mssng data s mputaton: fllng the holes n the data set wth plausble values. Many mputaton methods exst, whch dffer by how they defne plausble. The mputaton methods can also be dstngushed as beng model based or parametrc, as opposed to data based or nonparametrc. Parametrc mputaton methods nclude replacng the mssng value by the varable s mean, or by the value predcted by a regresson analyss on the avalable complete varables for that case. A model based technque called EM-estmaton can also be used for parametrc mputaton. The EM method s a regresson approach, but t uses all the nformaton n the avalable data, both complete and ncomplete varables, whch makes t a very effectve method to mpute mssng data ponts. Nonparametrc or data based mputaton replaces the mssng values by an approprate value that s observed elsewhere n the data set. By usng a donor case to provde the mputed value, nonparametrc mputaton nsures that the mputed value s a value that can actually exst. Hot deck mputaton sorts the respondents and nonrespondents on a set of auxlary varables nto mputaton classes. Mssng values are then mputed, usng randomly chosen observed values from donors that are n the same mputaton class. To create mputaton classes, we need auxlary varables that are related to ether the mputed varable or to the mssngness mechansm. The mputaton methods descrbed above wll mpute a value that s optmal, accordng to some crteron. As a result, they wll underestmate the varance, and thus lead to based sgnfcance tests. A remedy s to add a random error to the mputed value. Ths random error can agan come from a model or from the observed data. For example, wth regresson mputaton t s possble to add a random error term, ether from the approprate error dstrbuton (normal or Student dstrbuton), or a randomly chosen resdual from the complete cases. The hot deck method mputes observed values, and thereby by mplcaton ncludes the error term that s present n that observaton. Snce addng error terms or usng hot deck ncorporates a chance mechansm, the results wll vary. Actually, when error s added to the mputed values, multple mputaton s the logcal next step. Multple mputaton means that the mssng data are mputed a number of tmes, typcally 3 to 5 tmes, wth a dfferent randomly chosen error term added n each mputaton. Ths leads to 3-5 separate data sets. These completed data sets are analyzed usng standard analyss methods, and the results are combned. The parameter estmates n the multply analysed data are combned by smply takng ther average: 9

10 Q ˆ 1 = m Q. (14) The varance of the analyss results across the multply mputed data sets provdes an estmate of the mputaton error (cf. Rubn, 1987). The approprate formula s 1 T = U + (1 + m ) B, (15) where U s the mean of the squared standard errors gven by U = 1 U, and B s the m varance of the parameter estmates between the mputed data sets gven by 1 ˆ 2 B = ( Q Q ) m 1. Whle combnng the multple estmates s relatvely straghtforward, constructng the multply mputed data sets s dffcult. Agan, there s a parametrc and a nonparametrc approach. In the parametrc approach, the Multply Imputed (MI) data sets are smulated draws from a Bayesan predctve dstrbuton of the mssng data. Ths requres a model for the complete data, and properly addng uncertanty about both the mssng values and the parameters of the predctve dstrbuton. Schafer (1996) descrbes such procedures for normally dstrbuted data and for other data models, ncludng categorcal and mxed normal-categorcal data, and panel data. In the nonparametrc approach hot deck mputaton s used. The approach s to use a bootstrap procedure to generate the random mputatons. Frst, a bootstrapped logstc regresson s used to predct the nonresponse. Next, the regresson equaton s used to compute the propensty to have mssng data for each case n the sample. The complete and ncomplete cases are then sorted nto subgroups based on these propensty scores. Fnally, mssng data are replaced by values taken from a randomly chosen donor n the same mputaton class. Ths procedure s repeated a number of tmes, to generate multple mputatons. The advantage of the nonparametrc approach s that t does not requre an assumpton about a data model; the model for the mssng data s provded by the observed data. It wll also always generate mputed values that actually exst. Multple mputaton may be somewhat laborous, but t s a powerful approach. The power of multple mputaton, especally n longtudnal research, s based on the noton that the mputaton model need not be the same as the analyss model. The most mportant mperatve for the mputaton model s that t should be at least as complex as the analyss model to be used. Thus, f t s antcpated that nonlnear curves wll be ftted to a longtudnal data set, nonlnear terms must also be used when the ncomplete data are mputed. When nteractons are tested, nteracton terms should be ncluded n the mputaton model. (In nonparametrc mputaton, where exstng values are mputed from a donor case, ths happens mplctly). But, there s no rule aganst the mputaton model beng more complex than the analyss model. Collns, Schafer and Kam (2001) nvestgated whether a lberal use of avalable auxlary varables mproves the results. They conclude that a lberal strategy s clearly superor to a restrcted strategy. Usng many auxlary varables not only decreases the chance of overlookng an mportant cause of mssngness, but also ncreases the accuracy of the mputed values. As we stated before, n longtudnal research there s a potental rchness of auxlary varables, both ntally measured and tme-varyng varables. In multple mputaton, the preferred strategy would be to use as many of these as possble. If t s necessary to select a smaller number of auxlary varables, Collns et al. recommend to look for auxlary varables that predct the ncomplete varables well, and/or auxlary varables that correlate wth the mssngness mechansm. For dagnostc procedures that help to dentfy such varables, we refer to De Leeuw, Hox and Husman (2003), 10

11 Concluson (Follows later. Dscuss/refer to software?) References Bethlehem J.G. and W.J. Keller (1987). Lnear Weghtng of Sample Survey Data. Journal of Offcal Statstcs 3, Collns, L.M., J.L. Schafer, & C.M. Kam (2001). A comparson of nclusve and restrctve strateges n modern mssng data procedures, Psychologcal Methods, 6, Devlle, J.C. and C.E. Särndal (1992). Calbraton Estmators n Survey Samplng. Journal of the Amercan Statstcal Assocaton 87, Hajek, J. (1981). Samplng from a fnte populaton. Marcel Dekker, New York. Kalton, G and I. Flores-Cervantes (2003). Weghtng Methods. Journal of Offcal Statstcs 19, Ksh, L. (1992). Weghtng for Unequal P. Journal of Offcal Statstcs 8, De Leeuw, E.D., Hox, J., and Husman, M. (2003). Preventon and treatment of tem nonresponse. Journal of Offcal Statstcs, 19, 2, Lttle, R.J.A and D.B. Rubn (2002). Statstcal Analyss wth Mssng Data (2 nd. ed.). New York, Wley. Potter, F. J., 1988, Survey of Procedures to Control Extreme Samplng Weghts. Proceedngs of the Amercan Statstcal Assocaton, Secton on Survey Research Methods, Potter, F. J., 1990, A Study of Procedures to Identfy and Trm Extreme Samplng Weghts. Proceedngs of the Amercan Statstcal Assocaton, Secton on Survey Research Methods, Potter, F. J., 1993, The Effect of Weght Trmmng on Nonlnear Survey Estmates. Proceedngs of the Amercan Statstcal Assocaton, Secton on Survey Research Methods, Rubn, D.B. (1987). Multple mputaton for nonresponse n surveys. New York: Wley. Schafer, J.L. (1996). Analyss of ncomplete multvarate data. New York: Chapman and Hall. 11