Finite Population Small Area Interval Estimation

Transcription

1 Journal of Offcal Statstcs, Vol. 23, No. 2, 2007, pp Fnte Populaton Small Area Interval Estmaton L-Chun Zhang 1 Small area nterval estmaton s consdered for a fnte populaton, where the small area parameters are treated as fxed constants. Desgn based drect estmaton yelds ntervals that are too long to be useful. Model based approaches are consdered. The desgn based areaspecfc coverages are uncontrollable. We propose to use populaton-specfc smultaneous coverage as the bass for evaluatng the small area confdence ntervals. Wage survey and census household data are used for llustraton. Key words: Desgn based coverage; random effects mxed model; bootstrap calbraton. 1. Introducton The man current approach to small area estmaton s based on predcton models. There s a consderable amount of theory on the estmaton (or predcton) of the small area parameters and the assocated mean squared error (MSE) wth respect to both the populaton and samplng models. We refer to Rao (2003) for a comprehensve overvew. For many survey practtoners and users t s appealng to have tradtonal desgn based measures of uncertanty, condtonal on the gven populaton and wth respect to the samplng alone. Rvest and Belmonte (2000) derved such a condtonal MSE of the composte estmators, whch nclude the emprcal best lnear unbased predctor (EBLUP) as a specal case. The problem s that the condtonal MSE estmator can be very unstable, especally when the shrnkage factor attached to the drect estmator s small. On the other hand, the model based uncondtonal MSE estmator has been found to track the condtonal MSE qute well n small smulaton studes (Rao 2003, Secton 7.1.6). In ths artcle we consder the related ssue of small area nterval estmaton. Desgn based drect estmaton s neffcent, and the confdence ntervals are too long to be useful n many cases. Snce short, area-specfc-ntervals are mpossble to construct, we propose to use desgn based, populaton-specfc, smultaneous coverage as the bass for evaluatng the performance of small area nterval estmators. It s shown that the smultaneous coverage of nterval estmators under the lnear mxed models asymptotcally acheves the nomnal level of confdence, under condtons smlar to those for the second-order uncondtonal MSE estmaton (Rao 2003). The methodology s set out n Secton 2. In Sectons 3 and 4 we llustrate the model-based approach usng, respectvely, the wage survey and census household data. A summary s gven n Secton 5. Statstcs Norway, Kongens gt. 6, P.B Dep., N-0033 Oslo, Norway. Emal: lcz@ssb.no Acknowledgments: I would lke to thank an assocate edtor and a referee for ther constructve comments that have greatly mproved a prevous verson of the artcle. q Statstcs Sweden

2 224 Journal of Offcal Statstcs 2. Small Area Interval Estmaton 2.1. Smultaneous Coverage Let ¼ 1; : : :, m denote the small areas (or domans). Let u denote the small area parameter of nterest. Let l ðaþ denote a confdence nterval for u constructed for 100a% nomnal level of confdence. Let I ¼ I ðaþ ¼1fu [ l ðaþ, and 0 otherwse. The desgn based, area-specfc coverage of l ðaþ s then d ðaþ ¼P p ði ¼ 1Þ ¼E p ði Þ ð1þ where P p. denotes probablty and E p denotes expectaton wth respect to samplng from the fnte populaton. We say that l ðaþ acheves the nomnal level of confdence f d ðaþ ¼a Regardng the set of l ðaþ s, all of them derved at the same nomnal level of confdence, we defne ther smultaneous coverage as! dðaþ ¼E p m 21Xm I ¼ m 21Xm d ðaþ ð2þ ¼1 ¼1 The smultaneous coverage s the expected proporton of small area parameters covered by the set of confdence ntervals on repeated samplng from the populaton. It s a meanngful desgn based measure n the context of small area estmaton, summarzng all the areaspecfc coverages n a sngle number. Snce correct area-specfc coverages for all the areas mply correct smultaneous coverage, but not the other way around, the smultaneous coverage s a weaker property of the nterval estmators. Whle such populaton-specfc, area averagng performance measures are often used n emprcal studes (e.g., Heady and Ralphs 2005), few systematc studes have been reported on how the model based methods should behave wth respect to such desgn based measures. Fnally, we notce that the smultaneous coverage s not the jont coverage of the l ðaþ s,.e., P p ð> m ¼1 I ¼ 1Þ. Nor has t anythng to do wth the so-called smultaneous ntervals n multple comparson problems Desgn Based Estmaton Denote by ^u a desgn based drect estmator of u. We assume that t s desgn unbased wth varance c. That s, let e be the samplng error of ^u, and then we have ^u ¼ u þ e where E p ðe Þ¼0 and V p ðe Þ¼c ð3þ The standard desgn based approach s then to assume normalty of e, whch yelds the 100ð2a 2 1Þ% nomnal confdence nterval p ffffffff pffffffff ^u 2 z a c ; ^u þ z a c where z a s the a-quantle of N(0,1), and a [ ð0:5; 1Þ. Gven the normalty assumpton, desgn based ntervals acheve area-specfc as well as smultaneous coverage.

3 Zhang: Fnte Populaton Small Area Interval Estmaton 225 There are two dffcultes wth ths approach. In the frst place, normalty of e, or even zero expectaton of e, may not be vald f the wthn-area sample sze s small and/or u s nonlnear. Secondly, n practce c s seldom known. Replacng c wth the desgn based drect estmator ^c s neffcent. Often n practce one chooses to use some smoothed, stable estmator of c wth small bas nstead. Whle both the nonnormalty of ^u and the potental desgn bas of ^c may cause problems for the area-specfc coverage, the smultaneous coverage often remans qute robust Estmaton Based on Best Predctor Let us start wth a specal case. Consder the followng smple random effects model ^u ¼ x T b þ y þ e (Fay and Herrot 1979), where x contans the area-level covarates, y s the ndependent random effect wth zero mean and varance s 2 y, and ^u s the drect desgn unbased estmator and e s ts samplng error as defned n (3). Suppose that the parameters ðb; s 2 y Þ are known, and so are the c s. The best predctor (BP) s then gven by ~u ¼ x T b þ g ^u 2 x T b where g ¼ s 2 y = s 2 y þ c The MSE of the BP s smply the so-called g 1 -term (Prasad and Rao 1990),.e., g 1 ¼ def E{ð ~u 2 u Þ 2 } ¼ g c ¼ð1 2 g Þs 2 y where E denotes expectaton wth respect to both the populaton model and the desgn generated samplng dstrbuton. Gven the normalty of ~u 2 u, a 100ð2a 2 1Þ% nomnal level predcton nterval for u s gven by p ffffffffff pffffffffff l ¼ ~u 2 z a ; ~u þ z a g 1 g 1 The nomnal level refers to accuracy of predcton because u s a random varable here. Under the assumed random effects model, l then acheves the uncondtonal coverage D ðaþ ¼PðI ¼ 1Þ ¼a where P s the probablty under both the populaton model and the samplng dstrbuton. How does ths BP based l perform from the desgn based pont of vew? Take frst the stuaton where c =s 2 y < 0 and s 2 y ¼ Oð1Þ. We have g < 1 g 1 < c ~u < ^u and l < ^u p ffffffff pffffffff 2 z a ; ^u þ z a Snce normalty s probably a reasonable assumpton n ths case, l ðaþ would approxmately acheve the desgn based area-specfc coverage,.e., d ðaþ < a. Next, suppose s 2 y =c < 0. We have g < 0 g l < s 2 y ~u < x T b and l < x T b 2 z as y ; x T b þ z as y Snce y s a constant gven the fnte populaton, we bascally have d ¼ 1fjy j # z a s y, and 0 otherwse. That s, the area-specfc coverage degenerates. In summary, the model based nterval l approxmately attans the desgn based area-specfc coverage as c =s 2 y c c ð4þ

4 226 Journal of Offcal Statstcs tends to 0 or, equvalently, g! 1. As g! 0, the desgn based area-specfc coverage cannot be mantaned n general, and tends towards the degenerate case. Consder now the desgn based smultaneous coverage of the l s. Let E 1, V 1 and Coy 1 denote, respectvely, expectaton, varance and covarance wth respect to the populaton model. Gven (4) under the assumed model, for all ¼ 1, :::, m, we have EðI Þ¼E 1 {E p ði Þ} ¼ E 1 {d ðaþ} ¼ a V 1 {d ðaþ} ¼ VðI Þ 2 E 1 {V p ði Þ} # VðI Þ¼að1 2 aþ Moreover, gven the populaton, I and I j depend on e and e j, for j, respectvely. Gven ndependent samplng wthn the small areas, we have Thus, d d j ¼ E p ði ÞE p ði j Þ¼E p ði I j Þ Coy 1 ðd ; d j Þ¼E 1 {E p ði 1 I j Þ} 2 E 1 {E p ði Þ}E 1 {E p ði j Þ} ¼ CoyðI ; I j Þ¼0 because I derves from ~u 2 u whch s a functon of (e ; y ) and therefore ndependent of I j from ~u j 2 u j. It now follows from the Law of Large Numbers that, as m!1, ðaþ dðaþ ¼m 21Xm ¼1 d ðaþ! P a.e., there s convergence n probablty wth respect to the populaton model. In other words, gven a large number of small areas, we may expect the desgn based smultaneous coverage of the model based ntervals to be close to the nomnal level of confdence, gven () correct uncondtonal area specfc coverage,.e., D ðaþ ¼a for ¼ 1, :::, m, and () ndependent random effects and ndependent samplng wthn the small areas Estmaton Based on EBLUP The model consdered above s a member of the class of lnear mxed models (LMMs). We refer to Rao (2003) for an account of the varous LMMs that have been used n small area estmaton. It s clear that the result (A) for the BP based ntervals remans vald, provded the random effects are uncorrelated across the areas under the LMM. In practce, however, the parameters are unknown, and the EBLUP s used nstead of the BP. Denote by ^u H the EBLUP of u. Denote by ^g the estmator of MSE ð ^u H Þ. The 100ð2a 2 1Þ% nomnal level predcton nterval for u s then l ¼ ^u H p ffffffff 2 z a ^g ; ^u H pffffffff þ z a ^g ð5þ wth respect to both the populaton and samplng models. The latter may be purely model based, such as n the case of unt-level mxed models. It can also be the desgn generated samplng dstrbuton, or a model of the actual samplng process. The desgn based area-specfc coverage of l s uncontrollable as before. When t comes to the desgn based smultaneous coverage, the condton (4) would stll mply E l {d ðaþ} ¼ a and V 1 {d ðaþ} # aðl 2 aþ as above. However, snce both d and d j depend on the parameter estmators, they are not uncorrelated as n the case of BP, even f the

5 Zhang: Fnte Populaton Small Area Interval Estmaton 227 samplng s ndependent wthn the small areas. Nevertheless, gven the EBLUP asymptotcally converges to the BP n dstrbuton, the EBLUP based ntervals wll converge towards the BP based ntervals, and we may expect the smultaneous coverage of the ntervals (5) to be close to the nomnal level of confdence. For ths we need consstent parameter estmators, wth respect to both the populaton model and the samplng dstrbuton. In the usual context where the EBLUP s appled, ths s for example the case gven the asymptotc settngs for the second-order MSE estmaton (Rao 2003). In partcular, the wthn-area sample szes can reman bounded. Snce consstent parameter estmaton s also necessary for the uncondtonal coverage (4), what we requre s that the condtons () and () above hold asymptotcally, as m! Normalty Assumptons Because we are usng a model based approach, the ntended desgn based coverage may fal n cases of model msspecfcatons. Good uncondtonal coverage of the nterval (5) depends on the normalty of ^u H 2 u and the accurate MSE estmator. Thus, f the asymptotc second-order MSE estmators (Prasad and Rao 1990) are beng used, then the regularty condtons for the MSE estmaton are also needed for the nterval estmaton. Notce that also the MSE estmaton assumes normalty of the random effects ntroduced by the model, n whch case the normalty of ^u H 2 u follows from the normalty of ^u H n addton. We do not know n general how robust the model based ntervals are under nonnormal u and ^u H although robustness of the MSE estmaton towards nonnormal area-level random effects has been consdered by Lahr and Rao (1995). Takng the smple Fay-Herrot arealevel model above, we have ^u H ¼ x T ^b þ y^ ¼ x T ^b þ ^g y þ e 2 x T ð ^b 2 bþ Of the random varables on the rght-hand sde, normalty of ^b s usually a plausble assumpton, whereas normalty of ^g s probably not entrely accurate even f ^s 2 y s normal. The normalty of y s a model assumpton, whereas the normalty of e s probably not true where the area sample sze s small. In short, normalty of ^u H 2 u s unlkely to hold n all the areas even f y s normal. However, gven normal y, nonnormalty of e may not be crucal to the smultaneous coverage. The reason s that the smultaneous coverage of the drect desgned based ntervals are often qute robust under nonnormal e, whch wll be llustrated n the smulaton studes later. Emprcally, dagnostcs can be used to check whether there are severe departures from the normalty assumptons of the LMM. We refer to Rao (2003) for examples from practces Bootstrap Calbraton As wth any confdence procedure, varous departures from the underlyng assumptons may cause the true coverage to devate from the nomnal level of confdence. A technque for adjustment s calbraton. That s, f the 100a% nomnal confdence ntervals do not have 100a% coverage, ntervals wth nomnal level 100f% may do. Bootstrap calbraton can be used to explore the mappng between f and a. The dea s straghtforward: gven

6 228 Journal of Offcal Statstcs the true populaton, we can repeatedly draw samples and derve confdence ntervals at chosen nomnal levels n order to fnd out ther true coverages. In practce, however, we do not know the true populaton so that the bootstrap calbraton must be conducted on the bass of some plug-n populaton. The rest s exactly the same. Booth, Butler, and Hall (1994) proposed a nonparametrc method for constructng the plug-n populaton based on stratfed smple random samples. Let N h and n h be, respectvely, the populaton and sample szes n stratum h, for h ¼ 1, :::, H. Let m h be the nteger part of N h /n h and let k h ¼ N h 2 m h n h. The plug-n populaton n stratum h s gven by m h replcates of the wthn-stratum sample, plus a sample of sze k h selected randomly and wthout replacement from t. The dfferent stratum populatons are formed ndependently of each other. Thus, resamplng from each stratum of the plug-n populaton does not dffer much from drect nonparametrc bootstrap resamplng (.e., randomly and wth replacement) from the selected wthn-stratum sample, provded the samplng fracton n h /N h s neglgble. The method s explored n Secton 4, where t does mprove the coverage. However, the extent to whch such mprovements wll hold n general remans an open queston at ths stage. Ths s because the bootstrap method s asymptotcally justfed when the populaton s consdered as a sample from a super-populaton, where all the N h s and n h s dverge to nfnty n such a way that each rato N h /N g and n h /n g converges to a fnte nonzero lmt. Whether ths holds for the gven fnte populaton s a queston dffcult to answer n general. For nstance, f the desgn strata concde wth the small areas of nterest, then many of the area sample szes are probably not large enough to justfy the asymptotc settng of n h!1. If possble, the performance of the bootstrap calbraton over repeated samplng needs to be evaluated by smulatons before t s endorsed Generalzed Lnear Mxed Models For categorcal data, such as bnary or count data, generalzed lnear mxed models (GLMM) are more approprate (Rao 2003, Secton 5.6). Let m be the mean of a response vector y. Let h() be a monotonc lnk functon, such that hðmþ ¼h ¼ Xb þ Zy where X and Z are the desgn matrces, b s the parameter vector contanng the fxed effects, and y s the vector of random effects. In the small area estmaton context, h ¼ðh 1 ;:::;h m Þ T s often a vector of m components correspondng to u 1 ;:::;u m.its convenent to consder predcton of h on the lnear scale. An nterval for h can be turned nto an nterval for u through the nverse transformaton h 21 ðhþ. Let ^h H be the predctor of h, and let ^g denote ts MSE estmator. A 100ð2a 2 1Þ% nomnal level predcton nterval s gven as (5) wth ^h H replacng ^u H. Asymptotc smultaneous coverage depends on the normalty of ^h 2 h as well as accurate MSE estmaton. Typcally, we need the normalty assumpton of y, whch partly depends on the choce of the lnk functon. Otherwse, the approach s smlar to that under the LMM. There s a harder computatonal ssue under the GLMM. Ths s an area wth rapd on-gong research developments. See McCulloch and Searle (2001) for an account of the feld. These authors favor the maxmum lkelhood estmaton wherever possble.

7 Zhang: Fnte Populaton Small Area Interval Estmaton 229 In practce, however, a group of the so-called penalzed quas-lkelhood (PQL) algorthms are often used because of the easness of mplementaton (Schall 1991; Breslow and Clayton 1993; McGlchrst 1994). We wll not go nto the computatonal detals here Interval Based on Synthetc Estmator Regresson-synthetc types of estmator are sometmes used n small area estmaton (Rao 2003, Secton 4.2). Even when the pont estmators are acceptable, the evaluaton of the uncertanty n the estmaton wll be msleadng f t s done wth respect to the underlyng model, because a fxed effects model s unlkely to be able to fully capture the between area varaton of the small area parameters of nterest. For example, assume the followng lnear regresson model y j ¼ x T j b þ 1 j where x j contans the covarates of the jth unt from the th area, and 1 j s ndependent of each other wth varance s1 2. The regresson synthetc estmator of u ¼ S N j¼1 y j s gven by ^u S ¼ X T ^b, where X ¼ S N j¼1 x j and N s the populaton sze, gven neglgble wthn area samplng fracton. Denote by t the approxmate desgn based samplng varance of ^u S. The 100ð2a 2 1Þ% nomnal level nterval ^u S p ffffff 2 z a t ; ^u S pffffff þ z a t would n general have very msleadng desgn based coverage, because t fals to recognze the desgn based bas n ^u S that s always present n practce. It s possble to ntroduce a second model to mprove the nterval estmaton. Put ^u S ¼ u þ j þ 1 where j ¼ E p ^u S 2 u and 1 ¼ ^u S 2 E p ^u S Assume that j, Nðj; s 2 Þ, where j s the desgn based bas of the synthetc estmator. Notce that ths s not the frst lnear regresson model under whch ^u S has been derved. Once ntroduced, t mples the followng 100ð2a 2 1Þ% nomnal level nterval l ¼ ^u S pffffffffffffffffffffffffffffff 2 j 2 z a s 2 þ t ; ^u S pffffffffffffffffffffffffffffff 2 j þ z a s 2 þ t assumng normalty of 1, and ndependence between j and 1. In partcular, ths ncludes an extra varance component s 2 not to be found n the nave confdence nterval, whch can be consdered as a unform adjustment due to the desgn based bas. An estmator of l s obtaned from replacng ðj; s 2 ; t Þ by ther estmates. For ths purpose, observe that ^u S 2 ^u ¼ j þ 1 2 e < j 2 e where ^u s the desgn based drect estmator and e s ts samplng error. Ths s an LMM wth one model based random effect j and two desgn generated samplng errors. The approxmate two-component model follows from the fact that ^b s a global parameter estmator dependng on the whole sample such that asymptotcally 1 wll have much smaller varance than e. It may be noted that the length of the adjusted nterval above vares much less across the areas than the drect desgn based ntervals. Frstly, the term s 2 s the same everywhere, and s comparable to t n many stuatons. Secondly, t s the samplng varance of the synthetc estmator ^u S, and thus does not drectly depend on the sample sze n the -th

8 230 Journal of Offcal Statstcs area. As a result, a smaller area does not necessarly have a longer nterval than a larger area. Thus, the effcency gans over the drect ntervals are typcally largest for the areas wth smallest sample szes. The two-model approach here s conceptually less appealng than the EBLUP based approach earler, where the same model s used for pont as well as nterval estmaton. Because the resultng nterval does not have an area-specfc bas correcton, t s on the whole less effcent than the mxed modelng approach. Nevertheless, t provdes a more realstc measure of uncertanty for the regresson synthetc estmator than that under the orgnal fxed effects model. 3. Smulaton Based on Wage Survey Data The Norwegan wage survey s based on a yearly sample of clusters of wage earners. The clusters are establshments, whch are stratfed accordng to the sze of the establshments. From each stratum except the largest one (where a census s carred out), a random sample of establshments are selected frst, and all the employees from the selected establshment are then ncluded n the sample. The prmary varable of nterest s the monthly wage of full-tme employees n varous subgroups of the populaton. For our smulaton study we extracted the data from occupaton group 5 (sales and servce) n ndustry group 52 (retalng) n 2001 and The panel from the three countes n North Norway contans 1,269 persons. We take the 66 muncpaltes n these three countes as the small areas, and estmate the average monthly wage n each muncpalty n 2002, usng the monthly wage n 2001 as the auxlary varable. Let ¼ 1; :::;m (and m ¼ 66) denote the small areas. Let y j be the monthly wage of the jth full-tme employee from area n 2002, and let x j be the monthly wage of the same person n Let y and x be the area sample means n the retreved panel. These are fxed to be the area means of the synthetc populaton for ths smulaton study, denoted by u * ¼ Y * and X *. To generate a sample from ths fxed populaton, we assume neglgble samplng fractons n all the areas, and draw randomly and wth replacement pars of resduals from {ðx j 2 x ; y j 2 y Þ; ¼ 1; :::;m and j ¼ 1; :::;n }, where n s the area sample sze n the panel. Addng ths to ð X * ; Y * Þ, we obtan a par of smulated observatons from the correspondng area, denoted by ðx * j ; y* jþ. Notce that, n ths way, the wthn-area covarance between the smulated par of varables s the same across all the areas. We have thus a stratfed random samplng desgn wth the areas as the strata. The total sample sze s 1,269, the area sample sze vares from 3 to 215, the medan area sample sze s 8 and the mean s On the bass of each bootstrap sample, we derve the drect desgn based nterval, as well as the model based nterval (5), under the followng nested-error regresson model y * j ¼ b 0 þ x * j b 1 þ y þ e j (Battese, Harter, and Fuller 1988), where y and e j are ndependent normal errors. The use of actual wage data mples that the normalty of the random errors remans a model assumpton. We record the proporton of u * s that are covered by each set of confdence ntervals. The average of ths proporton over ndependent repeated bootstrap smulatons yelds then a Monte Carlo approxmaton to the smultaneous coverage of the correspondng nomnal level of confdence.

9 Zhang: Fnte Populaton Small Area Interval Estmaton 231 In Table 1 the confdence ntervals are compared to each other wth respect to the smultaneous condtonal coverage and the average relatve length (ARL), gven by X p ffffffff m 21 2za ^g =u for the 100ð2a 2 1Þ% model based ntervals. The smultaneous coverage of the drect desgn based ntervals approxmately acheves the nomnal level of confdence. The EBLUP based ntervals (5) have smultaneous coverage about 2 4 percent below the nomnal levels that we have looked at. Dagnostcs for normalty of y suggest that the dstrbuton of y has heaver tals than the normal dstrbuton on both ends, whch seems to be the reason that the under-coverage ncreases slghtly wth the nomnal level of confdence. However, consderable gans n effcency have been acheved: the model based nterval on average reduces the length of the drect confdence nterval by more than 50%. In Fgure 1 the desgn based area specfc coverages of the drect ntervals are compared to those of the model based ntervals. The nomnal level of confdence s 95% n ths case. It can be seen that the coverage of the model based ntervals can be qute erratc as the area sample sze decreases, for reasons explaned earler. The coverages of the desgn based ntervals are closer to the nomnal level of confdence. In partcular, the average coverage across the small areas appears robust even when the sample sze s as low as three. 4. Smulaton Based on Census Household Data 4.1. Smulaton Desgn Small area household compostons are area populaton counts of households wth respect to some classfcaton of nterest (e.g., the sze of the household). Complete enumeraton of the populaton s only avalable n the census. Post-censal updates need to be based on household surveys conducted at the natonal statstcal offces. For our smulaton study we retreved the Norwegan census household compostons n 1990 and The 89 economc regons (analogous to the NUTS4 regonal classfcaton of EUROSTAT) are fxed as the domans of nterest,.e., m ¼ 89. We set the area proporton of sngle-person Table 1. Desgn based smultaneous coverage and average relatve length of confdence ntervals based on 250 smulated wage survey samples: A, Drect estmaton; B, EBLUP under nested-error regresson model. Monte Carlo standard error n parentheses Smultaneous coverage A (0.003) (0.002) (0.002) (0.001) (0.001) B (0.003) (0.002) (0.002) (0.002) (0.001) Average relatve length A (0.000) (0.001) (0.001) (0.001) (0.001) B (0.000) (0.000) (0.001) (0.001) (0.001)

10 232 Journal of Offcal Statstcs Fg. 1. Desgn based area-specfc coverage of 95% nomnal level confdence ntervals (marked by the dotted horzontal lne). Drect estmaton (o) and EBLUP based estmaton (x) households n 2001 as the nterest of estmaton, usng the correspondng area proporton n 1990 as the auxlary varable. Let u be the area proporton of nterest n Let p be the correspondng proporton n Let h ¼ logðu Þ 2 logð1 2 u Þ and let x ¼ logð p Þ 2 logð1 2 p Þ. Smple regresson of h on x yelds approxmate normal resduals wth apprecable varance. We shall therefore adopt the followng GLMM for ths populaton logtðu Þ¼h ¼ b 0 þ x b 1 þ y and y, d Nð0; s 2 y Þ For samplng from the populaton we use stratfed smple random samplng wth the 89 areas as the desgn strata. The samplng desgn vares wth respect to the samplng fracton, denoted by f. Smulatons are carred out at f ¼ 1=50, 1/150 and 1/500. On the bass of each smulated sample, we derve drect desgn based ntervals as well as the GLMM based ntervals, usng the PQL algorthm outlned by McGlchrst (1994) to estmate the parameters (b 0, b 1, s 2 y ) and the random effects y. In partcular, s 2 y s estmated by the REML procedure. The MSE of ^h H 2 h s estmated on the bass of a normal approxmaton to the quas log-lkelhood for (b 0, b 1, y 1, :::, y m ) Coverages Denote by l ðkþ the nterval calculated from the kth smulated sample, for ¼ 1; :::;m. Let I ðkþ ¼ 1fu s covered by l ðkþ and I ðkþ ¼ 0 f not. The Monte Carlo approxmaton of the true smultaneous coverage based on B smulated samples s then gven by! ^d ¼ m 21Xm ^d ¼ m 21Xm B 21XB ¼ B 21XB d ðkþ where ¼1 d ðkþ ¼ m 21Xm ¼1 I ðkþ ¼1 k¼1 I ðkþ k¼1

11 Zhang: Fnte Populaton Small Area Interval Estmaton 233 The d (k) s ndeed the proporton of the u s covered by the set of ntervals based on the kth sample. It s the realzed smultaneous coverage. The smultaneous coverage d s ts expectaton on repeated samplng. But one would also be nterested n the whole dstrbuton of d (k). In partcular, an estmate of the medan of ths dstrbuton s gven by the medan of d ð1þ ; :::;d ðbþ, whereas the Monte p ffffff Carlo standard error of ^d s an estmate of the standard devaton of d (k) dvded by B. In Table 2 the desgn based smultaneous coverage and the ARL of the confdence ntervals are gven. Also shown are the medan values of d (k). It can be seen that the coverages of the desgn based drect ntervals are on the whole qute good. Negatve bas arses as the samplng fracton decreases, but s not serous even at f ¼ 1=500. The man problem s neffcency. The drect ntervals are smply too long to be useful even at f ¼ 1=50. The GLMM based ntervals are much shorter. The relatve effcency compared to the drect estmaton ncreases as the samplng fracton decreases. The coverages of the model based ntervals, however, are negatvely based. The problem has lttle practcal consequences at f ¼ 1=50, but ncreases as the samplng fracton decreases. The loss of coverage occurs manly between f ¼ 1=50 and f ¼ 1=150. For example, the smultaneous coverage of the 95% nomnal ntervals drops from 94.1% at f ¼ 1=50 to 85.2% at f ¼ 1=150, whereas the coverages are about the same at f ¼ 1=150 and f ¼ 1=500,.e., wthn the margns of Monte Carlo error. The medan values of the realzed smultaneous coverages d ðkþ are very robust as the samplng fracton decreases, and are n close agreement wth the correspondng nomnal confdence levels. The dscrepancy between the mean and medan values of d (k), however, suggests skewed dstrbutons of the realzed smultaneous coverage over repeated samplng at f ¼ 1=150 and 1/500. The Monte Carlo standard errors ncrease qute a lot from f ¼ 1=50 to f ¼ 1=150 and 1/500. Together they ndcate that the realzed coverage may be partcularly low on certan occasons Calbraton We explore the bootstrap calbraton to see f t can mprove the smultaneous coverage of the GLMM based ntervals. For each smulated sample from the populaton, we carry out bootstrap calbraton by stratfed resamplng usng the method of Booth, Butler, and Hall (1994). Let B be the number of smulated samples from the true populaton, and let K be the number of resamples for each bootstrap calbraton. The total number of smulated samples s then B K. Table 3 shows the smulaton results at f ¼ 1=150 wth B ¼ 500 and K ¼ 40. The Monte Carlo estmates of smultaneous coverages are consstent wth the correspondng estmates n Table 2 wthn the margns of Monte Carlo error. Each par of a nomnal level of confdence and the correspondng calbrated smultaneous coverage s an estmate of the mappng between f and a by the bootstrap calbraton. Thus, accordng to the results n Table 3, the true coverage s at the nomnal level 0.95, and t s at the nomnal level 0.99, and so on. In partcular, a nomnal level of should yeld a smultaneous coverage of 0.949, or approxmately 95%. On account of the Monte Carlo error and the fact that the calbrated coverage can only be checked at chosen grd values,

12 Table 2. Desgn based smultaneous coverage and average relatve length of confdence ntervals based on smulatons of household data: A, Drect estmaton; B, GLMM based estmaton. Medan value of the realzed smultaneous coverages (n talcs). Monte Carlo standard error n parentheses Samplng fracton f ¼ 1=50, number of smulatons B ¼ 500 Smultaneous coverage 234 A (0.002) (0.001) (0.001) (0.001) (0.000) B (0.005) (0.004) (0.003) (0.003) (0.002) Average relatve length A (0.000) (0.000) (0.000) (0.000) (0.000) B (0.001) (0.001) (0.001) (0.001) (0.002) Samplng fracton f ¼ 1=150, number of smulatons B ¼ 500 Smultaneous coverage A (0.002) (0.002) (0.001) (0.001) (0.001) B (0.010) (0.010) (0.009) (0.008) (0.007) Average relatve length A (0.000) (0.000) (0.000) (0.000) (0.000) B (0.002) (0.002) (0.003) (0.003) (0.004) Journal of Offcal Statstcs

13 Samplng fracton f ¼ 1=500, number of smulatons B ¼ 500 Smultaneous coverage A (0.002) (0.002) (0.001) (0.001) (0.001) B (0.010) (0.009) (0.008) (0.007) (0.006) Average relatve length A (0.001) (0.001) (0.001) (0.001) (0.001) B (0.003) (0.004) (0.005) (0.005) (0.006) Zhang: Fnte Populaton Small Area Interval Estmaton 235

14 236 Journal of Offcal Statstcs Table 3. Bootstrap calbraton wth smulatons, f ¼ 1=150 Nomnal confdence level Smultaneous coverage Calbrated smultaneous coverage Bas of bootstrap calbraton the calbrated coverage cannot always be brought to the exact nomnal level of confdence, even after repeated trals. For example, n ths case we got 94.9% nstead of exactly 95%. It now follows that the dfference between the smultaneous coverage and the calbrated smultaneous coverage s an estmate of the coverage bas of the calbrated ntervals over repeated samplng. For nstance, the coverage bas s for calbrated ntervals amng at the true coverage 0.95, whch s a great mprovement compared to the bas of the uncalbrated ntervals. 5. Summary The desgn based propertes of the model based small area nterval estmaton have been consdered. The area-specfc coverages are uncontrollable n general. We propose to use the smultaneous coverage to evaluate the small area confdence ntervals. Beng the mean of the area-specfc coverages, the smultaneous coverage s the expected proporton of areas covered by the set of confdence ntervals, all of whch are derved at the same nomnal level of confdence. It s a meanngful concept n small area estmaton, summarzng all the area specfc coverages n a sngle number. It has been shown that the coverage of the EBLUP based ntervals acheves the nomnal level of confdence asymptotcally, as the number of small areas tends to nfnty, gven correct uncondtonal area specfc coverage wth respect to both the populaton model and the samplng dstrbuton, and asymptotcally ndependent samplng wthn the small areas. The asymptotc settngs are smlar to those requred for the second-order MSE estmaton under the LMMs. The adapton to the GLMMs s straghtforward. Bootstrap calbraton may mprove the coverage over repeated samplng n stuatons where the normal approxmaton s poor. We have also dscussed nterval estmaton based on regresson synthetc estmators. Whle ths s less effcent than estmaton based on the random effects models, t provdes more realstc measures of uncertanty than the nave ntervals derved under the fxed effects model underlyng the synthetc estmator. 6. References Battese, G.A., Harter, R.M., and Fuller, W.A. (1988). An Error-components Model for Predcton of County Crop-Areas Usng Survey and Satellte Data. Journal of the Amercan Statstcal Assocaton, 83, Booth, J.G., Butler, R.W., and Hall, P. (1994). Bootstrap Methods for Fnte Populatons. Journal of the Amercan Statstcal Assocaton, 89, Breslow, N.E. and Clayton, D.G. (1993). Approxmate Inference n Generalzed Lnear Mxed Models. Journal of the Amercan Statstcal Assocaton, 88, 9 25.

15 Zhang: Fnte Populaton Small Area Interval Estmaton 237 Fay, R.E. and Herrot, R.A. (1979). Estmates of Income for Small Places: An Applcaton of James-Sten Procedures to Census Data. Journal of the Amercan Statstcal Assocaton, 74, Heady, P. and Ralphs, M. (2005). EURAREA: An Overvew of the Project and Its Fndngs. Statstcs n Transton, 7, Lahr, P. and Rao, J.N.K. (1995). Robust Estmaton of Mean Squared Error of Small Area Estmators. Journal of the Amercan Statstcal Assocaton, 82, McCulloch, C.E. and Searle, S.R. (2001). Generalzed, Lnear, and Mxed Models. John Wley and Sons. McGlchrst, C.A. (1994). Estmaton n Generalzed Mxed Models. Journal of the Royal Statstcal Socety, Seres B, 56, Prasad, N.C. and Rao, J.N.K. (1990). The Estmaton of Mean Square Errors of Small Area Estmators. Journal of the Amercan Statstcal Assocaton, 85, Rao, J.N.K. (2003). Small Area Estmaton. New York: Wley. Rvest, L.-P. and Belmonte, E. (2000). A Condtonal Mean Squared Error of Small Area Estmators. Survey Methodology, 26, Schall, R. (1991). Estmaton n Generalzed Lnear Models wth Random Effects. Bometrka, 78, Receved June 2004 Revsed January 2007