Are We Really Doing What We think We are doing? A Note on Finite-Sample Estimates of Two-Way Cluster-Robust Standard Errors

Transcription

1 Are We Really Doing What We think We are doing? A Note on Finite-Sample Estimates of Two-Way Cluster-Robust Standard Errors Mark (Shuai) Ma Kogod School of Business American University Shuaim@american.edu Comments and Suggestions are welcome! Summary Archival researchers heavily rely on statistics software to deliver large sample analyses. To provide valid empirical analyses, an author needs to know the best statistical solution to the research issue and also the correct computer language that exactly carries out such statistical tests. However, unfortunately, researchers rarely explain what they exactly do in empirical analyses. Consequently, it is difficult for a reader to figure out the validity of the empirical results. This note uses two-way cluster-robust standard errors as an example to explain these points. Two-way cluster-robust standard errors are getting widely used in the accounting and finance literature. There are multiple different alternative specifications of two-way cluster-robust standard errors, which could result in very different significance levels than unadjusted asymptotic estimates. However, researchers rarely explain which estimate of two-way clusterrobust standard errors they use, though they may all call their standard errors two-way clusterrobust standard errors. Specifically, I first provide a short-discussion on alternative estimates of two-way cluster-robust standard errors. Second, I discuss two common mistakes in calculating two-way cluster-robust standard errors. Third, I show that popular statistics software (SAS and STATA) have options that could generate several alternative estimates of two-way cluster-robust standard errors.. Therefore, if not explained, no reader would know which estimate is used. Finally, I suggest that future empirical research should carefully explain how it implements estimates of two-way cluster-robust standard errors in finite samples. In addition, a SAS macro code for two-way clustered standard errors is available at my website. If you use this code, please add the following footnote "To obtain unbiased estimates, the clustered standard errors are adjusted by (N-1)/ (N-P) G/(G-1), where N is the sample size, P is the number of independent variables, and G is the number of clusters." Key Words: statistical software, two-way cluster-robust standard errors, finite sample. * I emphasize that this note is written purely and solely for communication purposes. I would thank Sutirtha Bagchi, Wayne Thomas, Louis Ederington, Xin Huang, Wenbin Cao, Lisa Yang and Ted Moorman for answering my questions regarding two-way clustered SEs, but errors are my own. If you find any error in this note, please let me know immediately.

2 1.Asymptotic estimate of two-way cluster-robust standard errors Here, I provide a brief summary of studies on two-way cluster-robust standard errors. For details, please see Petersen 2009; Gow et al. 2010; Thompson 2011; and Cameron and Miller According to the literature (page 458, Petersen 2009; page 2, Thompson 2011), the estimate for the VARIANCE-COVARIANCE (VAR-COV) matrix for two-way cluster-robust standard errors (SEs) is expressed as V firm&time= V firm+ V time V white (1) Where, V firm&time is the VAR-COV matrix clustered by two-way (i.e., firm and time). V firm is the VAR-COV matrix clustered by one dimension (i.e., firm). V time is the VAR-COV matrix clustered by another dimension (i.e., time). V white is the white (1980) heteroskedasticity-robust VAR-COV matrix. V white is subtracted off, because it is included in both V firm and V time. So, V white would solve the double counting problem. Importantly, if there are multiple observations in each intersection of firm-time, then the V white should be replaced by V firm-time intersection, as suggested in footnote 19 on page 458 on Petersen (2009). For example, when an author wants to cluster by firm-year in a sample of firm-month observations, then there might be 12 observations each firm-year. Then, the equation for two-way clustered SEs would be expressed as V firm&time= V firm+ V time V firm-time (2) To estimate two-way clustered SEs, there are three steps we can follow: Step 1. Estimate the firm-clustered VARIANCE-COVARIANCE matrix V firm, Step 2. Estimate the time-clustered VARIANCE-COVARIANCE matrix V time, Step 3. Estimate the heteroskedasticity robust white VARIANCE-COVARIANCE matrix (V white) when there is only one observation in each firm-time intersection, or, estimate the firm-time intersection clustered VARIANCE-COVARIANCE matrix (V firm-time) when there is more than one observation in each firm-time intersection. 1

3 2. Finite Sample Estimates for Clustered Standard Errors Prior studies carefully discuss two-way clustered SEs, which is becoming widely used in the accounting and finance literature. In finite samples, there are several alternative estimates for two-way cluster-robust standard errors. These finite-sample estimates could result in different significance levels than unadjusted asymptotic estimates. However, researchers rarely explain whether they use finite-sample adjusted estimates or unadjusted asymptotic estimates of two-way cluster-robust standard errors. To my knowledge, no prior work has talked about the empirical implementation of clustered standard errors in finite samples by popular statistics software (e.g., SAS and STATA). This note tries to fill this void in the literature. Asymptotic estimates of clustered SEs (as in equations 1 and 2) are likely downward biased in finite samples with a limited number of clusters. Therefore, in finite samples, Cameron and Miller (2011) suggest there could be three possible ways to adjust the clustered VAR-COV matrix (see Cameron and Miller 2011 for details about the three alternatives). One of these alternative specifications is to adjust the VAR-COV matrix by G/(G-1), where G is the number of clusters. STATA and SAS both have an approach that is similar to this finite sample specification, though SAS also provides an estimate which is not finite sample adjusted and can be misused by researchers (see details below). The other two specifications are alternative jackknife estimates of the clustered standard error. 1 Specifically, one specification uses the adjustment factor [I N -H], where I N is an identity matrix, H = X (X X) -1 X ; another specification uses the adjustment factor G/(G-1) [I N -H] -1. My note focuses on the first specification of finite sample adjustment G/(G-1), because this is similar to what several popular software packages use (see discussions below) and is used in several programs shared by other professors online. 1 These are analogs of HC2 and HC3 in MacKinnon and White (1985). MacKinnon and White (1985) discussed alternative specifications of white standard errors in finite samples, which are knows as HC, HC1, HC2, and HC3. See MacKinnon and White (1985) for details. 2

4 However, no study has examined whether G/(G-1) provides better finite sample adjustments than other specifications. 2.1 Two Common Mistakes in Calculating Clustered Standard Errors Mistake 1: Not using a finite sample-adjustment. As a result of this finite sample adjustment G/(G-1), the finite-sample estimates of oneor two-way standard errors could be very different than those calculated based on the asymptotic estimate. For example, if an author wants to cluster by year (or by both firm and year) in a sample with 5 years (G=5), the adjustment factor for V time would be 5/(5-1)=1.25. However, if this adjustment factor is not used, the t-statistics could be 11.8% over-estimated, 2 leading to completely different conclusions. Even if the number of years increases to 20, the t-statistics could still be 2.4% ( 20/19 1) overestimated if the finite sample adjustment is not used. Therefore, it is very important to explain whether and how the author implemented finite sample adjustments, especially when the number of clusters is limited even in a large sample with many observations. 3 Mistake 2: Use V firm-time when there is only one observation per firm-time. As discussed above and also in Petersen (2009) and Thompson (2011), V white should be used rather than V firm-time, when there is only one observation per firm-time. However, if a researcher always uses V firm-time rather than V white regardless of whether there are one or more observations per firm-time, this would also result in inaccuracy. This is because V firmtime is adjusted by G/(G-1), but V white is not adjusted by G/(G-1). Generally, because G/(G- 1)>1, V firm-time is larger than V white. So, when V firm-time instead of V white is subtracted 2 V is over-stated by Thus, standard error is over-estimated by 1.118, which is the square root of For example, if there are 1,000 firms each year for 5 years, the sample is still large (5,000 observations). But, the number of clusters for years is only 5. This type of sample is very common in the literature. 3

5 off, as in equation 2, V firm&time would be under-estimated. Then, the t-statistics would be over-estimated, potentially leading to wrong conclusions. Therefore, it is also very important to explain whether V white or V firm-time is used in calculating V firm&time. In the discussion below, I try to show how clustered and white standard errors are calculated in STATA and SAS. I hope this note can help researchers understand what their software actually gives them. 2.2 STATA Finite Samples Adjustment Formula STATA could use the following code from Professor Mitchell Petersen s website to estimate one-way cluster-robust standard errors. regress dependent_variable independent_variables, robust cluster(cluster_variable) According to page 54 STATA USER MANUAL 20 Estimation and postestimation commands, STATA uses the following adjustment for finite samples: V firm is multiplied by (N-1)/(N-P)* G 1 /(G 1-1). 4 N is the sample size, P is the number of independent variables, G 1 is the number of firm-clusters. For example, if there are 100 firms, then G 1 is 100. When N becomes large (relative to P), this adjustment factor is approximately G 1 /(G 1-1). V time is multiplied by (N-1)/(N-P)* G 2 /(G 2-1). N is the sample size, p is the number of independent variables, G 2 is the number of time-clusters. V firm-time is multiplied by (N-1)/(N-P)* G 3 /(G 3-1). N is the sample size, P is the number of independent variables; G 3 is the number of firm-time intersections/clusters. White SEs can be obtained by using the following STATA code: 4 The STATA manual uses M instead of G for notations. But, to be consistent in my discussion, I keep using G. 4

6 regress dependent_variable independent_variables, robust Importantly, the V white matrix given by this code is the unadjusted matrix as in White (1980). If a user wants to use alternative specifications of heteroskedasticity consistent SEs (i.e., HC1,HC2, or HC3), this code need to be modified. 2.3 SAS Finite Samples Adjustment Formula In SAS, there are two way to estimate clustered SEs: proc surveyreg and proc genmod. There is one major difference between these two functions 5 : that is, proc genmod does not provide finite sample adjustment, while proc surveyreg adjust the SEs by (N-1)/(N-P)* G/(G- 1). Therefore, the SEs given by proc genmod are likely to be under-estimated in finite samples. proc genmod; class identifier; model depvar = indvars; repeated subject=identifier / type=ind; run; quit; from Professor Stoffman s website. To adjust the proc genmod results, multiply the VAR-COV matrix by G/(G-1) or (N-1)/(N-P)* G/(G-1). Another SAS code for clustered standard error is proc surveyreg, which does provide finite sample adjustment. This proc surveyreg is mentioned on Professor Petersen s website proc surveyreg data=mydata; cluster cluster_variable; model dependent variable = independent variables; run; from Professor Petersen s website. According to SAS/STAT 9.2 User s Guide PAGE 6556, the matrix is as following 5 Professor Noah Stoffman s website also explains this. See 5

7 proc surveyreg is designed to analyze survey data. h is the stratum index. nh is the number of clusters. fh is the sampling rate for stratum h. The number of fh is negligible, unless a unique sample rate is specified. Therefore, fh is generally negligible when using the code above. To simplify this and make this matrix comparable to my discussions of the STATA matrix, I translate this adjustment as the finite sample adjustment factor in SAS is also approximately (N- 1)/(N-P)* G/(G-1). In addition, for the VAR-COV matrix to be weighted by G/(G-1) rather than (N-1)/(N- P)* G/(G-1), tone must specify VADJUST=NONE in the model statement. V white in SAS is also the same as in White (1980), unless specific forms of alternative specification of White SEs are chosen by the user. Specifically, set the option HCCMETHOD to 0 or 1 or 2 or 3 to get the white HC, HC1, HC2, HC3 SEs as described in MacKinnon and White (1985). 6 The white SEs can be obtained using the following code: proc reg; model y=x / hcc HCCMETHOD=0; run; quit; 3. Suggestions I provide two suggestions to authors using two-way clustered SEs. First, I suggest that authors explicitly identify how they calculate two-way cluster-robust standard errors. Specifically, the author needs to explain 1) whether V white or V firm-time is used in estimating V firm&time and 2) whether and how the estimate is finite sample adjusted. Second, SAS and STATA have options that provide a finite sample adjustment to the clustered SEs in a way similar to one specification suggested by Cameron and Miller (2011). However, SAS also has another option that does not do so. Therefore, if not explained, no reader 6 See 6

8 would know which estimate is used. Also, no studies in accounting and/or finance have compared alternative finite sample estimates of two-way clustered SEs. I suggest that future research carefully examines which alternative specification gives the best finite sample adjustment. In addition, similar issues of finite-sample adjustment exist for one-way clusterrobust standard errors. Finally, a SAS macro code for two way clustered standard errors is available at my personal website: To use this code, please add the following footnote "To obtain unbiased estimates in finite sample, the clustered standard errors are adjusted by (N-1)/(N-P) G/(G-1), where N is the sample size, P is the number of independent variables, and G is the number of clusters." Thus, a reader could better interpret the empirical results. When other researchers share their codes online, they should also explain to the users how their codes perform finite sample adjustments. References Cameron A. C. and D. L. Miller, 2011,Robust Inference with Clustered Data, in A. Ullah and D. E. Giles eds., Handbook of Empirical Economics and Finance, CRC Press, MacKinnon, J. and H., White, Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties, Journal of Econometrics 29(3), Petersen, M., 2009, Estimating Standard Errors in Finance Panel Data Sets: Comparing Approaches, Review of Financial Studies, 22,1: Thompson, S., 2011, Simple formulas for standard errors that cluster by both firm and time, Journal of Financial Economics, 99, 1: White, H. 1980, A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity, Econometrica, 48, Gow, I., G. Ormazabal, and D.Taylor, 2010, Correcting for Cross-Sectional and Time-Series Dependence in Accounting Research, The Accounting Review, 85, 2,