Research with Large Databases

Size: px

Start display at page:

Download "Research with Large Databases"

Gervais Foster
6 years ago
Views:

1 Research with Large Databases Key Statistical and Design Issues and Software for Analyzing Large Databases John Ayanian, MD, MPP Ellen P. McCarthy, PhD, MPH Society of General Internal Medicine Chicago, 2004

2 Today s Objectives Practical issues in data management and potential pitfalls Overview of Sampling Designs Key analytic issues to consider when analyzing data with complex sampling designs Overview of command language for SUDAAN Similarities and differences between SAS and SUDAAN What to look for in survey documentation before beginning analyses

3 Practical Issues in Data Management of Large Databases Convert ASCII (raw) data to SAS database Construct smallest analytic file possible Merge or concatenate files (if necessary) Identify study sample Select variables of interest What to look for in documentation of complex surveys Common sources of Error using SUDAAN

4 Commonly used Statistical Software such as SAS Useful to prepare data for analysis Merge and combine information from multiple data sources Subset data to identify the sample that you want to study Create and recode variables Limited number of procedures to properly analyze data from complex survey designs in version 8+

5 SUDAAN Software for the Statistical Analysis of Correlated Data Analyze data from complex sample designs, including repeated measures, multistage samples, and clustercorrelated data Computes appropriate standard errors that account for the sample design Flexible - includes many design options (SRS, WR, WOR) SAS-Callable SUDAAN - execute SUDAAN procedures within SAS

6 Identify Appropriate Statistical Software for Analyses SAS SAS-callable SUDAAN SEER Public Use NHIS SEER-Medicare NAMCS NIS BRFSS

7 Construct Smallest Analytic File Possible SAS Analytic file should include ONLY study sample and variables of interest SUDAAN Analytic file requires ALL observations to perform analyses, but limit the number of variables Limit variables using DROP or KEEP option in your DATA statement DROP age - - race diag1-diag10; KEEP age - - race diag1-diag10;

8 Merging Data Files Example using NHIS: You want to take a few variables from the person file (i.e. health insurance status) and merge it onto the Adult core Person File N=100,000 Adult Sample Core N=32,000

9 Steps to merging in SAS Merging Data Files Continued 1) Each file must be sorted by the merge variable(s) 2) Select only the variables you want to add into your analytic file 3) Merge files in new DATA step 4) Whenever possible construct permanent dataset with added variables Note: SUDAAN users must re-sort data by variables listed in NEST statement

10 Merging Data Files Example SAS Code Goal: Link insurance from person file to sample adult file DATA person; SET IN.PERSON00 (KEEP=IDNUM INSURE); PROC SORT; BY IDNUM; RUN; DATA adult; SET IN.ADULT00; PROC SORT; BY IDNUM; RUN; DATA OUT.newADULT; MERGE person adult (in=case); BY IDNUM; IF CASE; retains only subjects in the adult file! PROC SORT; BY STRATUM PSU; RUN;

11 Concatenate Data Files Example using NAMCS: You want to combine data from different survey years (e.g., 1998 to 2000) to a single dataset (i.e., 75,000 visits) Year 1998 n=25,000 Year 1999 N=25,000 Year 2000 N=25,000

12 Concatenate Data Files Example SAS Code Goal: Increase sample size for specific subgroup estimates by combining multiple years of data DATA OUT.namc9800; SET IN.NAMC98 IN.NAMC99 IN.NAMC00; PROC SORT; BY stratm psum subfile prostrat year provider dept su clinic: RUN; Which weight do I use? Often you will want to use WEIGHT/3 to get an average annual weight

13 Working with ICD-9-CM Diagnosis and Procedure Codes Identify all codes of interest Codes available at Example: Identify cases of AMI (diagnosis code 410.xx) or CHF (diagnosis code 428.x) in hospital discharge abstracts In SAS, use ARRAYS and DO LOOPS to search all of the diagnosis fields

14 Sample SAS Code: Working with ICD-9-CM Diagnosis Codes Example: Identifying cases of AMI or CHD across 10 potential diagnosis fields ARRAY diag(10) DIAG1-DIAG10; ami=2; chf=2; DO i=1 to 10; IF SUBSTR(diag(i),1,3) = 410 then AMI=1; ELSE SUBSTR(diag(i),1,3) = 428 then CHF=1; END; Similar code can be used to identifying procedures of interest across multiple procedure fields

15 Identify Study Sample SAS POTENTIAL PITFALL SAS and SUDAAN differ in the way you subset data Example: Identify women age years SAS smallest analytic file possible Apply inclusion (and exclusion) criteria to your data and DELETE the subjects that do not meet criteria IF SEX=2 AND (AGE GE 50 and AGE LE 69);

16 Identify Study Sample SUDAAN SUDAAN requires all observations to be present to compute variance estimates Apply inclusion (and exclusion) criteria to your data and construct an INDICATOR variable to identify subjects who meet inclusion criteria IF SEX=2 AND (AGE GE 50 and AGE LE 69) THEN SAMPLE=1; ELSE SAMPLE=2; SUBPOPN ~ maintains the integrity of the study design while allowing analysis on only a subgroup of observations (subjects/respondents) SUBPOPN SAMPLE=1;

17 Overview of Sampling Goal: To obtain a study sample that is representative of the population of interest Sample designs are commonly used to control or minimize survey costs, facilitate survey administration, improve estimates for subgroups Most complex sampling designs tend to produce LARGER variances relative to a Simple Random Sample

18 Overview of Sampling Many national and other large surveys use sampling methods to identify respondents Examples of Sampling Designs include Simple Random Sampling Systematic Sampling Stratified Sampling Cluster Sampling

19 Examples of Sample Designs National Health Interview Survey uses a stratified multistage sample design to identify a nationally-representative civilian household population State-level stratification (Primary Sampling Units or PSUs) Census blocks within strata (Secondary Sampling Units or SSUs) Allows for over-sampling specific minority groups (African Americans and Hispanics) Households Persons within households One adult and one child Details at

20 Variance Estimation and Test Statistics Failure to take complex sampling designs into account will produce biased variance estimates and test statistics Analyzing survey data with a complex sampling design under the assumptions of a Simple Random Sample will produce smaller p-values and narrower confidence intervals and increase the likelihood of a Type I error Most standard statistical packages do not take into consideration the sample design to compute variances and test statistics SAS version 8+ has limited capabilities

21 Sample Weights Each respondent has a sampling weight which is equal to: WEIGHT = 1/(probability of selection) Weights are often adjusted for age, race/ethnicity, and non-response in national surveys Weights are used to inflate sample back to the population of interest Sum of WEIGHTS = size of the population of interest (e.g., non-institutionalized civilian US population) Failure to take weights into account will produced biased point estimates (proportions, means, regression coefficients)

22 Commonly Used Procedures SAS SAS-Callable SUDAAN PROC FREQ; PROC CROSSTAB; PROC MEANS; PROC DESCRIPT; PROC UNIVARIATE; PROC DESCRIPT; PROC LOGISTIC; PROC RLOGIST;

23 SUDAAN Language DESIGN NEST WEIGHT NOTE: This information is always provided in the survey documentation!

24 SUDAAN Language WEIGHT Inverse probability of selection Commonly adjusted for age, race/ethnicity, nonresponse REQUIRED to produce unbiased estimates and national estimates WEIGHT WTFA_AD;

25 SUDAAN Language DESIGN Specifies sample design Most common WR - With Replacement (NHIS, NIS, MEPS) WOR - Without Replacement (NAMCS) DESIGN=WR; Leads to specific variance estimation procedures

26 SUDAAN Language NEST Specifies sampling levels or stages in your design Most common STRATA: stratification variable PSU (primary sampling unit): primary cluster variable NEST STRATUM PSU; NOTE: Data set MUST be SORTED by the variables as listed on the NEST statement

27 Example Code: SAS Version of Bivariable Analysis PROC FREQ; TABLES race*mammo /chisq; WEIGHT wtfa_ad; RUN; SAS without WEIGHT option provides Accurate sample size estimates INACCURATE point estimates, test statistics and p- values! SAS with WEIGHT option provides ACCURATE point estimates INACCURATE test statistics and p-values!

28 Example Code: SUDAAN Version of Bivariable Analysis PROC CROSSTAB data=work.nhis filetype=sas DESIGN=WR; NEST stratum psu /missunit; SUBPOPN sample=1; WEIGHT wtfa_ad; SUBGROUP race mammo ; LEVELS 3 2 ; TABLES race*mammo; SETENV colwidth=9 decwidth=2 colspce=2; PRINT nsum wsum rowper colper / wsumfmt=f9.0 nsumfmt=f9.0; PRINT chisq chisqdf chisqp / chisqdffmt=f8.0 chisqpfmt=f8.4 style=nchs; RUN;

29 SAS Results: Proc Freq Without WEIGHT With WEIGHT

30 SAS Versus SUDAAN Results SAS with WEIGHT SUDAAN

31 Which WEIGHT do I use in my analysis? Example using NHIS Goal: Representative of US Population PERSON File N=100,000 WTFA Sample Adult Core Cancer Control Module N=32,000 WTFA_SA Sample Child Core N=16,000 WTFA_SC

32 Example Code: SUDAAN With SUBPOPN Statement PROC CROSSTAB data=work.nhis filetype=sas DESIGN=WR; NEST stratum psu /missunit; SUBPOPN sample=1; WEIGHT wtfa_ad; SUBGROUP race mammo ; LEVELS 3 2 ; TABLES race*mammo; SETENV colwidth=9 decwidth=2 colspce=2; PRINT nsum wsum rowper colper / wsumfmt=f9.0 nsumfmt=f9.0; PRINT chisq chisqdf chisqp / chisqdffmt=f8.0 chisqpfmt=f8.4 style=nchs; RUN;

33 Requirements of SUDAAN Quirks/Common Sources of Error Input dataset MUST be sorted by the design variables listed in the NEST statement All observations must be present to compute variance estimates Use SUBPOPN statement All analytic variables MUST be NONZERO (1,2,3 ) EXCEPTION: Outcome of a logistic regression model must be coded as 0,1 MISSUNIT option in NEST statement for most surveys NEST stratum psu /missunit;

34 Complex Surveys What to look for in documentation What is the survey design (NEST)? Strata, PSU If you combine multiple years of a survey then YEAR must be considered in the design How were respondents sampled? With replacement (WR) or without replacement (WOR) Which WEIGHT variable do I use? Final analysis weight (final basic weight) Use weight from smallest component of survey

Methods for Estimating Change from NSCAW I and NSCAW II

Methods for Estimating Change from NSCAW I and NSCAW II Paul Biemer Sara Wheeless Keith Smith RTI International is a trade name of Research Triangle Institute 1 Course Outline Review of NSCAW I and NSCAW