0.1 Stata Program 50 /****---*---*****/ 31 /* Obtain Data - Populate Source Folder */

Size: px

Start display at page:

Download "0.1 Stata Program 50 /********-*********-*********-*********-*********-*********-*********/ 31 /* Obtain Data - Populate Source Folder */"

Andra Lloyd
5 years ago
Views:

1 0.1 Stata Program 1 capture log close master // suppress error and close any open logs 2 log using RDC3-master, name(master) replace text 3 // program: RDC3-master.do 4 // task: Demonstrate basic Stata Workflow 5 // version: First draft 6 // project: Texas Census Research Data Center Workshop on 7 // project management 8 // author: Nathanael Rosenheim \ Jan clear all // Clear existing data files 11 macro drop _all // Drop macros from memory 12 version 10 // Set Version 13 set more off // Tell Stata to not pause for --more-- messages // What it is the root directory on your computer? 16 * NOTE * Use forward slash (/) - Recognized by all OS 17 * NOTE * MacOSX and Windows require different directory structures 18 local rootdir = "C:/Users/Nathanael/Dropbox/MyProjects" cd " rootdir " // change to project directory 21 * Stata can create folders if they do not exist 22 capture mkdir RDC3 // Project directory 23 cd RDC3 24 capture mkdir Source // Original, unchanged data sources 25 capture mkdir Derived // Constructed from the source data 26 capture mkdir Clean // Data to explore and interpret 27 capture mkdir Tables_Figures // Output Tables and Figures 28 capture mkdir Text // Text to share /********-*********-*********-*********-*********-*********-*********/ 31 /* Obtain Data - Populate Source Folder */ 32 /********-*********-*********-*********-*********-*********-*********/ 33 * US Census Bureau - Texas County Characteristics Datasets: 34 * Annual County Resident Population Estimates by 35 * Age, Sex, Race, and Hispanic Origin: April 1, 2010 to July 1, copy " " /// 37 "Source/POP_CC-EST2013-ALLDATA-48.csv", replace 38 copy " /// 39 "Source/POP_CC-EST2013-ALLDATA-48_Codebook.pdf", replace * Small Area Income and Poverty Estimates 42 * Texas County Estimates for copy " /// 44 "source/saipe_est10all.xls", replace 45 copy " /// 46 "source/saipe_est10all_codebook.txt", replace /********-*********-*********-*********-*********-*********-*********/ 49 /* Scrub Data - Derive Stata Files from Sources */ 50 /********-*********-*********-*********-*********-*********-*********/ 51 /* Common scrubbing tasks 52 - Convert data from one format to another 53 - Filter observations 54 - Extract and replace values 55 - Split, merge, stack, or extract columns */ 56 1

2 57 * Create Population Estimates Stata file from CSV 58 insheet using "Source/POP_CC-EST2013-ALLDATA-48.csv" 59 save "derived/pop_cc-est2013-alldata-48.dta", replace * Create SAIPE Stata file from Excel 62 import excel "Source/SAIPE_est10ALL.xls", clear 63 save "derived/saipe_est10all.dta", replace * Drop and Add Variables Population File Files 66 use "derived/pop_cc-est2013-alldata-48.dta", clear 67 keep if year == 1 // 4/1/2010 Census population 68 keep if agegrp == 0 // Total age groups 69 keep state county tot_pop wa* ba* h_* // Keep white, black Hispanic totals 70 gen tot_wa = wa_male + wa_female // total white alone population 71 gen tot_ba = ba_male + ba_female // total black alone population 72 gen tot_h = h_male + h_female // total Hispanic alone population 73 keep state county tot* // drop sex variables 74 save "derived/pop_2010_tx", replace * Clean SAIPE Excel Files 77 use "derived/saipe_est10all.dta", clear 78 drop E-G K-AE // Do not need variables 79 rename H PALL // Poverty Percent All Ages 80 label variable PALL "Poverty Percent All Ages" 81 * Example of Stata native variables 82 drop if _n <= 3 // Drop first 3 rows 83 keep if A == "48" // Keep Texas 84 destring, replace // Convert Strings to numeric 85 save "derived/saipe_2010_tx", replace * Add Merge ID - FIPS County Pop Data 88 use "derived/pop_2010_tx", clear 89 // generated FIPS_Code from State and County Codes 90 gen str5 FIPS_County = string(state,"%02.0f")+string(county,"%03.0f") 91 sort FIPS_County 92 save "derived/pop_2010_tx_id", replace 93 * Add Merge ID - FIPS County SAIPE Data 94 use "derived/saipe_2010_tx", clear 95 // generated FIPS_Code from State and County Codes 96 gen str5 FIPS_County = string(a,"%02.0f")+string(b,"%03.0f") 97 sort FIPS_County 98 save "derived/saipe_2010_tx_id", replace * Merge SAIPE and SEER Data 101 use "derived/pop_2010_tx_id", clear 102 merge FIPS_County using "derived/saipe_2010_tx_id" 103 save "derived/saipe_pop_2010_tx", replace * Drop uneeded variables and reorder 106 use "derived/saipe_pop_2010_tx", clear 107 drop state county A B _merge 108 order FIPS_County D C 109 save "derived/saipe_pop_2010_tx_fltr", replace * Add pop percent variables, label new variables 112 use "derived/saipe_pop_2010_tx_fltr", clear 113 * EXAMPLE OF LOOP 114 foreach re in wa ba h { // loop through white, black Hispanic 115 gen p_ re = tot_ re / tot_pop * format p_ re %04.2f // 2

3 117 } 118 * Label variables 119 label variable p_wa "Percent White" 120 label variable p_ba "Percent Black" 121 label variable p_h "Percent Hispanic" 122 /* */ 123 /* Clean Data - Final scrub - Save File to Clean Folder */ 124 /* */ 125 save "clean/rdc3-saipe_pop_2010_tx", replace 126 outsheet using /// 127 "clean/rdc3-saipe_pop_2010_tx.csv", comma replace // To use in R /********-*********-*********-*********-*********-*********-*********/ 130 /* Explore Data - Create Tables and Figures to Interpret */ 131 /********-*********-*********-*********-*********-*********-*********/ 132 * ssc install estout, replace // to create tables install estout 133 * Create Table with Descriptive Statistics 134 use "clean/rdc3-saipe_pop_2010_tx", replace 135 local dscrb_vars PALL p_* // Variables to describe 136 capture noisily eststo clear 137 capture noisily estpost tabstat dscrb_vars, /// 138 statistics(min max p50 mean sd count) columns(statistics) 139 capture noisily esttab using /// 140 "tables_figures/rdc3-tables.rtf" /// 141, alignment(r) replace label gaps modelwidth(6) nonumbers /// 142 cells("count(fmt(%4.0f)) min(fmt(%4.2f)) max(fmt(%4.2f)) p50(fmt(%4.2f)) mean(fmt(%4.2f)) sd(fmt(%4.2f))") noobs /// 143 title(basic Descriptive Statistics Poverty and Population Data for Texas Counties 2010) /// 144 addnote(" c(filename) c(current_date) ") 145 eststo clear * Create Histogram of poverty 148 use "clean/rdc3-saipe_pop_2010_tx", replace 149 local graphcaption = "Histogram of Percent Poverty for Texas Counties 2010." 150 histogram PALL, frequency normal kdensity /// 151 title( graphcaption ) /// 152 caption(" c(filename) c(current_date) ", size(tiny)) 153 graph export "tables_figures/histpall.pdf", replace 154 notes: Poverty has a normal distribution 155 save "clean/rdc3-saipe_pop_2010_tx", replace 156 /********-*********-*********-*********-*********-*********-*********/ 157 /* Model Data */ 158 /********-*********-*********-*********-*********-*********-*********/ 159 * Output Regression Table 160 use "clean/rdc3-saipe_pop_2010_tx", replace 161 capture noisily eststo: regress PALL p_* capture noisily esttab using /// 164 "tables_figures/rdc3-tables.rtf" /// 165, b(%4.3f) se(%4.3f) ar2 onecell append label modelwidth(6) nonumbers /// 166 title(parameter Estimates from Models of Poverty with Race and Ethnicity) /// 167 alignment(c) parentheses /// 168 addnote(" c(filename) c(current_date) ") 169 capture noisily eststo clear 170 notes: Race and ethnicity predictors have significant coef. 171 notes: Include median income? Will race still be significant? 172 save "clean/rdc3-saipe_pop_2010_tx", replace 173 /********-*********-*********-*********-*********-*********-*********/ 174 /* Interpret Data */ 3

4 175 /********-*********-*********-*********-*********-*********-*********/ 176 notes // View notes 177 // Example of how Stata stores regression results 178 tempname handel1 179 file open handel1 using "text/rdc3-interpret.rtf", write replace 180 file write handel1 "For Texas in 2010, " 181 file write handel1 "there is a significant association between " 182 file write handel1 "poverty and race." _n 183 file write handel1 "The model had e(n) counties and " _n 184 file write handel1 "an adjusted r-sqaure of e(r2_a). " _n 185 file write handel1 "See notes store in c(filename) " 186 file close handel /********-*********-*********-*********-*********-*********-*********/ 189 /* End Log */ 190 /********-*********-*********-*********-*********-*********-*********/ log close master 193 * Exit Program 194 exit * NOTE * Nothing below "exit" will run or be included in the log file eststo, esttab come from estout 199 For more information on estout see: Making Regression Tables in Stata /********-*********-*********-*********-*********-*********-*********/ 203 /* QUICK STATA REMINDERS */ 204 /********-*********-*********-*********-*********-*********-*********/ 205 Good resources: Programming Stata Stata is case sensitive - for variable names as well as commands. 211 Stata sees a return as then end of a line Create indents with spaces instead of tabs - if possible. 214 Avoid spaces in folder and file names Macros - See help macro 217 /********-*********-*********-*********-*********-*********-*********/ 218 local name = expression 219 name defined by expression 220 Use for numbers, strings, nested macros ie. name = " var i " 221 During execute in the do-file where created local name variable list 224 name contains list of variables * NOTE * Missing equal sign 225 Great for model variables 226 During execute in the do-file where created global same options as local 229 $name or ${name} - when using {} name can be nested ${ var i } 230 Great for project name, filter options 231 During current Stata session, across all do-files STATA If Conditions 234 Operator Meaning 4

5 235 /********-*********-*********-*********/ 236 == equal to 237 > greater than 238 >= greater than or equal to 239 < less than 240 <= less than or equal to\ 241!= or = not equal to 242 & combine operators AND 243 combine operators OR Stata has two built-in variables called _n and _N. 246 _n is Stata notation for the current observation number. 247 _n is 1 in the first observation, 2 in the second, 3 in the third, and so on. 248 _N is Stata notation for the total number of observations Comment types 252 Comments may be added to programs in three ways: 253 o begin the line with *; 254 o begin the comment with //; or 255 o place the comment between /* and */ delimiters /// is one way to make long lines more readable 258 Like the // comment indicator, the /// indicator must be preceded by one or 259 more blanks Additional ways to control Stata 262 set varabbrev off // Turn off variable abbreviations 263 set linesize 80 // Set Line Size - 80 Characters for Readability Stata 12 can not read *.dta files saved in Stata saveold will fix this problem To create a directory use the command capture mkdir /********-*********-*********-*********-*********-*********-*********/ 271 /* Excercises */ 272 /********-*********-*********-*********-*********-*********-*********/ 273 Add Median income and include a second model in the output Make the program more robust to run for different states. 5

6 0.1.1 SAS Program 1 * Turn on SYBMBOLGEN option to see how macro variables are resolved in log; 2 * global system option MPRINT to view the macro code with the 3 macro variables resolved; 4 options SYMBOLGEN MPRINT; 5 /********-*********-*********-*********-*********-*********-*********/ 6 /* Description of Program */ 7 /********-*********-*********-*********-*********-*********-*********/ 8 * program: RDC2-master.SAS 9 * task: Run all do-files associated with project 10 * project: Texas Census Research Data Center Workshop on 11 * project management 12 * author: Nathanael Rosenheim \ Jan *; 13 /* */ 14 /* Important Folder Locations */ 15 /* */ 16 %LET rootdir = C:\Users\Nathanael\Dropbox\MyProjects\RDC3\; 17 * Where is the source data?; 18 %LET dd_data = &rootdir.source\; 19 * Where will the sas7bdat files be saved?; 20 %LET dd_saslib = &rootdir.; 21 /* */ 22 /* Define SAS Library */ 23 /* */ 24 %let library = Derived; 25 LIBNAME &library "&dd_saslib.sas_derived"; /********-*********-*********-*********-*********-*********-*********/ 28 /* Obtain Data */ 29 /********-*********-*********-*********-*********-*********-*********/ 30 * Data downloaded and located in RootDir/Source/; 31 /********-*********-*********-*********-*********-*********-*********/ 32 /* Scrub Data - Derive Stata Files from Sources */ 33 /********-*********-*********-*********-*********-*********-*********/ 34 * Create Population Estimates SAS file from CSV; 35 PROC IMPORT DATAFile = "&dd_data.saipe_est10all.xls" 36 DBMS = XLS OUT = &library..saipe_est10all REPLACE; 37 DATAROW=3; 38 GETNAMES = NO; 39 MIXED = YES; 40 RUN; 41 * Clean SAIPE Excel Files; 42 DATA &library..saipe_2010_all_messy REPLACE; 43 SET &library..saipe_est10all; 44 IF A = "" THEN DELETE; 45 IF B = "" THEN DELETE; 46 stfips = input(a, Best12.); 47 County = input(b,best12.); 48 State = input(c,$char2.); 49 CountyName = input(d,$char20.); 50 Year = 2010; 51 PALL = input(h,comma15.); 52 PALLLB = input(i,comma15.); 53 PALLUB = input(j,comma15.); 54 RUN; 55 * Drop the original variables to create an unsorted or messy temp file; 56 DATA &library..saipe_2010_all REPLACE; 57 SET &library..saipe_2010_all_messy (DROP = A B C D E F G H I J K L M N O P Q R S T U V W X Y Z AA AB AC AD AE); 6

7 58 RUN; 59 * Add Merge ID - FIPS County SAIPE Data; 60 DATA &library..saipe_2010_all_id REPLACE; 61 SET &library..saipe_2010_all; 62 If stfips LT 10 Then do; 63 StateFP = "0" PUT(stfips, 1.); 64 If County LT 10 Then 65 FIPS_County = "0" PUT(stfips, 1.) "00" PUT(County, 1.); 66 ELSE If County LT 100 Then 67 FIPS_County = "0" PUT(stfips, 1.) "0" PUT(County, 2.); 68 ELSE If County LT 1000 Then 69 FIPS_County = "0" PUT(stfips, 1.) PUT(County, 3.); 70 End; 71 Else If stfips LT 100 Then do; 72 StateFP = PUT(stfips, 2.); 73 If County LT 10 Then 74 FIPS_County = PUT(stfips, 2.) "00" PUT(County, 1.); 75 ELSE If County LT 100 Then 76 FIPS_County = PUT(stfips, 2.) "0" PUT(County, 2.); 77 ELSE If County LT 1000 Then 78 FIPS_County = PUT(stfips, 2.) PUT(County, 3.); 79 End; 80 RUN; 81 * Add labels; 82 Data &library..saipe_2010_all_labeled; 83 Set &library..saipe_2010_all_id; 84 LABEL 85 StateFP = "State FIPS" 86 FIPS_County = "County FIPS" 87 State = "Postal" 88 CountyName = "Name" 89 PALL = "Estimated percent of people of all ages in poverty" 90 PALLLB = "90% CI LB of estimate of percent of people of all ages in poverty" 91 PALLUB = "90% CI UB of estimate of percent of people of all ages in poverty" 92 run; 93 * Sort Data; 94 DATA &library..saipe_2010_all_labeled REPLACE; 95 Retain 96 FIPS_County 97 Year; 98 SET &library..saipe_2010_all_labeled; 99 RUN; 100 /* */ 101 Clean Data - Final scrub - Save File to Clean Folder */ 102 /* */ 103 %let library = Cleaned; 104 LIBNAME &library "&dd_saslib.sas_clean"; Proc Sort data = Derived.SAIPE_2010_All_labeled 107 out = &library..rdc3_saipe_pop_2010_tx; 108 by FIPS_county year; 109 Run; 7

8 0.1.2 R Program 1 # program: RDC3-master.R 2 # task: Demonstrate basic Stata Workflow 3 # version: First draft 4 # project: Texas Census Research Data Center Workshop on 5 # project management 6 # author: Nathanael Rosenheim \ Jan # What it is the root directory on your computer? 10 setwd("c:/users/nathanael/dropbox/myprojects/rdc3") # *******-*********-*********-*********-*********-*********-*********/ 13 # Obtain Data - */ 14 # *******-*********-*********-*********-*********-*********-*********/ 15 # Data output from Stata RDC3-Master.do file clean <- read.csv("clean/rdc3-saipe_pop_2010_tx.csv", header=true) 18 attach(clean) # *******-*********-*********-*********-*********-*********-*********/ 21 # Model Data */ 22 # *******-*********-*********-*********-*********-*********-*********/ #Figure 6.2 on page m1 <- lm(pall p_wa+p_ba+p_h) 26 summary(m1) detach(clean) 8

Advanced Stata Skills

Advanced Stata Skills Contents Stata output in Excel & Word... 2 Copy as Table... 2 putexcel... 2 tabout (v3)... 2 estout / esttab... 2 Common Options... 2 Nice Looking Tables in Stata... 3 Descriptive