EXAMPLE 3: MATCHING DATA FROM RESPONDENTS AT 2 OR MORE WAVES (LONG FORMAT)

Transcription

1 EXAMPLE 3: MATCHING DATA FROM RESPONDENTS AT 2 OR MORE WAVES (LONG FORMAT) DESCRIPTION: This example shows how to combine the data on respondents from the first two waves of Understanding Society into a long format. This Example also shows how to compute transitions (change in marital status and in personal income from wave 1 to wave 2) in the case of long format. The results obtained in this example (long format) should be the same as the ones in Example 2 (wide format). FILES: a_indresp and b_indresp WAVES: 1 and 2 STEPS: 1. Merge waves 1 and 2 into long format 2. Inspect the data, deal with missing values 3. Identify transitions (e.g. change in marital status and change in pay) 4. Conversion from wide to long format and from long to wide format NEW COMMANDS: - Combining waves and data manipulation: proc append, proc rename, proc sort, data set - Repeating the same series of commands using a macro 1 E x a m p l e 3

2 3.1 COMBINING WAVE 1 AND WAVE 2 DATA INTO LONG FORMAT In this section we will discuss how to combine data from wave 1 and wave 2 into a data file in long format. To do that we need to first make some changes to the dataset of each wave: create a variable to identify which wave the data belongs to: 1 for wave 1, 2 for wave 2,... drop the wave prefix from all variables so that SAS can recognize the variables from two waves as the same variable. Before we get started set all the relevant directories: /*Exercise 3*/ proc datasets lib=work kill nolist memtype=data; quit; /*for sharing with others you will want to write to a log file - but for this exercise we have commented it out*/ /* writing all output to a log file */ /*filename myoutput "SET OUTPUT FILE HERE.txt"; proc printto print=myoutput new; */ /* writing to a log file */ /*filename mylog "SET LOG FILE HERE.log"; proc printto log=mylog new; */ /* defining a library where files can be permanently stored */ libname dir 'define area where files will be permanently stored; options nofmterr; Let us start with wave 1 Read data file a_indresp into SAS Rename all the variables by dropping the prefix a_. This is necessary so that each variable has the same name across waves and can be appended into a single variable. Create a variable that identifies which wave the observation comes from. /* reading in wave 1 data from SPSS file into SAS */ proc import datafile = "specify where your data file is" out=a_indresp dbms= sav replace; /* drop the wave prefixes */ data wave1 (rename=(a_hidp=hidp a_istrtdaty=istrtdaty a_sex=sex a_mastat_dv=mastat_dv a_julkjb=julkjb a_paygu_dv=paygu_dv a_sclfsato=sclfsato a_jbstat=jbstat)); 2 E x a m p l e 3

3 set a_indresp; waveno=1; keep waveno a_hidp pidp a_istrtdaty a_sex a_mastat_dv a_julkjb a_paygu_dv a_sclfsato a_jbstat; The data step option rename renames the variables by specifying first the original name and then the name we want it to be changed to. So, a_hidp will be renamed to hidp, a_sex to sex and so on. Original variable name a_hidp pidp a_sex a_paygu_dv New variable name hidp pidp sex paygu_dv We could deal with missing values here, as we did in the wide format in Example 2, but it is more efficient to do it after the two data files have been combined. Now perform the same steps for wave 2: /* reading in wave 2 data from SPSS file into SAS */ proc import datafile="specify where your data file is" out=b_indresp dbms= sav replace; data wave2 (rename=(b_hidp=hidp b_istrtdaty=istrtdaty b_sex=sex b_mastat_dv=mastat_dv b_julkjb=julkjb b_paygu_dv=paygu_dv b_sclfsato=sclfsato b_jbstat=jbstat)); set b_indresp; waveno=2; keep waveno b_hidp pidp b_istrtdaty b_sex b_mastat_dv b_julkjb b_paygu_dv b_sclfsato b_jbstat; There are multiple ways to combine the files into long format. Here we use a data step, but another method is explored below in 3.4. /* appends dataset wave2 to dataset wave1 and creates a new longitudinal dataset in long format*/ set wave1 wave2; This appends dataset wave2 to wave1 and then creates a new longitudinal dataset in long format called wave12long in the work directory. 3 E x a m p l e 3

4 Note: Although care is taken to retain the same categories of variables across waves, sometimes these need to be changed when new categories become relevant. So, always check the documentation to see if that has happened: This can also be checked by using proc format, for example: proc format library=work fmtlib; select a_istr2a b_istrta; Now look at the combined data file. /* examine the new long format file */ proc univariate data=wave12long; title ''; /*resetting the title option from previous exercise */ var sex mastat_dv; proc freq data=wave12long; tables waveno; 3.2 LOOKING AT TRANSITIONS DATA PREPARATION As in Example 2 recode negative values to. so that SAS can recognize these as missing. /*Looking at transitions*/ /* recode negative values to system missings */ /*this is a simpler method to accomplish the same thing at the array - but it doesn't scale as easily*/ /*if istrtdaty<0 then istrtdaty=.; if sex<0 then sex=.; if mastat_dv<0 then mastat_dv=.; if julkjb<0 then julkjb=.; if paygu_dv<0 then paygu_dv=.; if sclfsato<0 then sclfsato=.;*/ *Changing all out of range coding values to missing; *The _numeric_ option means this command creates an array of all numerical variables; array aa (*) _numeric_; do i=1 to dim(aa); if aa(i) < 0 then aa(i) =.; drop i; end; 4 E x a m p l e 3

5 As we are interested in transitions across the two waves, we will be creating variables that identify these transitions. First however, we need to know which observations are in both waves so that we can construct a balanced panel. Using proc sort, we put the data in order, sorting first by pidp and then by waveno. proc sort data=wave12long; by pidp waveno; To have a balanced panel we keep only those present in both waves. This command uses a proc sql command (SAS also understands SQL). The command tells SAS to count the number of observations by pidp and create a table called t1. This is in the work directory. proc sql; create table wave12long as select *, count(pidp) as counter from wave12long group by pidp order by pidp, waveno; Then we run a data command to only keep respondents who are in both waves of data. if counter=1 then delete; 3.3 IDENTIFYING TRANSITIONS CHANGE IN MARITAL STATUS AND PAY We will now create the same variable as in Example 2 to identify and summarise changes in marital status. But as the data is now in LONG format rather than WIDE format we will need different commands. As in Example 2, first recode marital status into smaller number of categories, generating a new variable, marital. if mastat_dv=1 then marital=1; if mastat_dv in (2,3,10) then marital=2; if mastat_dv in (4,5,7,8) then marital=3; if mastat_dv in (6,9) then marital=4; If the format marital is already defined from Exercise 2, it can be applied to the dataset without recreating it. format marital marital.; 5 E x a m p l e 3

6 To create a variable marchg that measures marital status transitions, we will make use of the array and lag function. The array command defines a list of variables which SAS run a set of commands on and loop over each variable in the array. The dataset needs to be sorted by pidp and wave but we have already done this. /* Creating measures of change */ by pidp waveno; lagmar=lag1(marital); marchg=(lagmar*10)+marital; diffwage=dif(paygu_dv); if first.pidp then do; diffwage=.; end; label marchg = "change in marital status between waves 1 and 2"; label diffwage = "difference in wages between waves 2 and 1"; proc format; value marchg 11= "remains single" 12= "was single now partnered" 13= "was single now separated" 14= "was single now widowed" 21= "was partnered now single" 22= "remains in partnership" 23= "was partnered now separated" 24= "was partnered now widowed" 31= "was separated now single" 32= "was separated now partnered" 33= "still separated" 34= "was separated now widowed" 41= "was widowed now single" 42= "was widowed now partnered" 43= "was widowed now separated" 44= "still widowed"; format marchg marchg.; Task: Label the variable and its values (see Example 2) and look at the frequency distribution of change in marital status. Compare with the frequency of this variable in Example 2. Task: Create a variable diffwage which measures the change in pay (paygu_dv) between the two waves. 6 E x a m p l e 3

7 Question: How does pay change vary by sex and by marital status and sex? It is a good idea to only keep the second observation for each person when you are finished creating change variables, to make it easier to interpret tabulations and other results. We can do this using a data step with an if statement. proc sort data=wave12long; by pidp waveno; if waveno=1 then delete; 3.4 COMBINING DATASETS INTO LONG FORMAT WHEN THERE ARE MORE THAN TWO WAVES When you are planning to merge data from more than two waves it is generally a good idea to write the commands into a loop. This reduces the amount of code you have to write, but more importantly it reduces the chance of error. For this exercise we will demonstrate how to append 4 waves of Understanding Society together. The basic operations are the same as in the first part of this exercise. For each wave: Open the file; Generate a variable (wave) to identify the wave of this data; Rename the variables by dropping the wave prefix; Save as a temporary file, one for each wave: To begin with we need to import the extra files into SAS. proc import datafile = "SPECIFY DIRECTORY WHERE DATA IS HELD" out=a_indresp dbms= sav replace; proc import datafile = "SPECIFY DIRECTORY WHERE DATA IS HELD" out=b_indresp dbms= sav replace; proc import datafile = "SPECIFY DIRECTORY WHERE DATA IS HELD" out=c_indresp dbms= sav replace; proc import datafile = "SPECIFY DIRECTORY WHERE DATA IS HELD" out=d_indresp dbms= sav replace; Now all the datasets are in SAS, we can use a macro to combine them. This is like a SAS program where some part of the program changes as it is completed. In our case, this macro uses the set function in a data step to add a dataset called wave1, to a file called wave2, and increases the number on the end of the dataset name by 1 each time until it gets to 4. The code between %macro and %mend defines the macro called combine, and %combine tells SAS to execute the program. 7 E x a m p l e 3

8 %macro combine(w=,y=); data wave&y.; set work.&w._indresp(rename=(&w._hidp=hidp &w._istrtdaty=istrtdaty &w._sex=sex)); waveno=&y.; keep hidp pidp istrtdaty sex waveno; proc freq data=wave&y.; tables sex; title "wave &y."; proc append BASE=wave1234long DATA=wave&y. force; quit; %mend; %combine(w=a,y=1); title; %combine(w=b,y=2); title; %combine(w=c,y=3); title; %combine(w=d,y=4); title; And then check that all four waves have been included using proc freq. proc freq data=wave1234long; tables waveno; That s that. Now that you have merged all data you would carry on creating your transition variables for all waves. You can now save your final dataset data dir.wave12long; One final task, clear all the datasets you ve created: /* Clean SAS work directory */ proc datasets lib=work nolist kill memtype=all; /*relevant only if you are printing to external log files - stop writing to external log and output files and simply write to the SAS windows proc printto; quit; */ 8 E x a m p l e 3