1 Files to download. 3 Macro to list the highest and lowest N data values. 2 Reading in the example data file

Transcription

1 1 2 22S:172 Lab session 10 Macros for data cleaning July 17, 2003 GENDER VISIT HR SBP DBP DX AE = "Gender" = "Visit Date" = "Heart Rate" = "Systolic Blood Pressure" = "Diastolic Blood Pressure" = "Diagnosis Code" = "Adverse Event?"; 1 Files to download FORMAT VISIT MMDDYY10.; patients.txt lab9.sas 2 Reading in the example data file options linesize = 72 mprint ; * mprint option puts macro debugging info in the log file ; /* Program to create SAS data set PATIENTS */ *LIBNAME CLEAN "C:\TEMP"; LIBNAME CLEAN "/space/kcowles/172/clean"; DATA CLEAN.PATIENTS; INFILE "C:\TEMP\patients.dat" PAD; * INFILE "/space/kcowles/172/clean/patients.dat" PAD; PATNO GENDER VISIT HR SBP DBP DX AE $1.; LABEL PATNO = "Patient Number" 3 Macro to list the highest and lowest N data values %SYSFUNC executes SAS langue functions and returns the results to the macro facility. Here, %SYSFUNC is used to open the dataset and determine the number of observations, and to close the dataset. The calculation of the number of groups uses the FLOOR function so that we err on the side of too few groups (more data values listed) rather than too many. Program 2-15 Creating a macro to list the top and bottom "n" data values, using PROC RANK * * Macro Name: HI_LOW_N Purpose: To list N highest and lowest values (approximately) Arguments: DSN - Data set name (one- or two-level VAR - Variable to test N - Number of highest and lowest values IDVAR - ID variable Example: %HI_LOW_N(CLEAN.PATIENTS,SBP,10,PATNO) * *; %MACRO HI_LOW_N(DSN,VAR,N,IDVAR); ***Find the number of observations in data set; %LET DSID = %SYSFUNC(OPEN(&DSN));

2 3 4 %LET N_OBS = %SYSFUNC(ATTRN(&DSID,NOBS)); %LET RETURN = %SYSFUNC(CLOSE(&DSID)); ***Compute number of groups, from N and N_OBS; %LET GRP = %SYSEVALF(&N_OBS / &N,FLOOR); ***Continue as in the macro based on percents; %LET TOP = %EVAL(&GRP - 1); PROC FORMAT; VALUE RNK 0= Low &TOP= High ; PROC RANK DATA=&DSN OUT=NEW GROUPS=&GRP; VAR &VAR; RANKS RANGE; ***Sort and keep top and bottom n%; PROC SORT DATA=NEW (WHERE=(RANGE IN (0,&TOP))); ***Produce the report; PROC PRINT DATA=NEW; TITLE "Approximate Highest And Lowest &N Values for %UPCASE(&VAR)"; ID &IDVAR; VAR RANGE &VAR; FORMAT RANGE RNK.; PROC DATASETS LIBRARY=WORK NOLIST; DELETE NEW; QUIT; %MEND HI_LOW_N; %HI_LOW_N(CLEAN.PATIENTS,SBP,10,PATNO) Here is the output: Approximate Highest And Lowest 10 Values for SBP 5 PATNO RANGE SBP 020 Low Low Low Low Low Low Low Low Low High High High 190 High High High High High 400 Note that there are only 9 low values and 8 high values listed. The difference from the number requested will be less in larger datasets. Be aware the PROC RANK assigns ranks only to nonmissing values. Programs 2-17 and 2-18 both provide listings of exactly n highest and lowest values, but at the expense of greater processing time. 4 Determining the number of nonmissing observations in a dataset If you have large numbers of missing values for variables in your dataset, there is an alternative (and more CPU-intensive) method for determining the number of nonmissing values instaed of the total number of observations in the dataset. Substitute the following lines for the three %SYSFUNC lines in the previous macro. Then add the data set TMP to the list of data sets fo delete at the end of the macro. Program 2-16 Determining the number of nonmissing observations in a data set ***Find the number of nonmissing observations in data set;

3 5 6 %LET DSN = CLEAN.PATIENTS ; %LET VAR = HR ; PROC MEANS DATA=&DSN NOPRINT; VAR &VAR; OUTPUT OUT=TMP N=NONMISS; SET TMP; ***Assign the value of NONMISS to the macro variable N_OBS; CALL SYMPUT("N_OBS",NONMISS); Here is the output if we make those changes. Approximate Highest And Lowest 10 Values for HR 6 09:05 Thursday, July 17, 2003 PATNO RANGE HR 020 Low Low Low Low Low Low Low Low Low Low 68 XX5 Low Low Low High High High High High High High High High High 88 High High High High High Using PROC SORT to list exactly the highest and lowest n values The disadvantage to this method is that the dataset has to be sorted each time the program is run for a different variable. This is fine for a small dataset, but very slow for a really large one. To make the program as efficient as possible: KEEP= option is used with PROC SORT only the nonmissing observations are placed in the sorted temporary dataset Here is a program to do all this. Program 2-17 Listing the highest and lowest "n values, using PROC SORT *LIBNAME CLEAN "C:\CLEANING"; %LET VAR = HR; ***Assign values to two macro variables; %LET IDVAR = PATNO; PROC SORT DATA=CLEAN.PATIENTS(KEEP=&IDVAR &VAR WHERE=(&VAR NE.)) OUT=TMP; TITLE "Ten Highest and Ten Lowest Values for &VAR"; SET TMP NOBS=NUM_OBS;

4 7 8 HIGH = NUM_OBS - 9; FILE PRINT; IF _N_ LE 10 THEN DO; IF _N_ = 1 THEN PUT / "Ten Lowest Values" ; PUT "&IDVAR = " &VAR; IF _N_ GE HIGH THEN DO; IF _N_ = HIGH THEN PUT / "Ten Highest Values" ; PUT "&IDVAR = " &VAR; Here is the output. Ten Lowest Values PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = Ten Highest Values PATNO = PATNO = PATNO = PATNO = PATNO = PATNO = 90 PATNO = Ten Highest and Ten Lowest Values for HR 7 09:05 Thursday, July 17, 2003 PATNO = PATNO = PATNO = Here is a macro version of the same program. Program 2-18 Creating a macro to list the "n" highest and lowest data values, using PROC SORT * * Program Name: TEN.SAS in C:\CLEANING Purpose: To list the 10 highest and lowest data values for a variable in a SAS data set using Data Step processing VAR - Numeric variable to be checked IDVAR - ID variable name Example: %TEN(CLEAN.PATIENTS,HR,PATNO) * *; %MACRO TEN(DSN,VAR,IDVAR); PROC SORT DATA=&DSN(KEEP=&IDVAR &VAR WHERE=(&VAR NE.)) OUT=TMP; TITLE1 "Ten Highest and Ten Lowest Values for %UPCASE(&VAR)"; TITLE2 "Program 2-18" ; SET TMP NOBS=NUM_OBS; HIGH = NUM_OBS - 9; FILE PRINT; IF _N_ LE 10 THEN DO; IF _N_ = 1 THEN PUT / "Ten Lowest Values" ; PUT "&IDVAR = " "&VAR = " &VAR; IF _N_ GE HIGH THEN DO;

5 9 %MEND TEN; IF _N_ = HIGH THEN PUT / "Ten Highest Values" ; PUT "&IDVAR = " "&VAR = " &VAR; %TEN(CLEAN.PATIENTS,HR,PATNO)