PROC FORMAT. CMS SAS User Group Conference October 31, 2007 Dan Waldo

Transcription

1 PROC FORMAT CMS SAS User Group Conference October 31, 2007 Dan Waldo 1

2 Today s topic: Three uses of formats 1. To improve the user-friendliness of printed results 2. To group like data values without affecting the underlying values themselves 3. To create a new variable that takes values based on the value of an existing variable 2

3 Full disclosure What will be covered Uses of SAS formats Retrieving SAS formats Making SAS formats Limitations Not all aspects of SAS formats will be explored... just the rudiments Examples will use PC-SAS v9 -- but except for JCL differences everything can be done on the mainframe as well. The information provided represents the opinion of the presenter only. It in no way represents the official policy of the Centers for Medicare & Medicaid Services nor the Department of Health and Human Services. Or the SAS Institute. Or anybody in Wisconsin Rapids, Wisconsin, for that matter. So there. 3

4 What is a format? Instrument that presents values for a variable in something other than their native look Assign a tag to a given value (e.g., MALE instead of 1) Group values together (e.g., DISABLED instead of 1 through 64) Gussy up a value (e.g. $1, instead of ) Does not change the native value of the variable 4

5 FORMAT vblname fmtnamew.d; vblname is the name of the variable to be formatted fmtname is the name of the format to be used formats that modify character variables must start with a $ format names do not end with a digit Note the period at the end of fmtname this convention identifies the name as a format W is optional, to specify the width of the resulting field D is also optional, to specify the number of decimals shown in numeric formats Association of fmtname with vblname persists Used in DATA step, remains in dataset Used in PROCs, remains in any downstream datasets Can be removed by null statement FORMAT vblname; 5

6 Where do you find formats? In the SAS system library In a permanent storage area known by the libname (ddname) LIBRARY In your temporary workspace, in a special folder called formats How did they get there? You put them there Somebody else put them there 6

7 Canned formats from SAS cover lots Character formats and lots of situations FORMAT lname upcase.; You have You see Waldo WALDO Numeric formats FORMAT people comma8.1; You have You see ,326.9 Date and time formats FORMAT fill_dte weekdatx.; You have You see Monday, 1 January

8 To get a list of SAS formats from online help, use these links

9 User-defined formats... are created using PROC FORMAT... are stored in a special subsection of WORK... or in a special LIBNAME (ddname) called LIBRARY Can be either a PICTURE overlay or a VALUE translator 9

10 PICTUREs simply lay a template over the native value Numeric data are presented with punctuation of your choosing Advantage of this over other types of formats You can scale the data round off to thousands, or millions, or even both with the same PICTURE Disadvantage of PICTURE SAS does not round off data that are being scaled 10

11 Numbers up to a million are shown in thousands, others in millions Notice: no rounding Notice that the scale factor has to account for decimal places, because the display is not a true number 11

12 VALUEs convert one or more native values to a text representation Numeric data can be shown as numbers or text Likewise for character data Advantage Unlike PICTURE, you are not limited to displaying some form of the native value of the data Disadvantage You cannot scale numbers on the fly, as you can in PICTURE 12

13 The simplest use of formats is to improve the user-friendliness of printed output We want to list the sex of respondents to the Medicare Current Beneficiary Survey Suppose that we have a nominal variable called SEX: 1=male 2=female 13

14 Locate the data Create a user-defined format Print the native values of SEX libname h "h:\sas course\data"; proc format; value sexfmt 1="MALE 2="FEMALE other= UNKNOWN ; run; proc print data=h.bmi (obs=10); * Data Values as They Exist in the Dataset; var baseid sex; run; Print a more user-friendly listing of SEX proc print data=h.bmi (obs=10); *Formatted Data Values; var baseid sex; format sex sexfmt.; run; 14

15 Using the format, the output is more intelligible proc print data=h.bmi (obs=10); var baseid sex; run; Native values Obs baseid sex proc print data=h.bmi (obs=10); var baseid sex; format sex sexfmt.; run; Obs baseid sex MALE MALE MALE FEMALE FEMALE MALE FEMALE FEMALE FEMALE FEMALE What you see 15

16 Remember: 1. name must begin with a $if it modifies character variables 2. name cannot end with a digit So, what did we do? Invoked the FORMAT PROC Defined a format name (sexfmt) Associated individual values with a text replacement (note that we have not identified a specific variable containing these values proc format; value sexfmt 1="MALE 2="FEMALE other= UNKNOWN ; run; 16

17 Gussying up the output applies to PROCs, too... proc freq data=h.bmi;table sex;run; Gender of SP Cumulative Cumulative sex Frequency Percent Frequency Percent proc freq data=h.bmi;table sex;format sex sexfmt.;run; Gender of SP Cumulative Cumulative sex Frequency Percent Frequency Percent MALE FEMALE

18 Formats can clump values, in addition to handling single values proc format; value bmif 0-<18.5="Under" 18.5-<25.0="Normal" 25-<30.0="Over" 30-high="Obese" other="unknown"; run; 18

19 Range notation conventions 1-8 includes 1 and 8 1-<8 includes 1 but not 8 1<-8 includes 8 but not 1 1<-<8 includes neither 1 nor 8 low means the smallest value (excluding missing) high other means the largest value means values not mentioned (including missing, if it hasn t been mentioned) 19

20 Clumping Data in a Printed Report Used in PROCs that display and those that analyze Remember: the underlying data remain unchanged from their native state 20

21 proc print data=h.bmi (obs=10); var baseid bmi; run; Obs baseid bmi proc print data=h.bmi (obs=10); var baseid bmi; format bmi bmif.; run; Obs baseid bmi Over Normal Over Over Obese Unknown Obese Normal Under Obese Native values What you see 21

22 Formats that clump values can help you summarize data Applicable to PROCs that analyze data Characteristics of the output variable you get one record per data clump the formatted variable will retain the characteristics it had in the input dataset the native value of the variable will be the lowest native value for the data clump in the input dataset* the format used to clump the data will be associated with the variable in the output dataset * NOTE: This does not refer to the results of a DATA step in such a case the underlying value of the variable in each record is transferred intact 22

23 Notice how the format statement clumps data together in the FREQ 23

24 Clumping data in an output dataset proc summary data=h.bmi nway missing; class sex bmi; format sex sexfmt. bmi bmif.; output out=bmi_summ (drop=_type); run; proc print data=bmi_summ; format sex bmi; run; proc print data=bmi_summ; run; 24

25 This This turns turns the the formats off off proc print data=bmi_summ; format sex bmi; run; proc summary data=h.bmi nway missing; class sex bmi; format sex sexf. bmi bmif.; output out=bmi_summ (drop=_type_); run; proc print data=bmi_summ; run; Obs sex bmi _FREQ_ Obs sex bmi _FREQ_ 1 MALE Unknown 35 2 MALE Under MALE Normal MALE Over MALE Obese FEMALE Unknown 45 7 FEMALE Under FEMALE Normal FEMALE Over FEMALE Obese 2004 Native values What is displayed automatically 25

26 Use the MULTILABEL / MLF format option to get overlapping ranges This allows you to use overlapping ranges For example, we can look at average bmi over different age groups all at once. N.b.: This does NOT work with PROC FREQ proc format; value magef (multilabel) 0-64="1. Disabled" 0-44="1a. Under 45" 45-64="1b " 65-high="2. Aged" 65-74="2a " 75-84="2b " 85-high="2c. 85+"; run; proc means data=h.bmi print mean order=formatted; var bmi; class age /mlf; format age magef.; run; 26

27 MULTILABEL tells PROC FORMAT not to freak over overlapping ranges / MLF tells the PROC MEANS that overlapping ranges are involved 27

28 So... how do you know what formats you have? And what s in them? 28

29 Here we have specifically allocated h and library, which are added to the libraries SAS opens automatically. 29

30 Notice how creating a temporary format results in SAS making a subfolder within work called formats 30

31 PROC CATALOG commands ask SAS to list the formats in the library (here the permanent library) 31

32 The FMTLIB option in PROC FORMAT asks SAS to show the contents of the selected formats (notice: no period at the end of the format name!) 32

33 Mapping values from old variables to new variables There are 4 ways to do this in SAS IF...THEN...ELSE within a single dataset SELECT... END within a single dataset MERGE values from one dataset onto another Direct mapping using a SAS format within a single dataset 33

34 Mapping values from old variables to new variables using IF...THEN...ELSE Nice Nice if if there there are are a few few values, values, and and you you do do see see exactly exactly what what you re you re doing doing data bmi; set h.bmi; length cat $8; if sex=. then cat= Unknown ; else if sex=1 then cat= Male ; else cat= Female ; run; 34

35 Mapping values from old variables to new variables using SELECT...END data bmi; set h.bmi; length cat $8; select(sex); when(1) cat= Male ; when(2) cat= Female ; otherwise cat= Unknown ; end; run; Not Not so so much much typing typing as as with with the the IF IF chain; chain; notice notice that that you you need need an an OTHERWISE (which (which can can be be null) null) and and an an END END 35

36 Mapping values from old variables to new variables using Merge Previously... data sex_nm; infile cards; length cat $8; input sex cat; cards; 1 Male 2 Female ; run; proc sort data=h.bmi out=bmi; by sex; run; Useful Useful for for data bmi; of different merge bmi (in=a) sex_nm(in=b); by sex; if a; if not b then cat= Unknown ; run; bringing in in lots lots of different attributes at at the the same same time time 36

37 Mapping values from old variables to new variables using the put() function and a SAS format Previously... proc format; value sexf 1="Male 2="Female other= Unknown ; run; data bmi; set h.bmi; length cat $8; cat=put(sex,sexf.); run; Really Really terse, terse, but but the the format format values values come come from from a different part part of of the the program, which which requires user user diligence 37

38 All four roads lead to the same end, so which road should you choose? Lots of variables to add, and not so many unsorted observations? Try the merge Lots of sorted observations? Try the merge Lots of observations, but not many output values? Try the if...then...else or select Lots of unsorted observations, and lots of possible values? Try the format 38

39 Creating a SAS format from a dataset Verrrrry handy if you have a big format to create and the values are already electronic Dataset requires 3 variables with specific names: FMTNAME is a character variable that has the format name (e.g., SEXF ) START is a numeric or character variable (depending upon the format type) that has the raw value (e.g., 1) LABEL is a character variable that has the formatted value (e.g., Male ) Other variables can be used as well

40 To see how to create a SAS format from a dataset, let s dissect the work format library and see the variables involved 40

41 Creating a SAS format from a new dataset data good_golly; infile cards missover end=eof; retain fmtname "newbmif"; length label $8 eexcl hlo $1; input start end eexcl label; if start=. and end=. then hlo="o"; else if start=. then hlo="l"; else if end=. then hlo="h"; keep start end eexcl hlo label fmtname; cards; Y Underwgt Capital Y Normal O (not Y Overwgt zero) N Obese.. N Unknown ; proc format cntlin=good_golly;run; FMTNAME must contain a valid SAS format name alphanumeric begin and end with a letter start with $ for a character format If the variable to be formatted is continuous, each successive START should equal the previous END and EEXCL (or SEXCL) set to Y (Notice that I did not use sexcl or other formatting tools... this was just because of the nature of the format) 41

42 Good golly! It worked! 42

43 Example of creating a SAS format from an existing dataset The task Associate a drug name and an average wholesale price with records on a file summarizing drug events The key 11-digit National Drug Code (NDC) uniquely identifies the manufacturer, drug, form, strength, package size The data: Plan-submitted data from the discount card program (n=8.7 million) Each record is unique for NDC, plan, service type, and date The dictionary: FirstDataBank National Drug Data File (n=118,000) Each record is unique for NDC 43

44 First Step: Create a format to map drug name onto NDC 44

45 Next, create a format to map AWP onto NDC (note the use of the SAS best12 format) 45

46 Use PROC FORMAT to put the formats into the work library Apply the format to the NDC to get a drug name Use the awpf format to get a text version of the AWP, and then the SAS input() function and best12 format to turn it into a number 46

47 47

48 Thanks for Coming Today! Questions? Comments? Overwhelming urge to shower me with money? Contact Dan Waldo