PROC FORMAT Jack Shoemaker Real Decisions Corporation

Transcription

1 140 Beginning Tutorials PROC FORMAT Jack Shoemaker Real Decisions Corporation Abstract: Although SAS stores and processes data intemally as either characters or numbers, you can control the external view of the data using explicit formats. SAS provides a generous supply of general purpose formats for displaying cbaracter and numeric data. Often you need a format specificauy tuned to the values in your data. PROC FORMAT allows you to define and maintain these user-defined formats. In addition to controlling the external view of data, you can use PROC FORMAT to create look-up tables which often eliminate the need for costly merges or extensive if-then-else coding. This paper discusses these uses of PROC FORMAT; how to maintain permanent user libraries; and, how to create PROC FORMAT value clauses from. existing SAS data sets. Review any report generated by Proc Contents. You'll discover that SAS stores all variables in either numeric or cbaracter format. Character variables may vary in length from one to two hundred characters, or bytes. By default SAS uses eight bytes to store numeric variables. That doesn't mean that you can only represent numbers up to 99,999,999 using SAS numeric variables. Rather, the eight bytes (or sixty-four bits) are used to hold the sign, mantissa, and exponent that represent the number. This is the stuff of computer science and not the subject of this paper. The point is that without some sort of transformation, numeric variables would look pretty silly when displayed. When you ask SAS to display a numeric variable it uses the BEST format to transform the internal representation of the number into a meaningful display of digits. This process is so natural and common-sense that you might forget it bappens. Not all numeric variables make sense even when using the BEST format. Today's date (10/06/92), for example, has a numeric value of 11,967 - the number of days since Using the BEST format, today would appear as 119(j7. To make SAS display 11,967 as 10/06/92, you associate the MMDDYYS. format with the variable holding today's date. If SAMPDATE is the variable name, then you could place the following format statement in either the data step which creates the variable, or the procedure which displays the variable: format saapdate -aclyy8. J This is an example of using a SAS-supplied format. SAS provides a score or so formats to display dates and times, and otherwise punctuate the data display. Even with all these formats available, you can easily run into a situation where none of the SAS-supplied formats do what you want. Proc Format provides the means to create user-defined display formats. Once created, you use user-defined formats exactly like SAS-supplied formats. Here is an example of using Proc Format to define a PICTURE format which will punctuate a numeric variable to make it look like a social security number; and, of using the user-defined format in a Proc Print. proc formatj picture ssn other = ' '; proc print; format myssn ssn.; This simple example illustrates the major components of the Proc Format grammar. Proc Format is called with a PROC statement just like any other SAS procedure. You can place the format definition anywhere in your source code; however, it must appear before you want to use it. Proc Format also recognizes a handful of options which we'll discuss later in this paper. The PICTURE keyword on the second line begins a clause definition. You can define three types of clauses using Proc Format: PICTURE clauses, VALUE clauses, and INV ALUE clauses. Immediately following the clause keyword comes the clause name, in this example SSN. You can choose any clause name subject to these restrictions: 1) the name may not exceed eight characters; and, 2) the name may not begin, or end with a number. (In early versions of SAS under MS-DOS, embedded numbers in clause names produced undesirable results.) Following the clause name comes the range definition. In this case the keyword OTHER defines the range. OTHER means all other values not previously defined. Since no other ranges appear in this example, OTHER means all values. After the range definition comes an equals sign. To the NESUG '92 Proceedings

2 Beginning Tutorials 141 right of the equals sign comes the formatted-value definition, in this case ' '. Clause definitions end with a semicolon. In PICTURE clauses, the formatted-value definition tells SAS how to place the number in a mask. In this example our mask is: three digits, dash, two digits, dash, four digits. The use of '9' in the formatted-value definition tells SAS to always display a digit, even if it's a leading zero. For example: SSB You can also define a PICTURE clause that will suppress leading zeroes. For example, here is a PICTURE definition that you could use to format paid amounts on checks. procl format; pioture chex other = '000,009.99' (fill='*' prefix='$'); COX *$4, *****$0.16 In this definition, the zeroes in the mask indicate that leading zeroes will not be displayed. In addition to a formatted-value mask, this picture definition contains two formatted-value modifiers. The FILL= modifier causes SAS to left fill the field with the specified character, in this case an asterisk. The PREFIX= modifier causes SAS to prepend the specified character, in this case a dollar sign, to the field. PICTURE clauses create templates for displaying numeric variables. You can also use Proc Format to define a format which displays a variable with a different value. A VALUE clause definition is used to build this type of format. For example, consider a data set of survey information where responses are coded 1 for 'Yes', 2 for"no', and 9 for 'None'. Here is an example of using Proc Format to define a V ALOE clause to map the coded responses into words; and, of using the userdefined format with Proc Freq to display the frequency of each response. value resp 1 = 'Yes' 2 = 'Ho' 9 = 'Hone'; proc freq; format response resp.; Using the user-defined format RESP. causes Proc Freq to label the rows of its output Yes, No, and None instead of l, 2, and 9. If a value other than l, 2, or 9 appeared in the survey data set, it would appear unchanged because no range definition applies. The V ALOE clause shown above has three pairs of range-definition and formatted-value. There is no practical limit to the number of pairs contained in a V ALOE clause provided the range definitions don't overlap. The preceding example uses a single value as the range definition. You can also use ranges of values, lists of values, lists of ranges, or a combination lists and ranges in range definitions. Here are some examples of the different types of range definitions: 1-9 1,2,3 1-9, ,2,8-15 range of values list of values list of ranges combination Range definitions may also contain the keywords OTHER, LOW, or mgh. So, LOW-lO means all values 10 and below, while 50-mGH means all values 50 and over. As mentioned earlier, the keyword OTHER means all values not previously defined. As an example of using ranges of values, consider a data set of accounts receivable information containing the amount due and the number of days elapsed since the due date. You'd like to see how much due is less than thirty days old, how much due is thirty to sixty days old, and how much due is over sixty days old. Here is an example of using Proc Format to define a V ALOE clause to define these ranges; and, of using the user-defined format with Proc Means to total the due amounts by age category. proo format; value ar low -< 30 = 'Under 30' = '30 to 60' 60 <- high = 'Over 60'; NESUG '92 Proceedings

3 142 Beginning Tutorials proc _ans n SUlll; class elapsed; var due; format elapsed ar.; The AR. format will cause Proc Means to group values of the classification variable, ELAPSED in this example, according to the formatted values. The output will have at most three rows because all possible values of ELAPSED will fall within one of the three defined ranges. Notice that the range definition for 'Under 30' contains a less-than sign. This modifier instructs SAS to exclude the upper value, 30 in this case, from the range definition. Similarly, the less than sign in the range definition for- 'Over 60' instructs SAS to exclude the lower value, 60, from the range definition. So far we've only discussed numeric formats - that is, formats associated with numeric variables. You can use Proc Format to associate formats with character variables as well. These are known as character formats. Here is an example of using Pmc Format to define a character format which maps the sex codes 'M' and 'F' to 'Male' and 'Female' respectively. value $sex 'f', 'F' = 'Female' '.' I'M' = 'Male' other = 'Unknown'; All character format names must begin with a dollar sign. Therefore, the format name for this VALUE clause begins with a dollar sign. Notice that the range definition for 'Female' is a list of character values - lower and upper case 'r. Range definitions for character formats follow the same rules as range definitions for numeric formats. That is, the range definition may consist of a single value, a list of values, a range of values, a list of ranges, or some combination of these elements. Under releases of SAS prior to 6.07, character values could not exceed sixteen characters. Or, more precisely, SAS truncated the character values at sixteen characters. Under version 6.07, character values may extend to two hundred characters. Also, in releases prior to 6.07 the formatted value could not exceed forty characters. Under 6.07, the formatted value may have two hundred characters. Character ranges operate in a natural way. That is, 'A' - 'Z' means all capital letters between A and Z. Even so, the way SAS handles a particular character range depends on the collating sequence in use. This means that shipping a format definition from a machine using ASCII collating to one using EBCDIC collating may produce undesirable effects. The same holds true if you are moving code between hosts in different countries which use different alphabets. Because you can use a user-detined format to change an internal value of 'M' to an external display of 'Male', you don't need to store the values of 'Male' and 'Female' in your data set. Rather, a single 'M' or 'F' will suffice. In general, user-defined formats give you the freedom to reduce redundant character values to shorter coded values thus reducing overall data storage requirements. Since formats provide a way to transform the internal value of variable to a different external display, the format can serve as a simple lookup table in a data step through the use of the PUT function. For example, you can use a user-defined format to convert a two-letter state abbreviation to a region name like this: value $region 'ME', 'DB', 'VT', 'MA', 'cr', 'RI' = 'New England' other = 'Elsewhere'; region = put(statecd,$region.); Remember: the PUT function always returns a character string. In this case the character string is assigned to a variable called REGION which is presumably a character variable. This doesn't mean you can't use the combination ofproc Format and the PUT function to do numeric lookups. For example, consider a health insurance claims data set which has a date of service field, DOS, and a paid amount, PAID. You would like to adjust all these paid amounts by a factor which is a function of the date of service. You could code a cascading IF-THEN-ELSE tree to accomplish this. if dos < 'OlAUG92'D then fac = 1.00; else if dos < 'OlSEP92'D then fac = 1.20; else fac = 1.50; paid = paid * fac; NESUG 192 Proceedings

4 Beginning Tutorials 143 Using Proc Format and the PUT function, your code would look like this: value fudge low -< 'OlAUG92'D = '1.00' 'OIAUG92'D -< 'OISEP92'D = '1.20' other = '1.50'; paid = paid * put(doa,fudge.); By using the latter method, you remove the run-time data from your source code, which, in tum, makes your code easier to maintain. That is, as the factors change, the first method requires modifying a set ofip-then ELSE statements in the source code to reflect the new factor data. The second method removes the factor data from your code and places it in a VALUE clause definition instead. Since the result of the PUT function is part of an arithmetic statement, SAS automatically converts the result to a numeric value to perform the multiplication. The PUT function also serves a useful role in subsetting IF statements. For example, using the $REGION. format previously defined, we can subset a master data set to contain only New England states as follows: data ne; set master, if put(statecd,$region.) = ']lew England'; Using a WHERE statement with the IN operator would provide a better solution for this trivial example. where atatecd in ('ME', 'VT' I 'lib', 'MA', 'CT' I 'RI'); However, as the length of the list grows the WHERE statement becomes more troublesome to code. Imagine you want to select observations from our health care claims data set based on the provider identifier. Another data set, call it WANTED, contains the thousand, or so, provider identifiers that you want. One solution to this problem is implementing a null merge. That is, merge the master data set against the WANTED data set by provider identifier and only keep those observations that are in both data sets. proc sort data = master;by provid; proc sort data = wanted;by provid; data select; merge master(in=mast) wanted(in=yep); by provid;if mast & yep; If you had a format available called $WANTED., instead of a data set called WANTED, which returned a value of 'I' for the provider identifiers in the data set WANTED, and '0' otherwise, you could create the data set SELECT as follows: data select; set master; if put(provid,$wanted.) = '1'; This method is far more efficient than the null-merge method. It eliminates the need to sort the MASTER data set and perform a MERGE only to exclude most of the observations. The only problem is that you don't have this format available. Typing the thousand, or so, provider identifiers into a value clause is subject to typographical errors. Fortunately, Proc Format provides a means to create a VALUE clause from a SAS data set by using the CNTUN = option. Proc Format will use the data set specified with the CNTUN = option to construct a VALUE clause provided the data set has at least these three variables: FMTNAME START VALUE the value-clause name the range-begin value the formatted value You can specify ranges by including a variable called END which contains the range-end value. Although Proc Format will infer the type of format from the value of FMTNAME, you can also include a variable called TYPE which has a value of 'C' for character formats, and 'N' for numeric formats. In our current example, we need an 'OTHER' range definition associated with a formatted value of '0'. Proc Format needs to know NESUG '92 proceedings

5 144 Beginning Tutorials about ranges which include keywords like HIGH, LOW, and 0lHER. For these value definitions, the CNTLIN = data set must contain a variable called HLO which takes on values of 'H', 'L', '0'. or some combination of these values. Here is an example of turning the WANTED data set into a $WANTED. format. data table (keep=fmtnaae start value hlo)~ set wanted (rename=(provid=start» end=lastree; retain mtda1118 ' $wanted' value '1'; output; if.lastree then do; start = 'OTHER'; value = '0'; hlo = '0'; output; end; proe format entlin = table; Under version 6.07 of SAS, the HLO variable may take on additional values to qualify the values in the START, END, and VALUE variables. Some of these values reflect enhancements to other parts of the Proc Format grammar available under version In particular, under version 6.07 the formatted value may refer to a SAS-supplied format, or a previously defined userdefined format. For example, consider the accounts receivable data set from a previous example. Let's say it contains a field called PAIDATE which contains the date value of the paid date, or the special SAS missing value.0 if payment is still due. Using the SAS-supplied MMDDYYS. format, SAS would display the following: MMDDYY /07/ Suppose you'd rather display 'Not Paid' in the date field instead of '0'. Here is an example of using Proc Format under version 6.07 to create a user-defined format which uses MMDDYYS. as part of the definition. proe format; value ourdate.0 = 'Rot Paid' other = [mmddyy8.]; Using this format results in the following display: OORDATE /07/92.0 Hot Paid By default, Proc Format places user-defined formats in a SAS catalog file in the WORK library. Consequently, these formats disappear when SAS terminates. Fortunately, Proc Format provides a means to permanently store user-defined formats by using the LIBRARY = option to specify a LIBNAME to store a permanent SAS catalog file. Under version 6.06 of SAS the name of the SAS catalog file is FORMATS. Once a user-defined format has been stored in the permanent catalog file, you can use the user-defined format by specifying a LIBNAME called LIBRARY which points to the SAS library which contains the FORMATS catalog file. For example: s~oring a User-Defined Format libname anyname ' some.where '; proe format library = anyn.. e; Using a User-Defined Format 1ibname library ' some.where '; When SAS encounters a user-defined format, it searches the WORK library for the FORMATS catalog file. Failing that it searches the LIBRARY library for the FORMATS catalog file. If it doesn't find the format in either of these places, SAS issues an error message. Yau would expect SAS to search for user-defined formats when used explicitly in a Proc Print. However, SAS also searches for all formats associated with all variables in a data set when the data set is referenced. So, if you use a FORMAT statement in a data step to associate a user-defined format with a variable, SAS will expect to find that user-defined format whenever the data set is used. If the user-defined format is not available, then SAS will terminate the current data step or procedure with an error status. There are a few solutions to this problem. The best solution is to store user-defined NESUG '92 proceedings

6 Beginning Tutorials 145 formats in a permanent SAS catalog and move the catalog with the data set. If that is not possible, the NOFMTERR system option will suppress all formatrelated error messages. If that seems too severe, and you don't need the variable which has the user-defined format associated with it, you can drop the variable using the data set DROP= option. For eumple: data two; set one(drop=noneed); Under version 6.07 of SAS you can specify a two-level name with the LIBRARY = option. This causes Proc Format to create a SAS catalog named after the second part of the two-level name in the library specified by the first part of the two-level name. For example, this specification creates a catalog named COOL in the library called WAY: proc format library = way.cool; When you create a SAS catalog other than FORMATS to store user-defined formats, you must instruct SAS to search in that catalog by using the FMTSEARCH= system option. You can also use the FMTSEARCH= system option to alter the order in which SAS searches for user-defined formats. For example, this OPTIONS statement causes SAS to search the WAY. COOL catalog, then the LIBRARY. FORMATS catalog, and finally the WORK. FORMATS catalog to find a user-defined format: This paper has only covered the major aspects of Proc Format which would interest a general user. In particular, no examples of the INV ALUE clause appear.!ny ALUE clauses create informats in the same way that VALUE clauses create formats. Nor have we seen any examples of the CNTLOUT = option which creates a SAS data set from a format catalog file. The SAS procedures manual provides a comprehensive review of these and all features ofproc Format. Version 6.07 also contains enhancements to Proc Format which this paper has not covered in detail, or at all. SAS technical report P-222 details all the changes to SAS under version 6.07, including the enhancements to Proc Format. P-222 probably won't make much sense unless you already understand Proc Format as described in the procedures manual. So read the manual first. (RTMF) Software products mentioned in this paper are copyright their respective publishers. Code presented in this paper is copyright the author; however, it may be freely distributed provided the right to redistribute the code is not restricted in any way. The author welcomes comments, questions, and criticisms. Jack Shoemaker Real Decisions Corporation 22 Thomdal Circle Darien, CT shoe@world.std.com options fmtsearcb=(way.cool library work); In alllevei-6 versions of SAS, the FMTIlB option causes SAS to print a report listing the entire contents of the permanent format library specified by the LIBRARY = option. If your format library contains many entries, but you only wish to review only one or two of them, you can use the SELECT or EXCLUDE statements to limit the number of format entries printed when FMTLIB is specified. For example, this code will only show the contents of the $W ANTED. and FUDGE. formats: proc format library=library fmtlib; select $wanted fudge; rudi NESUG 192 Proceedings