The data step allows for creation, assembly and modification of SAS data sets.

Transcription

1

2 The data step allows for creation, assembly and modification of SAS data sets. Sources of information include existing SAS data sets, database files, spreadsheets and other raw data files. Like a procedure, a data step is a block of code that opens with the data statement and is typically closed with the run statement.

3 Many different statements are permitted in a data step. data name(s); SAS statements The named data set(s) will be created by the data step. The set statement allows current SAS data set(s) to be selected for use. General assignment statements can be used to create new variables. Format and label statements can be used to set or change defaults for the data set.

4 For the following data step: data new; cla=salary*0.015; A data set called new will be created in the work library: It will be derived from the employees data set from the mysas library It will contain one additional variable, cla computed as shown. How does this occur?

5 Two phases, compilation and execution. During compilation: data new; cla=salary*0.015;

6 Two phases, compilation and execution. During compilation: data new; cla=salary*0.015; Sets up output data set

7 data new; cla=salary*0.015; Gathers all information on variables in this data set Division HireDate LastName FirstName Phone EmpID JobCode Salary And creates a working space with information on all of these variables, including types, labels and formats Known as the Program Data Vector (PDV)

8 data new; cla=salary*0.015; Scans other statements for variable definitions Always of the form: variable = expression; Division HireDate LastName FirstName Phone EmpID JobCode Salary

9 data new; cla=salary*0.015; Scans other statements for variable definitions Division HireDate LastName FirstName Phone EmpID JobCode Salary cla Adds any it finds to the PDV Since the definition of cla involves a numeric expression on the right side of the =, it is defined as numeric

10 data new; cla=salary*0.015; Once all statements are scanned and complied, execution starts. Division HireDate LastName FirstName Phone EmpID JobCode Salary cla

11 data new; cla=salary*0.015; Read observations (starting from 1 st ) from this data set Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25,000. PDV is populated with information from first record, other variables are missing

12 data new; cla=salary*0.015; Process other statements in order. Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25, cla is computed and placed in its location in the PDV

13 data new; cla=salary*0.015; When the bottom of the data step is reached, two things happen... The current PDV is output as a record to the data set being created Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25, work.new Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25,

14 data new; cla=salary*0.015; Automatic return to read next record Variables not present in data are reset to missing by default. Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FIN 19DEC1983 BOWER EILEEN A E00002 FINCLK $27,000. work.new Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25,

15 data new; cla=salary*0.015; And the process continues like before Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FIN 19DEC1983 BOWER EILEEN A E00002 FINCLK $27, work.new Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25,

16 data new; cla=salary*0.015; With the implicit output and return occurring each time, until all records are read Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FIN 19DEC1983 BOWER EILEEN A E00002 FINCLK $27, work.new Division HireDate LastName FirstName Phone EmpID JobCode Salary cla FLI 11MAR1992 MILLS DOROTHY E 2380 E00001 FLTAT3 $25, FIN 19DEC1983 BOWER EILEEN A E00002 FINCLK $27,

17 Here is the table view of the resulting data set:

18 Order of certain statements can make a significant difference in a data step, consider: data newa; cla=salary*0.015; new_salary=salary+cla; vs. data newb; new_salary=salary+cla; cla=salary*0.015; One of these does what you would like

19 Run PROC CONTENTS on this data set and the original: Formats and labels will be the same as those from the original data set, new variables will have none Format and label statements can be made in a data step to set/reset defaults.

20 Consider the following modification of the previous: data new; cla=salary*0.015; format cla dollar8.; label cla='cost of Living Adjustment';

21 Consider the following modification of the previous: data new; cla=salary*0.015; format cla dollar8.; label cla='cost of Living Adjustment'; These attributes are now associated with the cla variable in the PDV at compilation. (They are inactive during execution)

22 Consider the following modification of the previous: data new; cla=salary*0.015; format cla dollar8.; label cla='cost of Living Adjustment'; These attributes are now associated with the cla variable in the PDV at compilation. (They are inactive during execution) The location of the format and label statements in the data step generally makes little difference.

23

24 Were it written like this, there is a slight difference in the result: data new2; format cla dollar8.; label cla='cost of Living Adjustment'; cla=salary*0.015;

25 Why do you think this column is first this time?

26 Not limited to working with only the new variables: data new3; cla=salary*0.015; format cla dollar8. hiredate mmddyy10.; label cla='cost of Living Adjustment' empid='employee ID';

27 Not limited to working with only the new variables: data new3; cla=salary*0.015; format cla dollar8. hiredate mmddyy10.; label cla='cost of Living Adjustment' empid='employee ID'; These attributes are now associated with the corresponding variables in the PDV at compilation, overriding the previous.

28 Old format and label are replaced.

29 Many functions are available to you in the data step, including character functions. PROPCASE(argument) Proper case. Converts all letters to lowercase. Converts the first letter and each letter after a delimiter (space, slash, tab, etc) to uppercase.

30 Try: data new4; cla=salary*0.015; lastname=propcase(lastname); firstname=propcase(firstname); format cla dollar8. hiredate mmddyy10.; label cla='cost of Living Adjustment' empid='employee ID';

31 Try: data new4; cla=salary*0.015; lastname=propcase(lastname); firstname=propcase(firstname); format cla dollar8. hiredate mmddyy10.; Same variable is used on both sides of assignment. Perfectly legal, current value is used in expression on right and then becomes the replacement value. label cla='cost of Living Adjustment' empid='employee ID';

32 Values have been converted.

33 Several character values can be assembled into one via concatenation CAT(argument1, argument2, ) Puts values together in listed order, can be variables or literal (quoted) references. CATS (argument1, argument2, ) Same as CAT except leading and trailing blanks are removed before concatenation. CATX(delimiter, argument1, argument2, ) Like CATS, but the specified delimiter (first item listed) is placed between all values.

34 Consider: data new5; lastname=propcase(lastname); firstname=propcase(firstname); name1=cat(firstname, lastname); name2=cats(firstname, lastname); name3=catx(' ', firstname, lastname); name4=catx(', ', lastname, firstname);

35

36 Consider: Since lengths vary, extra space is data new5; preserved. lastname=propcase(lastname); firstname=propcase(firstname); name1=cat(firstname, lastname); name2=cats(firstname, lastname); name3=catx(' ', firstname, lastname); name4=catx(', ', lastname, firstname);

37 Consider: Trimming all the spaces is not data new5; quite optimal for this example. lastname=propcase(lastname); firstname=propcase(firstname); name1=cat(firstname, lastname); name2=cats(firstname, lastname); name3=catx(' ', firstname, lastname); name4=catx(', ', lastname, firstname);

38 Consider: The delimiter in CATX allows me data new5; to fix the previous problem. lastname=propcase(lastname); firstname=propcase(firstname); name1=cat(firstname, lastname); name2=cats(firstname, lastname); name3=catx(' ', firstname, lastname); name4=catx(', ', lastname, firstname);

39 Consider: I might consider this form as well, note the importance of the space data new5; in,. lastname=propcase(lastname); firstname=propcase(firstname); name1=cat(firstname, lastname); name2=cats(firstname, lastname); name3=catx(' ', firstname, lastname); name4=catx(', ', lastname, firstname);

40 For whichever is chosen, the previous can be made a bit simpler by nesting functions: data new6; name=catx(' ',propcase(firstname),propcase(lastname)); Proper casing is applied to each of first name and last before the concatenation function is applied.

41 It may seem redundant to have both firstname and lastname in the data set with the new variable name.

42 Variables can be selected to be dropped or kept in two basic ways, via a drop/keep statement or via a drop/keep option. DROP/KEEP statement Legal only in the data step Syntax: drop variable1 variable2 ; keep variable1 variable2 ; Only one of the two should be used in any one data step.

43 How does it work? Consider: data new6; name=catx(' ',propcase(firstname),propcase(lastname)); drop lastname firstname; At compilation

44 data new6; The PDV will include a flag to drop these variables when records are output. name=catx(' ',propcase(firstname),propcase(lastname)); drop lastname firstname; Drop Drop Division HireDate LastName FirstName Phone EmpID JobCode Salary name

45 By default, all variables from input data new6; data sets and all created with expressions are flagged as keep. name=catx(' ',propcase(firstname),propcase(lastname)); drop lastname firstname; Keep Keep Drop Drop Keep Keep Keep Keep Keep Division HireDate LastName FirstName Phone EmpID JobCode Salary name

46 If a keep statement is used, only data new7; those listed are flagged to be kept, all others get a drop flag. name=catx(' ',propcase(firstname),propcase(lastname)); keep name empid division jobcode salary; Keep Drop Drop Drop Drop Keep Keep Keep Keep Division HireDate LastName FirstName Phone EmpID JobCode Salary name

47 Either way, all variables are available during processing for computational purposes, keeping/dropping only applies at output. data new7; name=catx(' ',propcase(firstname),propcase(lastname)); keep name empid division jobcode salary; Keep Drop Drop Drop Drop Keep Keep Keep Keep Division HireDate LastName FirstName Phone EmpID JobCode Salary name

48 DROP/KEEP option Legal nearly anywhere a data set is selected. Syntax: dataset(drop=variable1 variable2 ) dataset(keep=variable1 variable2 ) Again, only one of the two should be used with any one data set.

49 So this is equivalent to the previous use of the drop statement: data new8(drop=lastname firstname); name=catx(' ',propcase(firstname),propcase(lastname));

50 This is not: data new9; set mysas.employees(drop=lastname firstname); name=catx(' ',propcase(firstname),propcase(lastname)); Check the log and resulting data set

51 Nothing

52 The names never existed as far as this data step is concerned they were not in the PDV because they were dropped during input. The unknown variables are presumed to be numeric hence the two dots.

53 Typically, in the data step, using the keep/drop statement is easiest. When using the option in other procedures, make sure you are aware of the difference in using them during input versus output.

54 Suppose for the projects data, the jobtotal was incorrectly computed. Rather than the sum of equipment and personnel costs, there needed to be an accounting for incidental costs, which were 3% on personnel 1.5% on equipment Produce the following modified data set:

55

56 Insert some reasonable formats and labels to produce:

57 Create single latitude and longitude variables for the fish data