17. Reading free-format data. GIORGIO RUSSOLILLO - Cours de prépara)on à la cer)fica)on SAS «Base Programming» 386

Transcription

1 17. Reading free-format data 386

2 Reading free format data: The list input A raw dataset is free-format when it is not arranged in fixed fields. -> Fields are separated by a delimiter List input allows reading standard and non-standard data DATA SAS-dataset(s) ; INFILE file-specifica+on <op+ons> ; INPUT variable <$> ; - In list input you dont need to specify startcol and endcol - you indicate a delimiter (Default = ) for separa)ng the fields Because list input does not specify column loca)ons, - All fields must be separated by at least one blank or another delimiter - Fields must be read in order from le to right - You cannot skip or re-read fields 387

3 The op)on DLM= DLM is an op)on of the INFILE statement: INFILE file-specifica+on DLM= delimiter(s); delimiter(s) can be: - a list of (up to 200) characters (inclosed in quota)on marks) to read as delimiters. - The delimiter must NOT be a character that occurs in a data value - a character variable whose value become the delimiter DELIMITER= is an alias for DLM= FILENAME import "\\psf\home\documents\my SAS Files\ReadingRawData\Freefinput.txt"; DATA imp_freefinput; INFILE import DLM= ","; INPUT Gender $ Age Height Years; PROC PRINT DATA = imp_freefinput; 388

4 Specifying a list of variables When using List input, you can specify a range of variables in the INPUT statement. If you specify a range of character variables both the variables names and the $ symbol must be enclosed in parentheses. E.g.: - INPUT Gio1-Gio3; - INPUT (Gio1-Gio3) ($); 389

5 Reading missing values at the end of a record MISSOVER is an op)on of the INFILE statement: INFILE file-specifica+on MISSOVER ; - The MISSOVER op)on prevents SAS from reading the next record if, when using list input, it does not find values in the current line for all the INPUT statement variables. - At the end of the current record, values that are expected but not found are set to missing. 390

6 Reading missing data at the beginning or middle of a record DSD is an op)on of the INFILE statement: INFILE file-specifica+on DSD ; The DSD op)on is used to read raw data when there is a missing value in the middle of a record The Delimiter Sensi)ve Data (DSD) op)on: - Treats two consecu)ve delimiters as a missing value - Removes quota)on marks from values - NB: It sets the default delimiter to a comma* *If the data uses mul)ple delimiters or a single delimiter other that a comma, simply specify delimiter values(s) in the DLM= op)on The DSD op)on can also be used to read raw data when there is a missing value at the beginning of a record, as long as a delimiter precedes the first value in the record 391

7 Example: MISSOVER and DSD (1) FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\FreefMDinput.txt'; DATA imp_freefmdinput; INFILE import; INPUT Gender $ Age Height Years; PROC PRINT data=imp_freefmdinput; SAS misinterprets the the second line and it cannot read the third we need MISSOVER FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\FreefMDinput.txt'; DATA imp_freefmdinput; INFILE import MISSOVER ; INPUT Gender $ Age Height Years; PROC PRINT data=imp_freefmdinput; Problem at the fi h line we need DSD 392

8 Example: MISSOVER and DSD (2) DATA imp_freefmdinput; INFILE import MISSOVER DSD; INPUT Gender $ Age Height Years; PROC PRINT DATA = imp_freefmdinput; The default delimiter when using DSD is comma DATA imp_freefmdinput; INFILE import MISSOVER DSD DLM=" "; INPUT Gender $ Age Height Years; PROC PRINT DATA = imp_freefmdinput;.. we need DLM= op)on! 393

9 The LENGTH statement - When using list input, character variables are assigned a default length of 8 - If they are longer than 8, they are truncated - You can avoid trunca)ng if you add a LENGTH statement before the input statement FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\Lengthlis)nput.txt'; DATA imp_lengthlis)nput; INFILE import OBS=3; NB: OBS= and FIRSTOBS= work also in INFILE statement LENGTH City $ 12; INPUT City Pop86; PROC PRINT data=imp_lengthlis)nput; N.B.: - Using LENGTH statement, you do not need to specify variable s type in the INPUT statement. However, leaving the $ in the INPUT statement will not produce an error - A variable that is defined in a LENGTH statement will appear first in the dataset, since it precedes the INPUT statement 394

10 Modified list input: Reading values containing embedded blanks - The ampersand (&) modifier enables you to read character values that contain single embedded blanks. - The value is read un)l two or more consecubve blanks are encountered - NO other delimiter can be used to indicate the end of each field FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\Lengthlis)nput.txt'; DATA imp_lengthlis)nput; INFILE import; LENGTH City $ 12; INPUT City & Pop86; PROC PRINT data=imp_lengthlis)nput; You can also read the values for City with the & modifier followed the $w. informat, which determines the variable length (it should be large enough to accomodate the longest value): DATA imp_lengthlis)nput; INFILE import; INPUT City & $12. Pop86; 395

11 Modified list input: Reading nonstandard values INPUT variable : informat ; - The colon (:) modifier enables you to read - nonstandard values - character values that are longer that 8 characters, but which contain no embedded blanks. - The : indicates that values are read un)l a delimiter is encountered, and then an informat is applied - If an informat for reading character values is specified, the w value specifies the variable s length in the SAS dataset (NOT the number of columns to be read in the source file!!), overriding the default length - N.B.: This is different from using a numeric informat with forma{ed input. In forma{ed input, you must specify a w value in order to indicate the number of columns of the field to read 396

12 Reading non standard values : an example Impor)ng the dataset Modlis)nput: FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\Modlis)nput.txt'; DATA imp_modlis)nput; INFILE import; INPUT Rank City & $12. Pop86 : COMMA. ; PROC PRINT data=imp_modlis)nput; Impor)ng the dataset Modlis)nput2: FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\Modlis)nput2.txt'; DATA imp_modlis)nput2; INFILE import DSD ; INPUT Rank City : $12. Pop86 : COMMA. ; PROC PRINT data=imp_modlis)nput2; 397

13 Mixing input styles: an example FILENAME import '\\psf\home\documents\my SAS Files\ReadingRawData\Mixinput.txt'; DATA imp_mixinput; INFILE import ; INPUT SSN $ HireDate Salary COMMA6. Dept : $9. Phone $ ; PROC PRINT data=imp_mixinput; Field DescripBon StarBng Column Field Width Data type Input Style SSN 1 11 character column Date of Hire 13 7 date forma{ed Annual Salary 21 6 numeric forma{ed Department 28 5 to 9 character list Phone Extension? 4 character list 398

14 Crea)ng Free-format Data: two alterna)ves FILE file-specifica+on <DLM= delimiter other-op+ons> ; PUT variable <: format> ; - variable : name of the variable whose value is wri{en - : : precedes format - format : specifies the format to use for wri)ng the data values PROC EXPORT DATA = SAS-dataset ; OUTFILE= filename <DLM= delimiter > ; RUN ; - SAS-dataset : name of the SAS-dataset to export - filename : the complete path and filename of the output - delimiter : specifies the delimiter to separate columns of data in the output file 399

15 Crea)ng Free-format Data: specifying a delimiter FILE file-specifica+on <DLM= delimiter other-op+ons> ; You can use the DLM = op)on with a FILE statement to create character-delimiter raw data flies PROC PRINT DATA=sasuser.finance; DATA _NULL_; SET sasuser.finance; FILE "\\psf\home\documents\my SAS Files\myoutputs\finance" DLM = "," ; PUT SSN Name Salary : COMMA. Date : DATE9.; 400

16 Crea)ng Free-format Data: using the DSD op)on FILE file-specifica+on <DLM= delimiter other-op+ons> ; Let s suppose you want to create a comma-delimited file in which there are values with a format requiring commas. The file finance may be misunderstood if the above program is used! You can use the DSD op)on in the FILE statement to specify that values containing commas should be enclosed in quota)on marks DATA _NULL_; SET sasuser.finance; FILE "\\psf\home\documents\my SAS Files\myoutputs\finance2" DSD ; PUT SSN Name Salary : COMMA. Date : DATE9.; NB: the DSD op)on uses a comma as a delimiter, so the DLM = op)on is not necessary here 401

17 Reading values that contain delimiters within a quoted string You can use the DSD op)on in an INFILE statement to read values that contain delimiters within a quoted string: FILENAME import "\\psf\home\documents\my SAS Files\myoutputs\finance2"; DATA work.finance2; INFILE import DSD; INPUT SSN : $11. Name $ Salary : COMMA. Date : DATE9. ; PROC PRINT DATA = finance2; FORMAT Date DATE9. ; 402