The Most Popularly Applied Vehicles in Health Industry for Data Transfer from SAS to Microsoft Excel

Transcription

1 PharmaSUG2010 Paper AD17 The Most Popularly Applied Vehicles in Health Industry for Data Transfer from SAS to Microsoft Excel Guiping Yang IMS Health Inc., Blue Bell, PA ABSTRACT SAS is the most accommodating vehicle in statistical world for transferring data between SAS and other systems. There are many different tunnels available to achieve this function; however, this paper puts its focus on the data flow from SAS to Excel or its close relatives because of their well-known popularity in pharmaceutical and health realm. We briefed totally fourteen tools in this review, covering from the most basic to the relative advanced, from the most popular to the less heard, from the very traditional to the high-level in modern world. These include, but not limited to, Excel Functions, Macro, Dynamic Data Exchange (DDE), Output Delivery System (ODS), extensible Markup Language (XML) and Excel Libname Engine. Both advantages and disadvantages of individual techniques were captured and evaluated for the users convenience. This investigation was not only to spread the knowledge and technique in data transportation, but also to serve as a reference tool for approach selection in some specific environments. Another benefit from this investigation is that, the data conveyance in reverse direction, from Excel to SAS, was also self-illustrated. In some cases, it s very likely that these approaches have their roads crossed; some other file formats, such as Word file, CSV file, Text file and alike, can also be accommodated sometimes. Keywords: SAS, EXCEL, Data Transfer INTRODUCTION In pharmaceutical and healthcare field, the clients are more and more comfortable to play with Excel files during their interactions with the scientists, market investigators, government officials, physicians and practitioners or their own colleagues. The advantages of Excel usage over other file formats are definitely uncountable in the eyes of many users. The basic skills of using Excel file data are easy to be grasped and they are able to handle the tasks in the majority aspects on daily basis. Excel window provides very friendly interface to the users; most menu functions are understandable and its logics behind the screen are not difficult to learn. During the conferences or remote communications, multiple viewers are able to open the same file at the same time and have their own total controls on file management from the different windows on different computers. Just by dragging, clicking or dropping, the summary level information, even the statistical tests would be exhibited flexibly in pivot table from Excel data. This extension from Excel facilitates the higher level 1

2 interpretation on each terminal. Moreover, PivotTable can also help to analyze data, make comparisons, and detect patterns and relationships. Pivot Graph is easy to create through Pivot Table to observe visual trend of the data. It s becoming a very useful tool to communicate with clients in terms of market exploration, product promotion, patient targeting, clinic trail, disease prevention or drug development, to name a few. The skills to export SAS tables into customer-friendly Microsoft Excel or Excel accessible files are largely developed in industries and they are being spread broadly to facilitate the information transfer in pharmaceutical realm. Depending on the version of SAS, the requirements of application, the platform to store data and finally the personal preference, the techniques to use are various and playing their roles in all kinds of environments. Some of options to perform this seemingly easy work are listed in this paper and the most focus will be put on some high-level vehicles. VEHICLES IN APPLICATIONS Here are some methods to be introduced in this paper: METHOD METHOD DESCRIPTION 1 SAS Export Wizard and SAS Display Manager 2 EXCEL Functions 3 SAS Explorer View 4 DATA Step 5 PROC EXPORT 6 SAS MACROs A. Build-in Macros B. Customized Macros 7 Dynamic Data Exchange (DDE) 8 Output Delivery System (ODS) and Markup Language 9 extensible Markup Language (XML) 1) SAS XML Libname Engine (SXLE) 2) ODS XML Tagset 10 EXCEL LIBNAME Engine 11 ODBC in Windows 12 PROC DBLOAD 13 SAS Server Technologies 1) Application Dispatcher 2) Stored Process Service 2

3 14 Enterprise Guide with Excel Method 1: SAS Export Wizard and SAS Display Manager A. This is the most traditional approach in SAS development. It is not necessary for the users to be equipped with much statistical or SAS knowledge. Like any button-clicking menu, this export facility works for SAS users at any levels and easy to pickup. Here are some basic steps for the guideline: 1) Open SAS edit window; 2) Click File and select Export Data, SAS export wizard will pop up; 3) Within Wizard window, choose the library and data you want to export; 4) Identify the type of file for the exported SAS data, data source can be either Standard or customized format; for standard source, some existing formats supported by system are listed for appropriate purpose. For sample, to use options like Microsoft Ecel97, 2000 and 2002 workbook, the SAS/ACCESS to PC File Formats must be licensed. If the customers are willing to populate the data in their own style, the customized data source is more optimal. The user needs to define a special file format using the External File Interface (EFI) facility. 5) Identify the location to save the exported file and name the new created file. This can be browsed through the window. The Microsoft Excel Workbook in the Connect to MS Excel box will be selected in this step. 6) Enter the sheet name for data to exist; 7) Type in the file name to save the PROC EXPORT code in, if desired. This step is optional; 8) Then, if the user chooses to define his specific format, the format window will be opened for customization. Otherwise, the window will be closed. A SAS note will appear in log window to show the location and file name to be created and check if the process is successful. This method is available in most systems as long as SAS/ACCESS to PC file Formats is licensed. The wizard works very well for only simple file types, CSV or tab delimited file won t be able to go through this method. Moreover, the wizard follows the same procedure for any available file types. B. Another way to duplicate the power of Export Wizard is to use the SAS Display Manger command DEXPORT, for example, DM DEXPORT sashelp.retail c:\text\retail.xls replace ; In contrary to its sister command DIMPORT, DEXPORT is used to export the tab delimited files, such as Excel or CSV. It certainly reduces the amount of typing required to export the SAS dataset. Another interesting point is that the DEXPORT command works great in both UNIX and PC. The 3

4 above sample can take care of most general work, if there is no Libname is specified, the dataset from the WORK directory is exported in an excel format to a specified location. SAS Documentation has very little information about how to use this facility as it should be, even searches in Google or other engines are not helpful enough. As the best development and testing environment for SAS, Display Manager marries the fast response of the line-prompt model with the editing and full-screen capabilities of the batch system. It allows the users to work in screens to accomplish editing, submitting codes for processing, reviewing the reports generated and returned without touching explorer window. However, some practical limits allow the users to work only on stored disk and small sample sizes. Method 2: EXCEL Functions If the data is small enough or partial data is to be selected, it is easy to select, copy and past the data directly from SAS into Excel. However, in most time, this can actually be the quickest way to get the SAS listing into Excel. Some editing work may be necessary for passing the data into Excel (Winder, 2004), which requires the familiarity of some Excel functions and their usages. The process for this parsing is listed here for guidance. 1) When the data was copied from SAS output window and pasted into Excel, the data line will stay in one column, column A, for example. OBS number is preserved in such a data move. 2) Remove those extra information, like title, footnote, date and time or page number, which is generated in SAS window automatically; 3) Sorting the data in column A by OBS number will bring all the paired observations together; this is useful when many variables exist where one record would be split into multiple lines. 4) Concatenate multiple lines for one record in use of CONCATENATE function in Excel, for example, for 3 lines split, using =concatenate(a1,a2,a3) ; 5) Then copy and paste the concatenated values in prior step to another spreadsheet, don t copy the format or other things, just value; 6) Split the concatenated text among columns by using text functions, including LEFT, MID, RIGHT, SEARCH, FIND or LEN, to name a few; column breaks can be inserted by FIXED WIDTH ; 7) Sort the data in any one column and remove the extra lines for same concatenated record; 8) Finally, the data fields can be put back as column headers. Method 3: SAS Explorer View This is a new feature from SAS 9.1. There is an option in SAS Explorer window to view SAS dataset in excel format. After right clicking the SAS dataset in Explorer window, a menu list is brought up, which 4

5 includes a new option View in Excel. By selecting this function, the dataset will be sent to an excel file and displayed in spreadsheet. Both file name and sheet name are default and can be renamed by the users. To select some records on certain conditions, where clause is a good approach in the viewer, select Edit/Filter/ from the viewer drop down menu, then you can type in where year=1981 to limit the data to year 1981 only. Method 4: DATA Step DATA step belongs to SAS Base skill, available in all environments; it s quite powerful and flexible, can create and output any types of data files. All column heads need to be specified within DATA step, therefore have to be recognized before being exported to Excel platform. As a result, the certain programming skills are required. Non-SAS users would have some difficulty to deal with the complicated dataset. However, DATA step offers the space for the users to customize the special format. Statements PUT and FILE work together to output data to any delimited files by varying Delimiter option, such as comma delimited CSV file, which can be read in excel window unless the record number is not beyond the row limitation of excel file. It can also output data to Excel directly. The following sample shows how to write out a tab separated file: Data _null_; Retain t 09 x; Set sashelp.retail; File 'C:\Retail.csv'; If _N_=1 then put Sales t Date t Year t Month t Day ; Put Sales t Date t Year t Month t Day; 5

6 Run; It works on any platform, like UNIX or Mainframe. The output file can be opened in excel or other text format. If the delimiter is comma, DSD option can be applied without any need to specify, for all variables. Other delimiters, like Tab, Space or Pipe, are also most popularly used. Method 5: PROC EXPORT This method is not available anywhere, SAS/ACCESS license is required. However, in most cases, it does not have any problem. REPLACE statement delete the old file and replace it with new file in same name. No matter what format the variables have, this procedure read them in exactly, just like copy and paste. If the variable has mixed format for it values, namely both numbers and characteristics are in same column, numbers would be blank. Options missing= m ; Proc export Data=sashelp.retail Outfile='c:\retail.xls' replace; DBMS=excel; Sheet= Sales ; Range= A2:C19 ; Run; DBMS=identifier specifies the type of data to export. To export the PC files, spreadsheets, and delimited external files, you do not have to specify DBMS= if the filename that is specified in OUTFILE= contains a valid extension so that PROC EXPORT can recognize the type of data. Both CSV file and Excel file can be generated in this way. SHEET= Sheet name is to identify a particular spreadsheet to write to in a workbook. The output can also be put on certain area by specifying the range. If the file with same name exists at same location, it will be replaced. By default, the missing values are displayed as the dot or space. If a special character is needed to override the dot or space, just simply define it in an OPTIONS statement. In above sample, the character m will be used to indicate the missing values, which was between single quote marks. Method 6: SAS MACROs Some available Macros stored in system or customized by users provide more convenience to convert SAS files to Excel Workbook with no need in logics and skills behind. Here, some generally used examples were brought up to the users. Among these listed examples, some have very similar functions except the different aspects to emphasize; others then prioritize themselves in certain circumstances. A. Build-in Macros 1) %SSCFLAT 6

7 This macro was written by Systems Seminar Consultants, will run in any environment and can convert a SAS data to a CSV file. The advantage is that the user does not have to recognize the field names first before calling the macro. It can honor most generally used formats by default. The header row will be from variable names or labels. To use this macro, it needs to be defined to the system first. %SSCFLAT (Msasds=sashelp.retail, Mflatout=c:\retail.csv, Mlabel=yes); These three parameters assign input data, output data path and file name, and label. The generated character variables will be double-quoted. 2) %DS2CSV This is a similar macro to %SSCFLAT, imported to SAS 9 or above version; it can convert SAS dataset to comma-separated CSV file. %DS2CSV (Data= sashelp.retail, Runmode=S/B, Csfile=c:\retail.csv); Runmode=B is used when it s run in SAS editor window. The character in CSV file will be doublequoted automatically in the layout. To use this macro, license SAs/IntrNet software is required. 3) %MAKEFILE; This macro is supplied by SAS Institute and was described by Winters (2004) for its basic logics. It generates a CSV file and can accept various user-determined parameters, input data and SAS licenses. The syntax of this macro looks like this: %MAKEFILE (Dataset=sashelp.Retail, Filename= Retail.csv, DLMR=,, Qtes= no, Header= no, Label= no ) Simply speaking, PROC CONTENT and PUT statement in DATA step were combined in usage to create a CSV file. Delimiter can be defined in DLMR parameter, if the quotation of all character variables is determined by QTES= no/yes ; the headline with column names or the label of column headers are also kept into consideration. B. Customized Macros 1) %SAS2XL The macro %SAS2XL proposed by Brown (2005) hybrids the power of ODS TAGSET and XML techniques. Its multi-tabbed workbook can absorb more than one SAS datasets. At the same time, it retains user-friendly features like traffic-light highlights, frozen column headers and more interpretable labels. The syntax of %SAS2XL showed here basically includes all parameters to define the properties of spreadsheet as well as the location to save the file. % SAS2XL( Path=C:\, File=DATAFORVP, 7

8 SALES DATA FOR / "SALES MONTH!EMP/ "SALESPERSON"!SALES/ "SALES TOTAL" W=15 BG=SALESF. = SALES DATA FOR / "SALES MONTH!EMP / "SALESPERSON"!SALES / "SALES TOTAL" W=15 BG=SALESF. To produce an Excel file, this macro goes through such procedures to transfer SAS data: i. Define all properties of a spreadsheet by parsing relevant parameters; ii. Based on requested options parsed above, a custom TAGSET is created for each spreadsheet; iii. To generate HTML files for each spreadsheet by means of custom TAGSET and PROC REPORT procedure; iv. If multi-tab workbook is needed, the file to link individual HTML sheet to a single Excel workbook is created; v. By request, HTML output is converted to an Excel format through the linkage file. 2) %SAS2XLS This macro uses XML library engine to write SAS datasets to an XML file (DelGobbo, 2002). It requires Perl and some Perl modules, both SAS and Perl need be installed on same platform. But, it does not handle formatting issues. The user has to wait for SAS to get off their duff and write an ODS Excel destination. %SAS2XLS( File=, Fileref=, Sheet1=, Sheet2=, Sheet3=, Perl= PERL W, DeleteXML=1); 3) %SAS2CSV This macro was derived from the above %SAS2XLS with same methodology and higher efficiency. The delimiter parameter determines the output file is CSV (Comma) file or TEXT (Tab) file. %SAS2CSV(Sasds=, Outdir=, Outds=, Delimit= ); 8

9 Same as %SAS2XLS, SAS dataset is read in automatically. However, no DDE technique is adopted for %SAS2CSV, which eliminates the waiting time for opening Excel file. The efficiency is further enhanced by absorbing multiple SAS datasets into multiple spreadsheets in one workbook. 4) %SAStoXL Proposed by Vyverman in 2000 and updated one year later, exporting SAS to Excel using DDE. The users do not need to understand the knowledge of DDE but can touch all its powerful futures. However, some options, like cell merge, headers exclusion and Excel formulas addition are not quite feasible. The user can also not take one sheet as the template of other sheets in same workbook. The numeric cells in Excel can not be pre-formatted before the application. %SAStoXL (Libin=, Dsin=, Cell1row=1, Cell1col=1, Nrows=, Ncols=, Tmplpath=, Tmplname=, Sheet=, Savepath=, Savename=, Stdfmting=); 5) %exporttoxl This macro was updated by Derby (2007) to compensate the futures lacking in %SAStoXL and adding more new features. This macro is majorly used in PC SAS on Windows This macro is also preferable to ExcelXP with respect of graphs and side-by-side table creation and pre-formatted worksheet. Anywhere ExcelXP does not work can be a good place to show the power of this macro. Language other than English is applicable. One workbook can absorb multiple spreadsheets by specifying sheet names. The syntax of macro is: %exporttoxl( LIBIN = WORK, TMPLSHEET =, STATVARS =, DSIN =, DELETETMPLSHEET = NO, WEIGHTVAR =, CELL1ROW = 1, SAVEPATH = C:\TEMP, MERGEACROSS = 1 CELL1COL = 1, SAVENAME = EXPORTTOXL OUTPUT, MERGEDOWN = 1, NROWS =, SHEET =, EXPORTTMPLIFEMPTY = NO, NCOLS =, WSFORMAT = DEFAULT, EXPORTHEADERS = YES, TMPLPATH =, LANG = EN, EXPORTVARFMTS = YES, 9

10 TMPLNAME =, SUMVARS =, ENDCLOSE = YES ); Format of Excel can be done manually or coding in SAS before exporting. 6) %DS2HTM This macro generates HTML file, which can be converted to Excel directly by opening it in Excel. Tens of parameters provide more flexibility for the users during conversion. %DS2HTM (Htmlfile=, Htmlfref=, Openmode=replace, Runode=B); 7) %MYEXCEL; This macro was created by Charles Patridge, dumping the contents of a SAS Data Set to a CSV (EXCEL) file with any stored SAS format; the output can be read in an Excel Spread Sheet. %MYEXCEL (Libdsn=, Download=, Varnlabl=, Mod=); The CSV file name is put in DOWNLOAD parameter, the first two lines of the output are reserved for variable names or labels by specifying Y ; the convenient characteristics of this macro is that, when the original output file exists, the new output can be appended to it by specifying MOD=MOD in last parameter in macro call. This feature is not mandatory. 8) %MakeCSV; The macro created by Lars L Jacobsen exports dataset to the delimited file, such as CSV file. %MakeCSV (Dset=, Out=, Delim=, Limit=, Label=, Title=); SAS dataset in first parameter can accept some dataset options like KEEP or DROP. In output file, the delimiter is optional, by default; it is comma for CSV file. If specified to a different delimiter, the single quote is necessary. The number of records to read into output file can be flexibly limited, which is optional as well. The first row of output file is reserved for column header, either variable name or label can be defined. If no header is required, the user can parse NONE into the macro variable LABEL. The column name is always retained when nothing was specified. The same is for title line, the user can choose to keep it or omit it during the transfer. Method 7: Dynamic Data Exchange (DDE) DDE is a communication protocol available on Windows and OS/2 and supported by SAS 6.08 version and above, permitting two applications to talk to each other by continuously and automatically exchanging data. DDE used to be the only approach to generate the customized Excel report from SAS output before those newer methods came into place. Some features from DDE still enable it to 10

11 overperforme these newer partners (Watts, 2005). DDE earns very good fame of being practical, effective and reliable method to have a customized report and can be used in any programs, like Word, Excel or PowerPoint. The most application is related to Excel in industry. The logic behind the screen is that SAS operates the interface of a PC application using commands in the application native language within a DATA _NULL_ step. Most things done manually in Word or Excel can be automated in DDE. DDE command works quite well in all versions of SAS, only SAS BASE is required; It uses Excel 4 Macro Language (X4ML) function calls into DDE commands. X4ML is a modular, each function majors a specific task. Only function name, purpose and parameters are necessary to understand when using; these functions can also be grouping into SAS macro to be user friendly; The prerequisite of using DDE is that both client and server applications are up and running. The file to write or read must be open and active in the server application. A client/server DDE communication link is triggered by SAS FILENAME statement in terms of Excel, like this: FILENAME <fileref> DDE Excel [xlsaf3.xls]sheet1! r1c1:r1c10 ; So called DDE triplet includes Serve=Excel, Topic=[xlsAF3.xls]sheet1 and Item= r1c1:r1c10. Full path name inside square brackets is not a must, but strongly recommended for robust self-documenting applications. When exporting SAS table to Excel, the destination file has to be open, cell format can be put in DATA step. This example sends data from a SAS session to an Excel spreadsheet with target cells rows 1 through 100 and columns 1 through 3. To do this, submit the following program: Filename Retail DDE Excel Sheet1!R1C1:R100C3 notab; Data Retail; File Retail; Set sashelp.retail; Run; Besides the application of DDE in DATA step, it can also be sandwiched to function within ODS statements. If you don t care about the label format, this is a relatively quick way to get data into a spreadsheet. However, in some variables, where missing values are not ignorable, Notab option is necessary. Regarding the limitations in DDE application, it s that both MS Office and SAS need be installed and active for the data transportation to take place. If either is not installed or active then DDE will fail to function correctly in data transfer. Method 8: Output Delivery System (ODS) and Markup Language By far, the ODS method is most convenient vehicle to capture the reports and widely distribute them, it s available on any platforms, both batch and interactively. On the other hand, because it is on developing stage, some minor disadvantages may exist during conversion process, such as the complete control on cell formatting which can only be solved by applying PROC REPORT/TABULATE and 11

12 writing a Visual Basic macro using the Excel Record VB Macro facility for post-excel data formatting. Most of them would be improved in newer versions. Both ODS HTML and ODS CSV destinations are used popularly by most users. Together, the most powerful destination is the MARKUP language. It can create bunches of output file types from CSV to chtml, LaTeX and an endless variety XML files. All these destinations were really just a part of the expanding universe of ODS MARKUP. Markup languages produces SAS output which is formatted using many different markup languages, same as the destinations mentioned above, which is accessible with a web browser. Both the build-in languages and user-defined languages provide a lot flexibility and power to transfer data from SAS to various applications, like Excel. Basically, a markup language is a textual way of adding context or formatting data. The simplest one is CSV destination, which has the only markup, commas, to separate the data values. The most popular one is HTML; it consists of the tags placed around the content to describe how the data is going to be formatted. Troff language is the oldest one, can be traced back to 1970 s. LaTeX is especially markup language for publishing. The most recent is XML, but is also the trickiest one. Their syntax is simple and can be used to mark up personal language from it. ODS tagset, the customized markup tags, is a template which defines how to generate a markup language output type from SAS format. A tagset can be specified to create certain markup language output from ODS. Over 50 tagsets are shipped with BASE SAS9. Of which, for example, HTML tagset embeds HTML tag to SAS output, MSOffice2K tagset generates HTML file that can be imported to Microsoft Excel and Word in versions 2000 and later (DelGobbo, 2003). If the output destination you want is not covered by any of the SAS pre-defined ones, then the user may have resort to building his/her own tagset through PROC TEMPLATE. By default all the user-defined tagsets will be stored in SASUSER.TEMPLAT item store by PROC TEMPLATE. SAS supplied tagsets are stored in your SASHELP.TMPLMST item store. The full list of tagsets in version SAS 9.2 can be obtained by using the TEMPLATE procedure, Proc TEMPLATE; List tagsets; run; The relationship between a tagset and the ODS destination that it defines are complicated and open for new applications. Based on the specific objective of the user, new features can be flexibly inserted into customized tagset from the existing parent tagsets. Any destination that is defined by a tagset can be changed to do almost anything you can imagine. Or, you can build one from the ground based on a procedures event map. So far, less and less destinations are not becoming tagsets, including ODS PS (PostScript), PCL and PDF. This list may disappear as well in the future. Each destination has a tagset to be associated. Tagset name and the destination name match in most cases except the HTML destination. If the tagsets are unique, multiple tagset output destinations can be open to output same file. The usage of each tagset with its markup destination is triggered by specifying the tagset in the ODS MARKUP statement, for example, ODS markup tagset=excelxp file='pat-to-server/outputfile.xsl'; Or, just using tagset name as destination name, 12

13 ODS tagset.excelxp file='pat-to-server/outputfile.xsl'; More and more tagsets have build-in help. To check the help text, this statement can be referred. ODS csv file='text.csv' options (doc= help ); The original ancestor of MARKUP languages is SGML (Standard Generalized Markup Language), a standard for documentation application. Besides, SGML is also the foundation for popularly used HTML and XML. In pharmaceutical industry, most SAS users are interested in the CDISC (Clinical Data Interchange Standards Consortium) set of standards. However, the increased uses of XML files will obviously push other uses beyond CDISC out of water. Due to the space limit for information at length, just couples of most popular destinations will be outlined here. 1) ODS HTML HTML (Hypertext Markup Language) destination uses HTML4, the most complex one, as tagset by default. There are also other tagsets which serve the different demand respectively (Gebhart, 2006). chtml is simple and basic HTML with no style control capability. It mainly provides black and white HTML and also serves as a good point to create customized HTML tagset. Imode tagset was derived from chtml, but does not offer table function. PHTML supports a very simple Cascading Style Sheet. HTMLCSS then generates more complete HTML with font and color added in. XHTML tagset creates identical feature to HTML4. To use the HTML destination to create the Excel files, the procedure or data step needs be specified within the ODS HTML statement with.xls or.csv extensions, and then Excel will be opened in the Results Viewer window. Excel itself is not an ODS destination, the file opened in Excel is a products of ODS. The following example will show you the simple syntax to generate an Excel file. In this example, the location to save file is specified on path=, file name to output is in body= statement. The whole report uses color style=banker. ODS HTML path='c:/ '; body='retail.xls'; style=banker; Proc print data=sashelp.retail; run; ODS HTML close; Since Excel 97 and newer versions can automatically convert HTML file to Excel file when opening it in Excel format, you can read it in Excel directly. Because the formatting is preserved with data in the Excel file, the file tends to be very large by default, it takes more time to load. However, the HTML tagset created using the ODS HTML Markup Language separates the formatting and the data, therefore, reducing the file size dramatically. There are also some other minor features in output file which need more attention in customization, such as row height and column width, borders, alignment and patterns, page setup, header and footers. All these can be modified by applying new definition in user-defined CSS styles and tagsets, particularly XML techniques. Moreover, macros in Excel can be used to 13

14 automate these various tasks by constructing programming language, Visual Basic for Applications (VBA), within Excel application (Parker, 2003). 2) ODS CSV ODS CSV destination is relatively new and can be used to create comma separated files with.csv extension which can be opened in Excel. This example provides the syntax of how to use ODS CSV destination to print a data. ODS CSV body='c:\retail.csv'; Proc print data=retail; Run; ODS CSV close; ODS CSV can t hold the title instead. Another noising thing is that CSV destination keeps the first two rows empty. This can be changed by modifying the destination or tagset to override it (Parker, 2003). Due to no formatting stored with data, the benefit of using the CSV destination is to reduce the file size quite a lot. Another two relevant destinations are ODS CSVALL and ODS CSVBYLINE. ODS CSVBYLINE can generate the similar output as ODS CSV destination except that it can bring the titles, footnotes, bylines and notes into final report. ODS CSVALL creates only tables and bylines if the data needs be grouped. These three original tagsets have been incorporated into one CSV tagset in latest version with different options for certain objective. 3) ODS ExcelXP Tagset If the SAS output is to be transferred to Excel directly, the ExcelXP tagset is very nice option. In terms of customized Excel report, ExcelXP tagset in ODS MARKUP is regarded as the most popularly used alternative tool. It offers complete control over worksheets in a workbook (Gebhart, 2006) and a welldesigned routine, making custom formats with many options, being quicker than DDE. You don t have to have Excel file open when the SAS result is exported to Excel. It automatically create and name new worksheet, control what to export to each sheet from each table, embed titles and footnotes for printing with options specified, detect format errors in numerical variables, enable to set formulas, frozen panes as needed. Most amazingly, it can also create and index a table of contents for whole workbook. Another main benefit of using ExcelXP (XML) tagset is that each table of SAS output is placed into a separate worksheet within same workbook, which outperforms MSOffice2K (HTML) tagset where all tables will be put on same page. However, this does not mean that ExcelXP tagset is perfect in any conditions. It is so complicated that the user needs some time to figure out how to use it for a special purpose. It can not support graphics generated, neither to create side-to-side tables. The worksheet can be pre-formatted. The newest version only works with newer versions of SAS (9.1.3 or above) and Excel (2002 and above). The basic syntax for ODS ExcelXP tagset usage is similar to other ODS features, both.xls and.html extensions can be created. In the following example, the predefined style STATDOC was used, some options enable the worksheet to control the output appearance. 14

15 ODS tagsets.excelxp file= c:\retail.xls style=statdoc options (sheet_name= Retail' frozen_headers='yes' autofilter='all' auto_subtotals='yes'); ODS tagsets.excelxp file= c:\retail.html ; Proc print data=retail; By year; Run; ODS tagsets.excelxp close; However, couples of things need to be remembered. Text-based columns in Excel output needs be resized when reading in XML file; The leading zero can not be retained and have to be corrected by using Excel format ; If the cell border lines are missing, a new ODS style is necessary to add them back. 4) ODS MSOffice2K Tagset Excel 2000 and greater versions is required for the application of this tagset. Because, Office 2000 is the first release with style information in CSS (Cascading style sheet) format, which is needed to modify the appearance of SAS output in Excel file. This example exhibits how a HTML file is generated using MSOffice2k tagset,.xls extension can also be used to transfer the data to Excel directly. ODS tagsets.msoffice2k style=msobanker file='retails.htm'; Proc print data=sashelp.retails noobs label; by year; var date / style={htmlstyle="mso-number-format:ddmmmyyyy"}; var sales / style={htmlstyle="mso-number-format: "}; Run; ODS tagsets.msoffice2k close; It also generates an HTML file, but with a different set of style sheets and characteristics (Granford, 2007). MSOffice2K tagset does not support creating multi-sheet workbooks, this function and any other features can be easily added to or revised in tagset by modifying the template definition (Pinet, 2007). On the other hand, MSOffice2K tagset is the best approach to generate graph files to be opened with Excel, which can not be accomplished by ExcelXP tagset. 5) ODS SYLK tagset. The whole name of SYLK is Spreadsheet SYmbolic LinK, which is a customer-contributed tagset, created by Jack Hamilton of First Health in West Sacramento, CA (Stetz, 2004). Basically, it s an ASCII text file whose commands can be recognized and used to build a spreadsheet by Excel. The detail descriptions of all currently used SYLK fields and the complete explanations about SYLK file can be found at this website, under the heading Spreadsheet/Database: To use the SYLK tagset is as easy as to use all other tagsets, here is an example of creating a SYLK file. ODS markup body=c:\retail.slk' tagset=tagsets.sylk; Proc print data= sashelp.retails; run; ODS markup close; 15

16 The SYLK file can be created on any platform, whether Excel if installed or not. If you use this technique and generate an Excel file on a platform with no Excel installed, you can move it to the place with Excel installed to open it and save it as.xls file. The entry for SYLK file can be seen in the Files of Type window. One attention is that all fields in this file will be text, for numeric variables, the user has to convert it first before any application. The uses of tagsets.sylk are not limited to PROC PRINT, other procedures like PROC REPORT, PROC CONTENTS and PROC FREQ can also be borrowed to generate the SYLK file. Some procedures may not work very well with tagset SYLK if they trigger events for which there is no definition. There must be a define event block in the tagset for every action the procedure can invoke, otherwise the SYLK file will ignore that element in its output form (Stetz, 2004). The event map of an output file has to be explored if the user likes to define a tagset for such events so that the tagset commends can be found out for editing the tagset. To get these, the statement tagset=event_map can be used to run the procedure. The only output will be the event map, likely hundreds or thousands of lines will be listed from this run. This is an example for simple illustration purpose. ODS markup body=c:\retail.slk' type=event_map; Proc print data= sashelp.retails; run; ODS markup close; PROC SYLK was recently included into SAS, allowing the user to generate Excel files from SAS output as well as read SYLK files and create SAS datasets from them. Method 9: extensible Markup Language (XML) XML is becoming more popular as an industry standard shouldering the task of data transmission and manipulation over the web. XML is a simple and very flexible text format derived from SGML (ISO 8879). Originally designed to meet the challenges of large-scale electronic publishing, XML is also playing an increasingly important role in the exchange of a wide variety of data on the Web and elsewhere. As a text-based language, IMS is based on the concept of documents composed of a series of entities (Objects). Each entity can contain one or more logical elements which have certain attributes describing how it is going to be processed. Both HTML and XML use the tags, HTML uses them, such as <TABLE>, <TR> or <TD>, to control how data is rendered, XML uses them to describe the structure and meaning of data (DelGobbo, 2004). Many techniques are available to use XML support to move data from SAS to Microsoft Excel (versions 2002 and later). They can be used in any platforms where SAS is installed, including Windows, OpenVMS TM, UNIX and z/os. This language is better supported in BASE SAS 9.1 or above software to transfer data between SAS and Microsoft Excel (version 2002 and above). 1) SAS XML Libname Engine (SXLE) One of SAS XML tools the users are most interested in for data switch is the SAS XML LIBNAME engine (SXLE). SXLE enable the exchange of data between SAS table and XML file. During the 16

17 transfer between XML file and SAS table, the new SAS XMLMap, which can be created manually or by XML Mapper, is needed to map any element or attribute in XML file to a corresponding column or row in SAS table (DelGobbo, 2004). Thus, the properties in the transferred file would be retained during transportation. The users can process XML file flexibly by using a Libname statement to associates a libref with an XML file. Libname out XML C:\Retail.xml xmltype=oimdbm xmlschema=yes; Data out.retail; Set sashelp.retail; Run; Unfortunately, both SXLE and the followed ODS MARKUP approaches will lose the format and labels in SAS data when it is transferred from SAS to XML. To prevent this, the user can specify XMLTYPE and XMLSCHEMA in SXLE statement (Li, 2005). XML Spreadsheet Schema (XMLSS) defined by the user is capable of manipulating the Excel format completely from the workbook level to the cell level in use of Tag Hierarchy (Juneja, 2004). The SXLE imports and exports XML file in a sequential order, processing the record one by one from the beginning of a file to its last line. The SXLE does not provide random access, like PROC SORT or ORDER BY statement in PROC SQL. Otherwise, a warning in log window will be exhibited for the processing invalidity for sequential access. To avoid this, a temperate SAS data can be borrowed for random process before any other manipulations. 2) ODS XML tagset XML documents can also be created using the ODS MARKUP statement. The engine can write to an XML document from a SAS data by translating the SAS proprietary file format to XML markup. Users already familiar with ODS may find this method to be a useful extension to their skill set. It s easy to see that the required syntax to create an XML file is alike to other ODS destinations. ODS xml file='c:\retail.xml'; Proc print noobs data=sashelp.retail; Run; ODS xml close; The resulting file being displayed on web page follows a very rigid data structure to detail the title, the name of the data, the descriptions of every column and the data of the entire table on a row by row basis (Cisternas, 2004). Many details carried by XML file is unrelated to the data itself, which increases the processing load during the data transfer, the final file output becomes huge. Another disadvantage for using ODS XML is that ODS can not be used to import the XML file back into SAS. Method 10: EXCEL LIBNAME Engine In version 9 or above, using DATA steps and procedures, SAS files can be written to multiple spreadsheets in an Excel workbook by using the Excel LIBNAME Engine. This technique needs the support from SAS/ACESS, where SAS/ACCESS interface to PC Files, SAS/ACCESS interface to OLE and SAS/ACCESS interface to DB/ODBC products installed. 17

18 The link to output excel file will be defined in LIBNAME statement, just like the regular LIBNAME folder except an excel indicator. If excel file has existed, SAS will remind you by means of an error, then a new excel file needs be created. The variables in SAS file can be reordered and spaces can be placed within the variable names when being written to the Excel file in first row as column headers. For numerical variables, the little n is required. Using SAS/ACCESS interface to PC Files, the following example outputs SAS table Retails into Excel file with sheet name retained, a new sheet Avg_sale is added to the file through the SQL procedure, which includes the average sale for each year. Options validvarname=any; Libname out EXCEL c:\retails.xls ver=2000; DATA out.'retail Report'n; Set sashelp.retails (Rename=( Sales='Total Sales($)'n Date='Quarter (year)' Year='Sale Year (4)'n)); Avg_month=Sales/4; Run; Proc sql; Create table out.avg_sale as Select Year, avg(sale) as avg_sale From sashelp.retails Group by 1; Libname out clear; The formatted values can be stored, but SAS formats, lengths and labels are not preserved. If needed, some other options have to be considered. A second drawback is that it does not allow the replacement of a worksheet. The deletion of old worksheet has to be done manually. Another two products linked are SAS/ACCESS Interface to OLE DB and SAS/ACCESS Interface to ODBC. The example using the LIBNAME-OLE DB Engine statement likes this, Libname out OLEDB provider="microsoft.jet.oledn.4.0" preserve_tab_names=yes preserve_col_names=yes properties=('data source'="c:\retails.xls") provider_string="excel 8.0"; For SAS/ACCESS Interface to ODBC product, the Excel ODBC data source has to be configured so that it points to the Excel workbook file that the user wants to access from SAS. Here is the LIBNAME statement for using this product. A simple example to use its LIBNAME Engine is, 18

19 Libname out ODBC dsn=" c:\retails.xls"; Method 11: ODBC in Windows Open DataBase Connectivity (ODBC) interface allows the process to write one or more SAS files into one excel workbook. The short concept of ODBC is to use window to manage the files. SAS dataset name will become worksheet name in excel file, therefore, extremely long SAS dataset name will be truncated in spreadsheet label. Because of the absence of REPLACE option, the previously existing excel file won t be overwritten; an error in log window will remind the user about this. In use of ODBC connection, the ODBC Administrator must be run to establish a connection to a data source. Then, to use the ODBC connection in SAS code, the LIBNAME statement helps to connect to the targeted excel workbook, just like a library connection in SAS system. Libname Retail ODBC NOprompt="DSN=excel files; DBQ=C:\Retail.xls"; DATA Retail.data1; set sashelp.retail; Run; DATA Retail.data2; set sashelp.retail; Run; DATA Retail.data3; set sashelp.retail; Run; DATA Retail.data4; set sashelp.retail; Run; Libname Retail clear; DSN specifies the data source to store new ODBC table. With option clear in last statement, SAS releases control of the file and allows Excel to access it. If Excel file exists, there are several approaches to delete it: 1) Delete it manually if necessary; 2) Delete it in use of DATA step like this: Data _null_; a=filename("retail","c:\retail.xls"); b=fileexist("retail"); msg_=sysmsg(); PUT _msg=; if a=0 and b=1 then c=fdelete("retail"); d=filename("retail",""); PUT d=; Run; FILENAME test if the specified file exists, returns 1 or other positive values if yes, otherwise 0; FILEEXIST returns 1 if the external file associated with fileref exists, 0 if it does not exist. FDELETE will return 0 if the deletion of an external file or empty directory. Once the operation fails, it will display a negative value. 3) Delete it using a system function: %SYSEXEC del c:\retail.xls; 19

20 Method 12: PROC DBLOAD The DBLOAD procedure for PC files is available only under Microsoft Windows operation environments. It loads data and creates PC files from SAS data, associate each SAS variable with a PC file column and assigns a default name and data type to each column if PUTNAMESYES statement is provided. Because Excel file has the maximum limit on record number and rows, LIMIT= has to be specified when the records are beyond the default. This procedure is not able to overwrite the existing file, when new file with same name needs be loaded, the old one has to be deleted first. Options Noxwait; X'DEL c:\retail.xls'; Run; Proc DBLOAD DBMS=XLS Data=sashelp.Retail; Path=C:\Retail.xls'; Version=7; Label; Limit=16000; PUTNAMESYES; Load; Run; Method 13: SAS Server Technologies When Window SAS license is not available, SAS table needs be transferred to an Excel-accessible format on a non-window SAS server and transferred to window office to open in Excel. These data interchange format between SAS and Excel include CSV, SYLK and HTML. SAS server program may be run on any platform, such as OS/390, OpenVMS, UNIX or Windows, but it must output the file in data exchange format, like HTML. The SAS programs can consist of any of these programs: DATA step, PROC MACRO or SCL code, or their combinations. There are some approaches to access a SAS server for absorbing data to Excel. To place SAS output into an Excel workbook, type the URL into the Open dialog box of Excel; For direct inclusion of SAS output in Excel from web server, the usage of ODS tagset ExcelXP is better practice as described in XML application above. Web Query in Excel is the second tool to allow the URL to be put into a text file to avoid the repeated typing effort, so that HTML file is automatically included in Excel output (DelGobbo, 2003). The third technique to enable Excel to exchange data using SAS server technology is Visual Basic for Applications (VBA) macros (DelGobbo, 2002). Two of relatively new technologies, Application Dispatcher and Stored Process Services, construct the path to support these data transfer. Here is just a brief description on these two technologies, more details can be found on SAS support online. 20

21 3) Application Dispatcher With the support of SAS/IntrNet software, using SAS server technologies, mainly the Application Dispatcher, SAS output can be dynamically absorbed into Excel file (DelGobbo, 2004). These are new features in SAS version 9.1. The SAS programs are executed from a web server or any where the client can open an HTTP connection to either these two servers, such as within the Office application. The initiation of execution is triggered by accessing a URL that points to the SAS server program. SAS server uses name/value pair in the URL to construct macro variables available to SAS program. These three components, Client, Web Server and SAS Application Server, compose the entire Application Dispatcher (DelGobbo, 2002). The client opens a connection to the web server and requests the program to run. SAS server executes the program and sends the results to web server. Then, the findings are sent back to the client to review. 4) Stored Process Service Another technique to automate the inclusion of SAS table into Excel is the Integrated Object Model (IOM), a critical component of the SAS Integration Technologies software (DelGobbo, 2002). The distributed object interfaces for traditional SAS features is supported. Java Server Page (JSP) or Active Server Page (ASP) based application server can be adopted for this kind of data transfer. IOM-based server, SAS Stored Process Service (SPS), has much same features as the SAS/IntrNet Application Dispatcher. SAS program that is stored on a server can be executed as required by requesting applications. SPS also provides the cross-platform access for existing SAS/IntrNet applications enabling them to take advantage of the SAS 9 BI Architecture. SAS/IntrNet Application Dispatcher programs can usually be converted to stored processes with minimal or no modifications. To get the customizable user interface, SAS Stored Process Control (SPC) is available in new release of SAS software to convenience the users in specific projects, it can run inside Excel. Method 14: Enterprise Guide with Excel This requires SAS and SAS/ACCESS interface to PC files format on the same machine. Like Export Wizard, this can be conducted by simply clicking the menu Enterprise Guide. Add an icon for SAS data into your Enterprise Guide project, then right-click on the data table icon in your project. Select Export and choose Local Computer at Save To prompt. When the Export dialog is open, under Save as type, choose Microsoft Excel Files (*.xls), provide a file name and click on Save. The file in SAS format will be saved as in Excel. CONCLUSIONS 21

22 The techniques for data exchange between SAS and Excel has spread to an increasing broader realm, marring the skills from many other softwares and platforms to SAS, particularly those relatively new and high-level devices. Hence, its priority in efficiency and convenience is being enhanced further. Depending on the environments where the data is transferred, different techniques can be applied selectively and many of them can also cross their roads sometimes. The introduction of some vehicles in this paper, although covering a lot of ground, is only a clue for your explorations, the practical experience and deeper understanding will be necessary for the users to equip him in a real world. REFERENCE Brown D %sas2xl: A Flexible SAS Macro That Uses Tagsets to Produce Complex, Multi-Tab Excel Spreadsheets with Custom Formatting. Proceedings of the 30 th SUGI Proceedings, Philadelphia, Pennsylvania, April Cisternas M. and Cisternas R Reading and writing XML files from SAS. Proceedings of the 29 th Annual SUGI Conference, Montréal, Canada, May Cranford K How to Excel with SAS. Proceedings of the 28 th Annual SCSUG Conference, Austin, Texas, September 30-October 2. DelGobbo V Techniques for SAS Enabling Microsoft Office in a Cross-Platform Environment. The 27 th SUGI Proceedings, Orlando, Florida, April DelGobbo V A beginner s Guide to incorporating SAS Output in Microsoft office applications. Proceedings of the 28 th Annual SUGI Conference, Seattle, Washington, March 30-April 2. DelGobbo V From SAS to Excel via XML. Proceedings of the 29 th SUGI Conference, Montréal, Canada, May DelGobbo V Moving data and analytical results between SAS and Microsoft Office. Proceedings of the 30 th SUGI, Philadelphia, Pennsylvania, April DelGobbo V Proceedings of the 31 st SUGI Conference, San Francisco, California, March Derby N Revisiting DDE: An updated macro for exporting SAS into custom-formatted excel. Annual Conference Proceedings of Western Users of SAS Software 2007, San Francisco, October Frey G SAS Excels! The 30 th SUGI Proceedings. Philadelphia, Pennsylvania, April Gebhart E The beginners guide to ODS Markup: Don t panic! Proceedings of the 31 st SUGI Conference, San Francisco, California, March Juneja R Using XML to create Excel spreadsheet in SAS: A step by step guide. Proceedings of PharmaSUG 2004 Annual Meeting, San Diego, California, May