Let There Be Highlights: Data-driven Cell and Row Highlights in %TAB2HTM and %DS2HTM Output

Transcription

1 Let There Be Highlights: Data-driven Cell and Row Highlights in %TAB2HTM and %DS2HTM Output Introduction The SAS System for Information Delivery provides powerful and flexible web publishing tools (SAS macros) called %TAB2HTM and %DS2HTM. These tools provide a terrific method for the direct conversion of SAS PROC TABULATE and data set display output to hypertext markup language - HTML, for ease of display over intranets or the Internet. At our site, that ease of use, combined with the power of the SAS System, has created the proliferation of large volumes of HTML output for information delivery to business partners. What was soon needed was a way to quickly identify and highlight particular data cells, rows and entire tables. In addition, it needed to be data-driven. The solution arrived at puts a new twist on a very old technique: reading back in an output file and postprocessing it to customize the output. Even though it uses an old tried-and-true technique, it remains a valuable tool in the SAS toolkit. This paper will walk you though the steps needed to add highlighting to your SAS HTML output, illustrating the ease and power of the technique along the way. Matthew Flynn, The Hartford Ray Pass, Ray Pass Consulting Simple non-html PROC TABULATE output We will be using the sample application found at the SAS Web Publishing Tools web site throughout this paper, with one minor change - we use an output format of comma9.0 instead of dollar14.2. The original code can be found at (TABULATE Formatter Usage Example). We start with traditional PROC TABULATE code and output, and make a few enhancements along the way. The first few changes follow those on the SAS website almost feature for feature. options nodate nonumber; title1 'World Wide Product Sales Report ; proc tabulate data=sashelp.prdsal2; class state; var actual predict; table state, (actual predict) * (sum * f=comma9.0 mean * f=comma9.0); keylabel sum ="Total" mean="average"; run; Figure 1 - Standard PROC TABULATE output Enhancement 1 - Adding the TABULATE Formatter Macro to the SAS Program The first enhancement uses %TAB2HTM to add HTML tags to the PROC TABULATE output in two parts which 1) mark the start of the data capture: %tab2htm(capture=on, runmode=b); /* TITLE AND TABULATE CODE GOES HERE */ and 2) mark the end of the data capture: %tab2htm(capture=off, runmode=b, openmode=replace, htmlfile=file.html, center=y, tsize=+2); Figure 1 shows the standard PROC TABULATE output from the above code.

2 The resulting output is shown in Figure 2. Figure 3- Row and column headers in color Figure 2- TAB2HTM output in web browser Note that the size of the title has been amplified by the tsize=+2 option in the %TAB2HTM macro. Enhancement 2: Changing Colors of Output Elements Next, we modify the %TAB2HTM macro call to manually add color to the table column and row labels. %tab2htm(capture=off, runmode=b, openmode=replace, htmlfile=file.html, center=y, tsize=+2, clcolor=blue, /*NEW*/ rlcolor=blue); /*NEW*/ Enhancement 3: Adding background We can continue dressing up our HTML output by adding background color, for example. Images can be added as well. %tab2htm(capture=off, runmode=b, openmode=replace, htmlfile=file.html, center=y, tsize=+2, clcolor=blue, rlcolor=blue, tbbgcolr=lightgrey); /*NEW*/ The resulting output is shown in Figure 4: The resulting output is shown in Figure 3. If you are viewing this paper online or on the CD version of the proceedings, you will see the row and column headers in blue. If you re holding hard copy, trust us.

3 Figure 4- Adding a background color Figure 5- Adding an image to the title We can also add an image to the title of the table by modifying the title code and inserting an animated GIF file, changing it from: to: title 'World Wide Product Sales Report'; title '<IMG ALIGN=CENTER SRC= globeanm.gif> World Wide Product Sales Report'; The method is to simply add appropriate HTML tags into the title text (in this case for example, we added <IMG ALIGN=CENTER SRC=globeanm.gif>). We must also add another option to the second %TAB2HTM call, namely encode=n. This tells the macro NOT to encode the < and > characters into ASCII, but to rather leave them as is. The macro then interprets them as HTML tag delimiters, obeys the instructions included in them and renders the enhanced title. The resulting output is shown in Figure 5. Once again, you ll have to trust us here. If you were viewing this table with an HTML browser, the globe at the left of the title would be spinning. Manual Highlighting of HTML files We are now ready to begin a different approach. We ll continue to make manual changes, but this time we are going to make our changes to the HTML file after it is generated with the %TAB2HTM macro for a single table, and highlight individual cells or rows. For example, suppose we wanted negative numbers to stand out. We could easily highlight them by changing the background color of the cells containing them. (There are no negative numbers in the sample we are working with, but we can manually create some. For illustration purposes, we have manually flipped the sign on the Ontario Predicted numbers by adding minus signs - assume they were there all along). Now, we can go into the HTML generated code and make the necessary changes. We ll simply add the HTML tag BGCOLOR=HOTPINK to the cell definitions for the Ontario Predicted data as generated, changing them from <TD ALIGN="RIGHT" VALIGN="BOTTOM" NOWRAP>$-469,832</TD> <TD ALIGN="RIGHT" VALIGN="BOTTOM" NOWRAP>$-816</TD>

4 to <TD ALIGN="RIGHT" VALIGN="BOTTOM NOWRAP BGCOLOR=HOTPINK >$-469,832</TD> <TD ALIGN="RIGHT" VALIGN="BOTTOM" NOWRAP BGCOLOR=HOTPINK >$-816</TD> The resulting output is shown in Figure 6. Interactive HTML Editors HTML editors, or any good text editor in fact, usually provide powerful search-and-replace functions. These could be used, for example, to change all negative numbers in a table to display as hotpink, simply by applying a global change from '>$-' to ' bgcolor=hotpink>$-'. UNIX SED utility In a UNIX production environment of mass-produced web pages, we can use the powerful UNIX textprocessing command SED to issue the global search-andreplace function. What's even better, it can be called directly from a SAS program after generating the output HTML file. SED (Stream-oriented, non-interactive EDitor) can be invoked by SAS with code like: x 'sed "s/>\$-/bgcolor=hotpink>$-/g" file.htm > newfile.htm'; The SED command s/target/replace/g' filename > newfilename is used to search for target text and to replace it with replace text. The g option says do all occurrences on each line. The backslash '\' is required because '$' is a keyword in SED that allows a search only on the last line. The '\' instructs SED to treat the character immediately following it as a literal. Figure 6- Manually highlighting cells Powerful Tools The above technique is quite useful for the customization of small numbers of static web pages, but how we one expand and enhance this general process? Better yet, how can we make the table modification process datadriven? That is, how can we achieve flexible automated complete formatting control of the generation of web pages created from SAS? The tedious manual HTML table-editing process shown above would benefit from an automation technique. Powerful tools for this process include HTML-editors and UNIX regular expressions. There are lots of ways to do this. First, we briefly discuss (GASP!) some non-sas options. Then we present the advantages of the SAS System that provide dynamic data-driven SAS output ready for web-based delivery. More Powerful Tools: Use SAS to Make it Data-driven The final technique we examine reads the HTML output data back into SAS for text string manipulation. In addition, we can make the process data-driven, allowing us to highlight any cell, row, column or table based on the data values in the cells (or in any of the cells in a particular row). These data values do not have to be predetermined or known in advance. How can we do it? One way is to generalize the above search-and-replace tools with a SAS MACRO and use the power of SAS formats to identify individual cells. As is often the case, this technique can best be explained by example. An easy, flexible, and powerful way to quickly identify target cells for highlighting is though SAS picture formats. The examples below insert a marker character into the display of data values which meet certain pre-set criteria. The arbitrary character that is inserted is removed before final display. For example, the numeric picture format abv600k defined below can be used to identify all data values above 600,000 by prefixing them in the output with an asterisk, *, (any character can be used as a marker).

5 proc format; picture abv600k low ='000,000,009' <-high='000,000,009 (prefix='*'); By applying this format to the data as they go through the TABULATE procedure and the %TAB2HTM macro, we can mark all the desired values in the output HTML file. HTML output is shown from this step in the process in Figure 7. filename in C:\HTML Highlights\file.htm'; filename out 'C:\HTML Highlights\newfile.htm'; data _null_; infile in missover length=flen; file out; length line $100; input line $varying. flen; if index(line,"*") then do; "*", "<B>"); "</TD>", "</B></TD>"); end; put line; run; The <B> and </B> HTML tags add bold text formatting. The resulting transformed HTML output file would then look like that shown in Figure 8. Figure 7- Marking target cells The next step is to process the output HTML file with a string manipulation/substitution macro to replace the marker with the desired HTML tags. The SAS System s facility with the input and output of flat files, combined with its powerful text string manipulation functions allow quick work of HTML formatting changes. We simply read in the HTML file, do a global search-and replace and then write out the file with a data _NULL_ step. The INFILE, FILE and PUT statements in the data step do the file handling while the TRANWRD function does the text swapping. Assuming that all data items above 600,000 are marked with an * (as in Figure 7 above), and that the HTML output file from the %TAB2HTM macro is called C:\HTML Highlights\file.htm', the code necessary to change the targeted output cells to BOLD text would look something like: Figure 8- Highlighted target cells Enhancement - highlighting entire rows based on cell values The next enhancement will be to select not just individual data cells for highlighting, but the entire rows in which the targeted cells are contained. An immediate problem

6 with this task is that HTML displayed table rows are usually described in the source file over multiple lines. As a reminder, the structure of a HTML table definition looks like: <TR> (begin table row definition) <TH> (table header for row label)</th> <TD> (cells definition) </TD> <TD> (as many <TD>s as columns) </TD>.. </TR>(end table row definition). The method that we use to highlight an entire row based on the value of a cell or cells in that row utilizes a combination of preprocessing of the data before it is used in the TABULATE procedure and the %TAB2HTM macro, and then follows the macro with text substitution processing of the HTML output file. The preprocessing of the data uses a technique that has been an occasional topic of discussion on SAS-L, the SAS web discussion list (thanks again, SAS-L!). This method uses PROC SQL to identify targeted records based on user selected criteria. It then stores either the targeted values themselves in a macro variable, or other associated variables. In our example, we create two macro variables called numst and rows600 which respectively hold the number of states whose total sales amounts are greater than 600,000, and a list of those states. We then use these macro variables later to identify rows in the HTML output data for highlighting. The preprocessing code looks as follows: proc sql noprint; select count(*), state into :numst, :rows600 separated by '#' from (select state from sashelp.prdsal2 group by state having sum(predict) gt ); quit; Based on the data as displayed in Figure 8 above, we would select the row labels for California, Colorado, Illinois, New York, North Carolina and Washington. The # separator character (any character can be used) in the SQL code line is necessary here because some of the state names may, and in this case do, contain more than one word. We have to end up with a delimited list of state names and if we didn t specify the #, the list would be space-delimited. New York for example would then be seen as two separate states. The result of the above code is a macro variable called rows600 with a value of: California#Colorado#Illinois# New York#North Carolina#Washington We then run our data through the usual PROC TABULATE and %TAB2HTM. Then we modify the HTML file that is output from the %TAB2HTM macro by running it through a DATA _NULL_ text substitution process as follows: %macro selrows; data _null_; infile in missover length=flen; file out; retain rowtag 0; length line $100; input line $varying. flen; if index(line, "<TH") then do; %do i=1 %to &numst; %let r=%qscan(&rows600,&i,#); ">&r.<", ">*&r.<"); %end; if index(line, "*") then do; line=compress(line, * ); "<TH", "<TH bgcolor=hotpink"); rowtag=1; end; end; if index(line,"<td") and rowtag = 1 then do; line=compress(line, * ); "<TD", "<TD bgcolor=hotpink"); end; if index(line, "</TR>") then rowtag=0; put line; run; %mend; %selrows The %SELROWS macro works as follows. First, we read each line of the source HTML file into a variable called LINE. If the line contains the opening portion of the HTML table header tag, <TH, we check to see if is the header for one of our previously identified rows (contained in &ROWS600.) If it is, we mark the row with an *. (When searching for the state names, we only want exact strings so we add the > character which will always appear directly in front of a state name in the HTML code, and the < character which will always appear immediately following the state name. This will insure that we pick California, and not Baja California.) We then remove the * from the state name in the header cell, change the background color of that header cell and turn on a switch called rowtag. We then remove the * s from, and change the background color of all data cell HTML rows (TD rows) following the marked TH row until we hit an end-of-row HTML row (/TR). At that point we turn off the rowtag switch and begin looking for another targeted HTML header row.

7 The resulting transformed HTML output file would then look like that shown in Figure 9. The overall task is to use restrained table highlighting to draw the report reader s eye and enhance or emphasize the points you want to make. Several more examples of uses of highlighting might be: general page formatting - every N rows, or every K columns. outlier or exception reporting. picking out statistically significant variables. following a particular variable though the results of several analyses or reports. Other Resources A companion paper to this one entitled You CAN Get There from Here (and Back Again): Adding Hot-link Drill-down Capabilities to %DS2HTM and %TAB2HTM Output, by Ray Pass, describes methods for creating navigational links to other Web Publishing tool output. Using the methods provided here and in that paper significantly enhance the potential productivity of these powerful web publishing tools. Figure 9 - Highlighted target rows An Automatic Flexible MACRO tool In the spirit of the powerful, flexible, easy to use %TAB2HTM macros supplied by SAS Institute, we ve built a macro system that automates the table cell and row highlighting process (column highlighting is also being developed.) This system is available from the primary author. After inserting a target swap character in your HTML output, the macro allows the insertion of a wide variety of HTML formatting tags for individual data cells and rows, and columns to be added. Next Challenge - Uses (and abuses) It is easy to go quickly overboard with easy flexible table highlighting - for example, inserting animated GIFs in multiple table cells. Too much highlighting can make it harder to pick out data elements than no highlighting at all! For table formatting tips, visit the SAS ODS Style Gallery at to view some well designed examples of SAS generated HTML pages. The release of Version 7 of the SAS System has presented other significant Web publishing tools which have been described recently. Chris Olinger, in SUGI 24 Paper 56, Twisty Little Passages, All Alike - ODS Templates Exposed, describes using the Version 7 CELLSTYLE-AS statement in an ODS Template to format individual data cells based on the data values. It does not, however, highlight the row or column conditional on the data values in a cell. SUGI 24 Paper 190, Getting Stylish with Version 7 Base Reporting, by David Kelly and Sandy McNeill describes using traffic lighting and adding hyperlinks with Version 7 PROC TABULATE and PROC REPORT. The techniques they describe are very close to those discussed here for Version using PROC FORMAT to define data ranges to highlight with PROC TABULATE. PROC REPORT, however, offers the power of data set programming to accomplish the same task. The main difference in Version 7 HTML formatting is the modification and use of templates and styles to achieve data-driven cell formatting. Also newly available in Version 7 PROC REPORT is a new _ROW_ symbol used to highlight entire rows as we have done here. In conclusion, the data-driven highlighting techniques introduced here, in combination with the method for adding data-driven hyper-links described in a companion paper sited above can combine to give you two powerful tools to quickly add increased functionality and ease-ofuse to your SAS created web pages.

8 References SAS is a registered trademark of the SAS Institute Inc., Cary, NC, USA. Author Contact Information Matt Flynn The Hartford Personal Lines 400 Executive Blvd. Southington, CT Matt.Flynn@TheHartford.com phone: (860) fax: (860) Ray Pass Ray Pass Consulting 5 Sinclair Place Hartsdale, NY raypass@att.net phone: (914) fax: on request