PH006 Audit Trails of SAS Data Set Changes An Overview Maria Y. Reiss, Wyeth Pharmaceuticals, Collegeville, PA

Transcription

1 PH006 Audit Trails of SAS Data Set Changes An Overview Maria Y. Reiss, Wyeth, Collegeville, PA ABSTRACT SAS programmers often have to modify data in SAS data sets. When modifying data, it is desirable to create an audit trail of the changes made. This paper presents an overview of the techniques commonly used to create audit trails for changes to SAS data sets. The code contained in this paper was developed using version 8.2 of SAS. The intended audience of this paper is beginner and intermediate SAS programmers. INTRODUCTION An audit trail is a log of all changes made to data. Information that is typically included in an audit trail is: the data before and after the change. the userid of the person who made the change. the date and time of the data change. the reason for the change. The advantages of an audit trail are: 1. An audit trail improves data security by making it more difficult to hide fraudulent changes to data. 2. An audit trail improves data integrity. An audit trail should contain a record of both unintentional and intentional changes to data, so you can use it to detect and fix unintentional data changes. 3. You can use an audit trail to regress to earlier versions of data. 4. You can reconstruct and reapply data updates that originally failed. 5. You can use audit trail data to develop usage statistics and patterns. 1 In the pharmaceutical industry, an audit trail of data changes is essential due to 21 CFR Part 11 compliance. 2, 3 SAS programmers have several techniques available for generating audit trails of SAS data set changes. This paper presents the pros and cons of commonly used techniques for creating audit trails of changes to SAS data sets. These commonly used techniques are: Visual review of data dumps. PUT statements within a SAS data step to explicitly write out a log of data changes. Using the new audit trail facility available in SAS Version 8. Using PROC COMPARE to list the differences in the data before and after changes were made. Using custom macros to list the differences in the data before and after changes were made. Only allowing data changes to be made in applications that are programmed to create an audit trail (such as SAS/AF or SAS/Intranet applications). Using the new SAS Drug Development Product. VISUAL REVIEW OF DATA DUMPS The simplest technique is to use an output procedure such as PROC PRINT or PROC REPORT to print out the data after the changes are made. A person can then visually review the output and verify that the data changes were made correctly. Obviously, this technique is prone to human error. Also, this technique is very tedious. It should be used only with small quantities of data. 1

2 PUT STATEMENTS WITHIN A DATA STEP Another simple method of generating an audit trail of data changes is to change the data within a data step and then use put statements within the same data step to write a log of the changes. For example: data NSMEDV&new; set NSMEDV&old; file print; title4 **** CORRECTION LOG for NSMEDV&new DATA **** ; title5 ; if patient = and visitdt = mdy(01,01,2003) & medx = Tylenol then do; put patient= visitdt= medptx= medx= startdt= /* key vars */ dose = 5; doseu = MG ; route = PO ; reason_change = Add new data for Patient Tylenol ; Correction Made: dose= doseu= route= ; /* changed variables */ reason_change= ; end; The advantages of this technique are that it is simple to implement, and it does not require extra storage space. A disadvantage is that it too can be prone to human error since it depends on the programmer entering the correct variable names in the put statement. Also, this technique will not detect unintentional changes to data. USING THE SAS AUDIT TRAIL FACILITY With Version 8, SAS added a new audit trail facility to the SAS system. The SAS audit trail facility is a read-only SAS data set that contains information about changes made to another SAS data set. The observations in the audit trail data set contains all the variables from the original data set plus the following additional variables 4 _ATDATETIME ATUSERID ATOBSNO ATRETURNCODE ATMESSAGE ATOPCODE_ Stores the date and time of a modification Stores the logon userid associated with a modification Stores the observation number affected by the modification, except when REUSE=YES Stores the event return code Stores the SAS log message at the time of the modification Stores a code describing the type of modification where DA = Added data record image DD = Deleted data record image DR = Before update record image DW = After update record image EA = Observation add failed ED = Observation delete failed EW = Observation update failed Example Creating a Simple Audit Trail The following SAS dataset, MYRDATA.NSMED, contains information about concomitant medications for patients in a clinical trial. PATIENT VISITDT MEDX DOSE ROUTE STARTDT STOPDT OCT2000 Methyl-Prednisolone. 20SEP SEP OCT2000 Methyl-Prednisolone. 23SEP SEP OCT2000 Methyl-Prednisolone. 24SEP SEP OCT2000 Methyl-Prednisolone. 25SEP SEP OCT2000 Methyl-Prednisolone. 26SEP SEP OCT2000 Methyl-Prednisolone. 27SEP OCT2000 2

3 I add an audit trail to the SAS data set using the code listed below. Note that I add one optional user variable called reason_change. This variable will contain the text of the reason for the data change. %* Add an audit trail data set. *; proc datasets lib=myrdata; audit nsmed; initiate; user_var reason_change $60; Then I modify the SAS dataset using PROC SQL. %* Update a record. *; proc sql; update myrdata.nsmed set dose = 5, reason_change = "Add dose information for first episode" where patient = '000101' & visitdt = '18OCT2000:00:00:00'dt & startdt='20sep2000:00:00:00'dt & stopdt='22sep2000:00:00:00'dt; %* Insert a record. *; proc sql; insert into myrdata.nsmed set patient = '000101', visitdt = '20DEC2000:00:00:00'dt, medx = 'Tylenol', route = "PO", startdt = '18DEC2000:00:00:00'dt, stopdt = '19DEC2000:00:00:00'dt, reason_change = "Add new non-study medication record for patient "; %* Delete a record. *; proc sql; delete from myrdata.nsmed where patient = '000101' & visitdt = '20DEC2000:00:00:00'dt & medx = 'Tylenol' & route = 'PO'; Here is a listing of the audit trail data set that results from these data changes. PATIENT VISITDT MEDX DOSE ROUTE STARTDT OCT2000 Methyl-Prednisolone. 20SEP OCT2000 Methyl-Prednisolone 5 20SEP DEC2000 Tylenol. PO 18DEC DEC2000 Tylenol. PO 18DEC2000 PATIENT STOPDT Reason_Change _ATDATETIME_ SEP MAY2003:14:38: SEP2000 Add dose information for first episode 22MAY2003:14:38: DEC2000 Add new non-study medication record for patient MAY2003:14:38: DEC MAY2003:14:38:32 PATIENT _ATOBSNO ATRETURNCODE ATUSERID ATOPCODE ATMESSAGE_ reissm DR reissm DW reissm DA reissm DD 3

4 The audit trail data set contains two observations for the record update, one observation for the record insertion, and one observation for the record deletion. Pros of the Audit Trail Facility 1. The audit trail facility is part of the Base SAS system. Thus, this feature is supported by SAS. 2. The audit trail facility stores the SAS log message at the time of the modification. 3. The audit trail facility stores information about failed appends. 4. The audit trail facility stores observations that were rejected by integrity constraints. 5. You can add optional user variables to the audit data set such as Reason for Change. 6. The audit trail facility allows you to roll back your data to any given point in time through the use of the timestamp variable _ATDATETIME_. Cons of the Audit Trail Facility 1. The audit trail facility only applies to data changes made by editing the data interactively using PROC APPEND MODIFY statement in the DATA step UPDATE, INSERT, and DELETE statements with PROC SQL According to SAS, An audit trail is not appropriate for data sets that are rebuilt using the DATA step or the SQL procedure. In fact, rebuilding the data set deletes the audit trail. 5 Also, The audit trail is not recommended for SAS data files that are copied, moved, sorted in place, replaced, or transferred to another operating system because those operations do not preserve the audit trail The audit trail data set can become very large since the observations in the audit trail data set contain all of the variables from the original data set plus additional variables. Also, the audit trail data set contains two observations for every record update. 3. The _ATMESSAGE_ variable can contain a long SAS message. In fact, SAS recommends setting a large linesize (e.g. linesize=250) when displaying the content of the _ATMESSAGE_ variable. 4. The logging of data into the audit trail data set can degrade system performance. USING PROC COMPARE PROC COMPARE is the SAS procedure for comparing two SAS data sets against each other. 6 One application of PROC COMPARE is logging and checking changes to a SAS data set. One can use PROC COMPARE to compare the data set before and after editing to make sure that the desired changes were made correctly and that no unintentional changes were made. In this way, the output from PROC COMPARE can serve an audit trail of changes made to the SAS data set. 7 Example Using PROC COMPARE to create a log of data edits. Version 01 of the PAYROLL data contains the following data: ID SALARY The data manager creates Version 02 of the PAYROLL data with edits: 4

5 data payrollv02; set payrollv01; if id=057 then salary=600; if id=026 then delete; To produce a report of the edits, use the code: title1 "Differences Between PAYROLLV02 and PAYROLLV01"; proc compare base = payrollv01 compare = payrollv02 briefsummary listall transpose; id id; This code produces the following output: Differences Between PAYROLLV02 and PAYROLLV01 17:03 Wednesday, April 23, The COMPARE Procedure Comparison of WORK.PAYROLLV01 with WORK.PAYROLLV02 (Method=EXACT) Comparison Results for Observations Observation 2 in WORK.PAYROLLV01 not found in WORK.PAYROLLV02: ID=26. ID=57: Variable Base Value Compare Diff. % Diff SALARY NOTE: Data set WORK.PAYROLLV01 contains 1 observations not in WORK.PAYROLLV02. NOTE: Values of the following 1 variables compare unequal: SALARY Example Using PROC COMPARE to detect unintentional changes to data. PROC COMPARE can also detect unintentional changes to data. As you know, one must be careful when merging data sets. When merging two data sets that contain a common variable with different lengths in the two data sets, SAS will take the length from the first data set in the MERGE statement. If the variable has a shorter length in the first data set, this can cause values from the second data set to be truncated in the merged data set. Data Set 1- LNAME Has Length $10 ID LNAME SALARY 11 REISS WILLIAMS SMITH 777 Data Set 2 - LNAME Has Length $17 ID LNAME 11 REISS 26 WILLIAMS 28 SMITH 34 HOSKINSON-ALVAREZ 5

6 57 JONES 60 BROWN These two data sets are merged using the code: data mergedata; merge ds1 ds2; by id; The variable LNAME in the resulting data set MERGEDATA has length $10. To produce a report of data changes, use the code: proc compare base = ds2 compare = mergedata briefsummary; id id; title1 "Comparison of DS2 and MERGEDATA"; This code produces the following output showing that variable LNAME has been truncated for the record with ID 34. NOTE: Values of the following 1 variables compare unequal: LNAME Value Comparison Results for Variables Base Value Compare Value ID LNAME LNAME 34 HOSKINSON-ALVAREZ HOSKINSON- Pros of Using PROC COMPARE to Create an Audit Trail 1. PROC COMPARE is part of the Base SAS system. Thus, this feature is supported by SAS. 2. PROC COMPARE detects all data changes both intentional and unintentional. It will detect data changes no matter how the changes were made (e.g. it detects changes made through the data step). 3. Using PROC COMPARE to generate an audit trail does not require additional storage space to store an audit data set. 4. PROC COMPARE generally does not require a lot of system resources to run. Cons of Using PROC COMPARE to Create an Audit Trail 1. PROC COMPARE does not provide all the features of a complete audit trail. It does not store information about failed appends or observations rejected by integrity constraints. It does not provide information on the reason for the changes, and it is difficult to roll back data changes using PROC COMPARE output. 2. PROC COMPARE output can be lengthy. 3. PROC COMPARE can be confusing to use because it has a large number of options. 6

7 USING CUSTOM MACROS PROC COMPARE is a very useful tool. However, it has the disadvantage of producing lengthy output. It can also be a little difficult to streamline and tailor PROC COMPARE output. Because of this, some SAS programmers have developed their own custom macros to compare SAS data sets. These custom macros can be used in the same manner as PROC COMPARE to compare the SAS data set before and after changes were made, thereby generating a log of the data changes. Below I list two macros written by SAS programmers as easier-to-use substitutes for PROC COMPARE. 1. %COMP_OBS macro by Kevin King, Wyeth Research, Cambridge, MA. 8 This macro is described in the paper Comparing 2 SAS Data Sets: An Alternative to Using PROC Compare available at the URL < >. This macro only requires as input the names of the two data sets being compared. The macro compares the two data sets and creates an output problem data set containing those observations that differ %EZCOMPAR macro by Doug Zirbel, Pinnacle Solutions, Indianapolis, IN. 9 This macro is described in the paper Finally An Easy Way to Compare Two SAS Files! available at the URL < >. This macro has three input parameters: the names of the two data sets being compared and an optional SORTVARS parameter. This macro can compare two files with the same name that are in different libraries. The macro is limited to comparing data sets with the same set of variables. It creates an output exception file containing any record pairs that do not exactly match. This macro is well-suited to many before-and-after testing situations. 9 Pros of Using Custom Compare Macros to Create an Audit Trail 1. These macros are easy to use. 2. These macros detect all data changes both intentional and unintentional. They will detect data changes no matter how the changes were made (e.g. it detects changes made through the data step). 3. Using these custom macros to generate an audit trail does not require additional storage space to store an audit data set. Cons of Using Custom Compare Macros to Create an Audit Trail 1. These macros are not part of the Base SAS system. Thus, these macros are not supported by the SAS Institute, Inc.. 2. These macros do not provide all the features of a complete audit trail. They do not store information about failed appends or observations rejected by integrity constraints. They do not provide information on the reason for the changes, and it is difficult to roll back data changes using their output. USING CUSTOM APPLICATIONS Another technique for producing an audit trail for all data changes is to only users to change data through a custom application that is programmed to generate an audit trail of all data changes. In his paper User-Interface Tool Choice and Audit Trail Tool Choice for a SAS Based Data Entry/Verify System for Clinical Trials Data, Barry R. Cohen of Planning Data Systems, Inc. in Ardmore, PA describes a system that his company has written. 10 This system is a double-key data entry/verify system for pharmaceutical clinical trials data. This system uses SAS/AF as a GUI front-end. The users of the system can only interact with the data through the GUI front-end. The system generates an audit trail of all data changes. Interestingly, Barry Cohen chose not to use the native SAS audit trail facility in their application. He was concerned about the size of the SAS audit trail data set. Plus, his customers wanted the ability to store reason why information for each variable value that changed. In order 7

8 to have this ability, Barry Cohen s application creates an audit data set with one record per variable changed. Barry Cohen s paper is available at URL < www2.sas.com/proceedings/sugi27/p pdf >. Pros of Using Custom Applications to Create an Audit Trail 1. You can develop your application so that it produces any type of audit trail you want. 2. You can develop a user-friendly interface to the data for your users. Cons of Using Custom Applications to Create an Audit Trail 1. You must spend the time and money to develop the custom application. 2. Your custom application is not part of the Base SAS system. So you must support it yourself. USING THE SAS DRUG DEVELOPMENT PRODUCT SAS has a new product called the Drug Development Product. This product requires an additional SAS license. The Drug Development Product is a web-based platform that can store SAS data, programs, and documents. Customers can choose to store their data and other objects on a server at the SAS Institute, Inc. in Cary, NC, or on their own server. The product provides versioning, audit trails, notification of changes, strong security features, and e-signatures. SAS describes the product as a secure biomedical knowledge platform. The platform becomes the central location for all information about that particular compound, including but not limited to pharmacokinetic, clinical and pre-clinical data; tables; reports; analysis systems; text and graphic documents; journal articles; and competitive analysis. 11 This product was specifically designed to address 21 CFR Part 11 issues. Audit trails of all objects within the product are an integral part of the product. According to SAS, the audit trail keeps a readable and printable history of all modifications made to an object including creation, changes, deletion, and electronic signatures. 12 The Drug Development Product has been available for about one year. At the time of the writing of this paper, SAS has twelve customers for this product. Here are links to information about the product: An Overview - SAS Drug Development Intelligent Research Management SAS Drug Development A solution for addressing 21 CFR Part 11 compliance CONCLUSION There are several different methods for producing audit trails for SAS data set changes. This paper provides an overview of the most commonly used methods. REFERENCES (1) Franklin, Gary and Jensen, Art (2000), Integrity Constraints and Audit Trails Working Together, SUGI 25 Paper 8 March < www2.sas.com/proceedings/sugi25/25/aa/25p008.pdf >. (2) FDA, 21 CFR Part 11, Electronic Records; Electronic Signatures; Final Rule. Federal Register Vol. 62, No. 54, 13429, March 20, (3) FDA, Guidance for Industry, Part 11, Electronic Records; Electronic Signatures Scope and Application, February 20, < > 8

9 (4) SAS Institute Inc. (1999), SAS OnlineDOC, Version 8, Chapter 28, SAS Data Files; pp , Cary, NC: SAS Institute Inc. (5) Thielbar, Melinda. Hey, Who Changed My Data? SAS Bits & Bytes. February < (May 6, 2003). (6) SAS Institute Inc. (1999), SAS OnlineDOC, Version 8, Chapter 9, The COMPARE Procedure; pp , Cary, NC: SAS Institute Inc. (7) Reiss, Maria (2003). Dare to Compare Tailoring PROC COMPARE Output, SESUG (8) King, Kevin (2000). Comparing 2 SAS Data Sets: An Alternative to Using PROC Compare, NESUG < >. (9) Zirbel, Doug (2002). Finally An Easy Way to Compare Two SAS Files! SUGI 27 Paper April < >. (10) Cohen, Barry R. (2002). User-Interface Tool Choice and Audit Trail Tool Choice for a SAS Based Data Entry/Verify System for Clinical Trials Data, SUGI 27 Paper April < www2.sas.com/proceedings/sugi27/p pdf >. (11) SAS Drug Development Intelligent Research Management. SAS Institute, Inc. White Paper < > (12) SAS Drug Development A solution for addressing 21 CFR Part 11 compliance. SAS Institute, Inc. White Paper < > CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: Maria Y. Reiss Wyeth Research 500 Arcola Road Collegeville, PA reissm@wyeth.com Phone: (484) SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. 9