Moving Beyond the Data. Using SAS Clinical Standards Toolkit, Version 1.2

Transcription

1 Moving Beyond the Data Using SAS Clinical Standards Toolkit, Version MAY2010

2 Copyright 2010, d-wise Technologies, Raleigh, NC, USA. All rights reserved. Printed in the United States of America. Use or reuse of this publication requires prior written permission of the publisher, d-wise Technologies. These materials are copyrighted. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of, or licensed to, SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. Microsoft and all other Microsoft product or service names are registered trademarks or trademarks of, or licensed to, Microsoft in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. All questions and comments about this training material should be directed to d-wise Technologies 4020 Westchase Blvd Suite 527 Raleigh, North Carolina Telephone: (919)

3 Table of Contents 1. Introduction to the Clinical Standards Toolkit 1.1 About This Training Overview of Functionality Installation Our Installation Experiences Documentation CST Folder Structure and Organization Framework Global Library Samples Understanding How to Run CST SDTM Validation Conversion to CRT-DDS Representation CRT-DDS Validation Conversion to define.xml Running SDTM Validation Checks 2.1 Objectives Understanding the Inputs and Outputs The Input Files and the Program Study Metadata Tables Control Files The List of Checks to Run References to all the Pieces Validation Properties Creating a Program to Run the Validation Process Understanding How the Pieces Fit Together Running the Validation Process Strategies for Running Validation Checks Creating the Study Metadata Tables Sorting Through the List of Validation Checks Creating a Subset of Validation Checks Cleaning up After a Run Revisiting the sasreferences Table Dynamically Generating Study Metadata More on Properties Making Sense of the Results Table i

4 3. Creating a define.xml File 3.1 Objectives Creating CRT-DDS Data from a Study Understanding the Inputs and Outputs Input Files and the Program Metadata Tables Study Control Table sasreferences Table Program to Run the Conversion Process Understanding How the Pieces Fit Together Running the Conversion Process Validating the CRT-DDS Data Understanding the Inputs and Outputs Input Files and the Program Metadata Tables List of Validation Checks to Run sasreferences Table Program to Run the Validation Process Understanding How the Pieces Fit Together Running the CRT-DDS Validation Process Creating a define.xml File Understanding the Inputs and Outputs Input Files and the Program sasreferences Table Program to Run the define.xml Process Understanding How the Pieces Fit Together Running the Program Validating the define.xml File Inputs and the Program sasreferences Table Program to Validate the define.xml Running the Program Controlling Log Level of XML Messages Using CST Macros 4.1 Overview and Objectives Macros for Managing Standards and Tables cst_createds cst_createstudyfromstandard cst_createtablesfordatastandard cst_getregisteredstandards cst_getstandardmetadata cst_getstandardsasreferences cst_registerstandard ii

5 cst_setstandardversiondefault cst_unregisterstandard Macros for Running CST Processes cst_setproperties cst_setstandardproperties cstutil_allocatesasreferences cstutil_cleanupcstsession Macros for Validation Reporting cstutil_createreport Examples of Using CST Macros Example 1: Create Validation Reports Example 2: Get a List of Registered Standards Example 3: Create Metadata and Control Tables from a Standard Example 4: Get Copies of a Standard s Metadata Tables Example 5: Create Empty Domain Tables from a Standard Example 6: Building and Registering a Validation Standard What Happens when you Register a Standard Tables and Files you need to Edit for a New Standard Steps for Creating a New Standard Step 1: Create a Folder Structure for the Standard Step 2: Create Metadata Tables for the Standard s Tables and Columns Step 3: Create a Table of Validation Checks for the Standard Understanding Table Scope and Column Scope Step 4: Copy the Properties Files from an Existing Standard Step 5: Create a sasreferences Table for the Standard Step 6: Create a standards table for the Standard Step 7: Register the Standard to CST iii

6 Course Description This instructor-based course is for technical users including biostatisticians, programmers and clinical data managers who have prior SAS software knowledge, have a technical background, and want to learn the functionality of SAS Clinical Standards Toolkit. Course topics include an overview of SAS Clinical Standards Toolkit functionality how to run SDTM validation checks on study data how to create define.xml files how to validate define.xml files how to use macros provided by SAS Clinical Standards Toolkit. Prerequisites Prior to this course, you should be able to do the following: understand basic clinical data structures understand the basics of SDTM data standards write SAS programs and understand SAS program statements understand basic SAS macros access and view SAS data tables and SAS catalogs Sample Data and Programs Sample data and programs are available from the d-wise web site at iv

7 1 Introduction to the Clinical Standards Toolkit Version MAY About This Training Overview of Functionality Installation Our Installation Experiences Documentation CST Folder Structure and Organization Framework Global Library Samples Understanding How to Run CST SDTM Validation Conversion to CRT-DDS Representation CRT-DDS Validation Conversion to define.xml

8 1. Introduction to the Clinical Standards Toolkit 1.1 About This Training This training covers SAS Clinical Toolkit, Version 1.2. It assumes that you have Base SAS installed on your PC under MS Windows have working knowledge of SAS programming and macros know how to open and edit SAS data tables. 1-2

9 Using the Clinical Standards Toolkit Version MAY Overview of Functionality The SAS Clinical Standards Toolkit (CST) consists of a database and a SAS macro library for managing and using clinical standards. CST provides a number of utility macros for managing standards and assisting in building control tables required to run CST validation and conversion processes. Key functionality: validate study tables against CDISC SDTM create a representation of a define.xml file as SAS tables (CRT-DDS) validate CRT-DDS SAS tables create a define.xml file validate a define.xml file register and manage custom standards. Standards provided by CST: CDISC-SDTM CRT-DDS 1.0 a snapshot of CDISC Controlled Terminology from October of Validation checks provided for study tables: CDISC SDTM define.xml files based CDISC CRT-DDS

10 1. Introduction to the Clinical Standards Toolkit Source of Check Number of Checks WebSDM only 6 Janus only 3 JanusFR only 7 WebSDM and Janus 46 WebSDM and JanusFR 50 WebSDM, Janus and SAS 1 SAS only NOTE: The underlying SAS code for the checks represents SAS s interpretation of the checks. 1-4

11 Using the Clinical Standards Toolkit Version MAY Installation CST is available under MS Windows for SAS and 9.2, and under UNIX for SAS 9.2. CST is available at no cost. For SAS 9.1.3, CST can be downloaded from the SAS website at For SAS 9.2, CST can be requested through your SAS representative. In order to install and run CST, the java runtime environment (JRE) must be installed on your PC. For SAS 9.1.3, hotfix E9BC95 must be installed in order to process dates in the ISO 8601 format. Refer to the hotfix at Our Installation Experiences CST is typically installed in your SAS installation folder. Installation uses Apache Ant scripts to copy and edit files. Ant is an open-source Java-based build tool from the Apache Software Foundation. Ant uses build files, which are XML files that tell the Ant program what to do. Because Ant is a Java program, it's a cross-platform tool. For SAS 9.2 Apache Ant is automatically installed and set up for you, and CST installation seems to be pretty straightforward. For SAS You have to download and install Apache Ant ( Installation can be a bit challenging, especially if your SAS installation is in a non-default location (e.g. someplace other than C:\Program Files\SAS) Be sure to carefully read the installation doc!! If all else fails, it is possible to manually install CST. You have to extract files from the installation ZIP file and create the proper folder structure. You have to rebuild transport files on your own. You have to manually edit SAS config files. 1-5

12 1. Introduction to the Clinical Standards Toolkit NOTE: d-wise can provide some documentation and sample programs for doing a manual installation if you get stuck. See the d-wise web site Documentation Two sources for documentation: SAS Clinical Standards Toolkit 1.2: User's Guide SAS Clinical Standards Toolkit 1.2: Quick Start for SDTM Validation for SAS 9.2 and SAS

13 Using the Clinical Standards Toolkit Version MAY CST Folder Structure and Organization CST consists of three major components: CST Framework contains the bulk of the validation check macros and utility macros. It also contains the java files needed for creating a define.xml. The CST Framework is installed in a folder named cstframework. Global Library Samples contains registered standards and any standard specific macros. It is installed in a folder named cstgloballibrary. contains examples of input tables, results, and programs for running CST processes. The sample programs are used as part of the installation qualification. There are two sets of samples: one for SDTM and one for CRT- DDS. These folders are typically installed in the same location as the CST Framework and the Global Library. Look for two folders with names that begin with SASClinicalStandardsTookit. The folder structure for the CST framework and global library. 1-7

14 1. Introduction to the Clinical Standards Toolkit Framework The Framework is installed in a folder named cstframework. Subfolders under cstframework: sasmacro contains SAS macros for utility tasks and for starting and running CST. Global Library The Global Library is installed in a folder named cstgloballibrary. Subfolders under cstgloballibrary: metadata contains the SAS tables with information on registered standards. schema-repository contains schemas for XML-based standards supported by CST. standards xls-repository contains a subfolder for each registered standard. Standards provided with CST include SDTM, CRT-DDS, Controlled Terminology and CST Framework. contains subfolders for standards that support XML. These subfolders contain XML stylesheets used for reading and writing XML files. Samples Two folders with examples are created as part of installation. You have to drill down a bit to get to the actual samples. 1-8

15 Using the Clinical Standards Toolkit Version MAY Understanding How to Run CST A number of files and tables have to be defined properly and referenced properly in order to get CST to generate results. The rest of this course focuses on how to set up the files and tables to run the tasks. The sections below provide a quick high-level overview. SDTM Validation Run SDTM checks against your study data and metadata. SDTM Validation Processing Study Data Study Metadata Run-time Control Files Standards Metadata Validation Processing Metrics and Results Tables Controlled Terminology Conversion to CRT-DDS Representation Read SDTM data and metadata to create SAS tables that represent the structure of a define.xml file. SDTM-to-CRT-DDS Processing Study Data Study Metadata SAS Tables that represent define.xml structure Run-time Control Files Convert Processing Results Table Controlled Terminology 1-9

16 1. Introduction to the Clinical Standards Toolkit CRT-DDS Validation Run checks against SAS tables representing CRT-DDS tables to validate that a proper define.xml file can be created. CRT-DDS Validation Processing SAS Tables that represent define.xml structure Metadata For Tables Validation Processing Metrics and Results Tables Run-time Control Files Conversion to define.xml Read SAS tables that represent CRT-DDS formatted data and metadata to create define.xml file. define.xml Creation CRT-DDS Data Run-time Control Files define.xml Processing Define.xml Results Table 1-10

17 2 Running SDTM Validation Checks Version MAY Objectives Understanding the Inputs and Outputs The Input Files and the Program Study Metadata Tables Control Files The List of Checks to Run References to all the Pieces Validation Properties Creating a Program to Run the Validation Process Understanding How the Pieces Fit Together Running the Validation Process Strategies for Running Validation Checks Creating the Study Metadata Tables Sorting Through the List of Validation Checks Creating a Subset of Validation Checks Cleaning up After a Run Revisiting the sasreferences Table Dynamically Generating Study Metadata More on Properties Making Sense of the Results Table

18 2. Running SDTM Validation Checks 2.1 Objectives Understand the input files and tables required for running SDTM validation checks. Know how to build input files and tables for a study. Know how to create a SAS program to run SDTM validation checks. Understand the output generated from a running SDTM validation checks. 2-2

19 Using the Clinical Standards Toolkit Version MAY Understanding the Inputs and Outputs In order to run SDTM checks against your study data and metadata, you have to understand the input files involved and the ones that you need to create. SDTM Validation Processing Study Data Study Metadata Run-time Control Files Standards Metadata Validation Processing Results Files Controlled Terminology Key Points: You have to provide metadata tables for a study s tables and columns. You can control what validation checks are run. Some validation checks compare your study metadata to a standard s metadata. Some validation checks evaluate the consistency of column values between columns, across records, and across tables. A references table tells a validation process where to find all the other files and tables needed for validation. You have to create a SAS program that calls appropriate CST macros to run checks. 2-3

20 2. Running SDTM Validation Checks 2.3 The Input Files and the Program Study Metadata Tables SDTM validation checks need a list of tables in a study to check and the columns to check. This information is provided in the form of SAS tables. The metadata control tables for SDTM validation checks contain the following information: List of Tables List of Columns provides the study tables to include in validation checks. The list is in the form of a SAS table. provides the columns from the study tables to include in validation checks. The list is in the form of a SAS table. Key Points: You must create these metadata tables for a study. The tables containing the list of study tables and the list of columns have specific structures. Some of the columns in these control tables contain the basic metadata that you would get from a PROC CONTENTS column name, label, type, length, for example. Some of the columns represent extended metadata that is specific to CDISC standards origin, qualifier, role, for example. Examples of study metadata tables: 2-4

21 Using the Clinical Standards Toolkit Version MAY10 Control Files The List of Checks to Run A validation process requires a list of validation checks to run. The list of checks is in the form of a SAS table. Key points: The validation_master table provides a list of the checks for the SDTM standards. It is located in cstgloballibrary\standards\cdisc-sdtm-3.1.1\validation\control. You can create the list of checks by copying and subsetting the master list that goes with a standard. DO NOT run all of the validation checks at one time. The output will be overwhelming. An example of a table of validation checks for the SDTM standard in CST: NOTES: Refer to Appendix 5 in the SAS documentation for details on the SDTM validation checks. References to all the Pieces The sasreferences table provides a list of pointers to the tables and files needed to run a validation check process. The sasreferences table is basically a list of librefs, filerefs, folder paths, and files to use. The list of items in the sasreferences table varies depending on the standards and the type of process you are running. There should be a record in the sasreferences table that references itself. 2-5

22 2. Running SDTM Validation Checks CST is not very forgiving and can be very picky if things don t match up exactly between what is in sasreferences and what is in the metadata tables. For example, the librefs referenced in the study metadata tables and columns tables must match the libref referecnced in the sasreferences table. An example of a sasreferences table: Notice the two macro variables used in the table: _cstgroot points to the root folder where folders for standards definitions are stored. It is set by CST initialization. _studyrootpath points to the root folder of your study data and metadata, assuming it is all under the same root study folder. Validation Properties Validation properties define how validation checks are processed. The named pairs converted to SAS macro variables. Validation properties must be loaded since the macro variables they define are used across CST macros. Validation properties can be explicitly loaded by the SAS program that runs a validation process or by referencing the validation properties file in the sasreferences table. A validation properties file is provided with the SDTM standards and can be used as the default. 2-6

23 Using the Clinical Standards Toolkit Version MAY10 Example of a validation properties file: _cstchecksortorder=_data cstmetrics=1 _cstmetricsds=work._cstmetrics _cstmetricsnumsubj=1 _cstmetricsnumrecs=1 _cstmetricsnumchecks=1 _cstmetricsnumbadchecks=1 _cstmetricsnumerrors=1 _cstmetricsnumwarnings=1 _cstmetricsnumnotes=1 _cstmetricsnumstructural=1 _cstmetricsnumcontent=1 _cstmetricscntnumsubj=0 _cstmetricscntnumrecs=0 _cstmetricscntnumchecks=0 _cstmetricscntnumbadchecks=0 _cstmetricscntnumerrors=0 _cstmetricscntnumwarnings=0 _cstmetricscntnumnotes=0 _cstmetricscntnumstructural=0 _cstmetricscntnumcontent=0 _cstmetricstimer=1 Creating a Program to Run the Validation Process Validation checks are executed on a study by running a SAS program that initializes the CST standards to use, defines the location and name of the sasreferences table to use, and runs the appropriate CST macros to process the checks. Example of a SAS program that runs validation checks: /* >>> Define a macro var that points to the study folder */ %let STUDYROOTPATH = D:\CSTStudy; /* Initialize the CST-FRAMEWORK. */ %cst_setstandardproperties( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _cstsubtype =initialize /* Initialize the CDISC-SDTM standard. */ %cst_setstandardproperties( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _cstsubtype =initialize /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &STUDYROOTPATH\sdtmFiles; %let _cstsasrefsname = sasreferences_sdtmvalidation; /* Allocate filerefs and librefs defined in sasreferences table */ 2-7

24 2. Running SDTM Validation Checks %cstutil_allocatesasreferences; /* Run the validation checks. */ %sdtm_validate; Key points about the program: At the beginning to the program, the macro variable studyrootpath is defined with a folder path. This macro variable is used in the sasreferences table to generalize the folder paths to study-specific folders. The CST framework and standards to use for the validation process are initialized before any other CST macros are run. Two CST macro variables are defined that identify the location and name of the sasreferences table to use, then a CST macro is called that allocates the librefs and filerefs defined in the sasreferences table. The CST validation macro is the last thing in the SAS program. Understanding How the Pieces Fit Together Study tables and metadata tables describe the study. A standard s metadata tables describe what a study should look like. A list of validation checks to run is an input to validation processing. A sasreferences table identifies all the input files that a validation process should use. SDTM Validation Processing Study DM AE LB source_tables source_columns sasreferences validation checks Validation Processing metrics results Standards reference_tables reference_columns validation_properties 2-8

25 Using the Clinical Standards Toolkit Version MAY Running the Validation Process Running an SDTM validation process generates two output data tables. Metrics contains a summary of the number of records processed by various checks and a count of the number of notes, warning messages and error messages generated. Use this table to get an overview of what ran and how many validation issues were encountered. NOTE: Refer to the documentation for the Clinical Standards Toolkit for details on columns in the metrics table. 2-9

26 2. Running SDTM Validation Checks Results contains the detailed results of the validation checks. Depending on the checks you included in your validation_control table and the SDTM conformity of your study, the results table can be very large. NOTE: Refer to the documentation for the Clinical Standards Toolkit for details on columns in the metrics table. 2-10

27 Using the Clinical Standards Toolkit Version MAY Strategies for Running Validation Checks Creating the Study Metadata Tables Use PROC SQL with a dictionary view to create metadata tables and populate them with the basic metadata for a study. Copy the metadata tables from the SDTM sample and edit them to match your study metadata. Use the %cst_getstandardmetadata macro to return metadata tables defined to a standard. Here is a sample program that creates and populates the basic metadata for a study: %let STDYLIB = STUDY; %let STUDYID = SASCSTDEMODATA; %let STUDYROOTPATH = D:\CSTStudy\data; %let SDTMDATA = D:\CSTStudy\sdtmfiles; %let STANDARD = CDISC-SDTM; %let STDVERSION = 3.1.1; %let STUDYDESC = CST Study; %let STUDYDS = &STUDYID._study; %let TABLESDS = &STUDYID._tables; %let COLUMNSDS = &STUDYID._columns; /* Point to where study data tables live */ libname &STDYLIB "&STUDYROOTPATH"; /* Point to where to write metadata tables */ libname sdtm "&SDTMDATA"; /* <<<<< THIS SECTION DEFINES BASIC TABLE AND COLUMN METADATA >>>>>*/ proc sql; /* Create list of study tables for SDTM metadata */ create table &TABLESDS as select libname as sasref, memname as table, memlabel as label, "&STANDARD" as standard length=20, "&STDVERSION" as standardversion length=20 from dictionary.tables where libname="&stdylib" ; /* Create list of study columns for SDTM metadata */ create table &COLUMNSDS (rename=(coltype=type)) as select libname as sasref, memname as table, 2-11

28 2. Running SDTM Validation Checks name as column, label, case when type="char" then "C" when type="num" then "N" end as coltype length=1, length, varnum as order, "&STANDARD" as standard length=20, "&STDVERSION" as standardversion length=20 from dictionary.columns where libname="&stdylib"; /* Create a descriptor table for the study */ create table sdtm.&studyds (OID char(64), Studyname char(128), DefineDocumentName char(64), SASRef char(8), ProtocolName char(15), StudyDescription char(100) insert into sdtm.&studyds set Studyname ="Test Study", DefineDocumentName="define", SASRef ="STUDY", ProtocolName ="&STUDYID", StudyDescription ="&STUDYDESC" ; quit; /* <<<<< THIS SECTION DEFINES EXTENDED TABLE AND COLUMN METADATA >>>>> */ /* Initialize the standard to use for macro calls below */ %cst_setstandardproperties( _cststandard =&STANDARD, _cststandardversion=&stdversion, _cstsubtype =initialize /* Create an empty sasreferences table */ %cst_createds( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _csttype =control, _cstsubtype =reference, _cstoutputds =sasrefs /* Add 2 records for references tables and columns */ proc sql; insert into sasrefs set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='table', sasref ='work', 2-12

29 Using the Clinical Standards Toolkit Version MAY10 reftype memname ='libref', ='reftables' ; quit; set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='column', sasref ='work', reftype ='libref', memname ='refcolumns' /* Get tables and columns defined to standards */ %cst_getstandardmetadata( _cstsasreferences =work.sasrefs, _cstresultsoverrideds=_cstresults /* Add extended metadata from standards to study metadata */ proc sql; create table sdtm.&tablesds as select mystudy.*, ref.keys, ref.standardref, ref.class, ref.structure, ref.purpose, ref.state, ref.date, ref.xmlpath, ref.xmltitle, ref.comment from &TABLESDS as mystudy left join work.reftables as ref on mystudy.table=ref.table; create table sdtm.&columnsds as select mystudy.*, ref.core, ref.qualifiers, ref.origin, ref.role, ref.term, ref.xmlcodelist, ref.xmldatatype, ref.displayformat, ref.comment from &COLUMNSDS as mystudy left join work.refcolumns as ref on mystudy.table=ref.table and mystudy.column=ref.column; drop table reftables, refcolumns, sasrefs, &TABLESDS, &COLUMNSDS; 2-13

30 2. Running SDTM Validation Checks quit; libname study clear; libname sdtm clear; Sorting Through the List of Validation Checks The validation_master table in cstgloballibrary\standards\cdisc-sdtm \validation\control provides a list of the validation checks for the SDTM standards, but it does not contain any descriptions of what the checks do. The messages table in the folder cstgloballibrary\standards\cdisc-sdtm-3.1.1\messages contains the source ID as well as a source description column. You can join the validation_master table and the messages table to create a list of checks and their descriptions. libname sdtmchks "E:\cstGlobalLibrary\standards\cdisc-sdtm-3.1.1\validation\control"; libname sdtmmsgs "E:\cstGlobalLibrary\standards\cdisc-sdtm-3.1.1\messages"; proc sql; create table work.sdtm_checklist as select distinct val.checkid format=$8., val.checktype format=$15., msg.sourcedescription, val.checksource format =$10., val.sourceid format=$8., val.tablescope, val.columnscope from sdtmchks.validation_master as val right join sdtmmsgs.messages as msg on val.checkid = msg.resultid ; quit; libname sdtmchks ""; libname sdtmmsgs ""; Example of the joined validation_master and messages tables: 2-14

31 Using the Clinical Standards Toolkit Version MAY10 Creating a Subset of Validation Checks Trying to run all validation checks at one time can create an overwhelming amount of output. Consider creating a set of validation check tables that you use in a standard validation check process. There are several columns in the validation_master table that you can use for subsetting the validation checks. It is located in cstgloballibrary\standards\cdisc-sdtm \validation\control. CheckSource CheckType CheckID provides a way to subset the checks based on the source of the check Janus, JanusFR, WebSDM, or SAS. provides categories of data or metadata that a check processes. Possible values are CntlTerm Column, ColumnAttribute, ColumnValue, Date, Metadata, Multirecord, Multitable. provides a way to select specific checks based on the check ID. Example of a program to subset the validation checks: %let SDTMDATA = D:\CSTStudy\sdtmfiles; %let SDTMVAL = E:\cstGlobalLibrary\standards\cdisc-sdtm-3.1.1\validation\control; %let CHECKSDS = sdtmchecks ; /* CheckSource Values: Janus JanusFR SAS WebSDM */ %let WHERECHKSOURCE = checksource in("janus", "JanusFR", "SAS" ) ; /* CheckType Values: CntlTerm Column ColumnAttribute ColumnValue Date Metadata Multirecord Multitable */ * %let WHERECHKTYPE = checktype in("columnattribute" %let WHERECHKIDS=checkid in("sdtm0001", "SDTM0022", "SDTM0023", "SDTM0030", "SDTM0101" libname sdtm "&SDTMDATA"; libname sdtmval "&SDTMVAL"; proc sql; create table sdtm.&checksds as select * from sdtmval.validation_master where &WHERECHKSOURCE and &WHERECHKIDS ; quit; libname sdtm ""; libname sdtmval ""; 2-15

32 2. Running SDTM Validation Checks Cleaning up After a Run During a CST session, a number of librefs and filrefs may be defined as well as macro variables and SAS options. If you are running CST processes as well as other SAS programs in the same SAS session, you may want to clean up CST artifacts before running other SAS tasks. The cstutil_cleanupcstsession macro provides a way to clean up after running a program that uses CST macros. Example of running the cstutil_cleanupcstsession macro: /* Clean up things created by CST. Comment out if you want to look at the stuff created by run */ *%cstutil_cleanupcstsession( _cstclearcompiledmacros=0, _cstclearlibrefs=0, _cstresetsasautos=0, _cstresetfmtsearch=0, _cstresetsasoptions=1, _cstdeletefiles=1, _cstdeleteglobalmacrovars=0 Refer to the CST documentation for more information on the cstutil_cleanupcstsession macro. This macro is also discussed in Module 4, Overview of CST Macros, of this training course. 2-16

33 Using the Clinical Standards Toolkit Version MAY10 Revisiting the sasreferences Table Question: Editing the sasreferences table for a validation process looks like a lot of work. Is there an easier way to do this? Answer: Reference macro variables in your sasreferences table. Set the macro variables in your program that runs the validation process. Here is an example of a sasreferences table that contains a number of macro variable references. Here is a revised SAS program to run a validation process. /* >>> Define macro vars that point to data and metadata */ %let STUDYID = SASCSTDEMODATA; %let STUDYROOTPATH = D:\CSTStudy; %let STDYLIB = STUDY; %let STUDYDATA = &studyrootpath\data; %let TABLESMETADS = &STUDYID_tables; %let COLUMNSMETADS = &STUDYID_columns; %let CHECKSDS = sdtmchecks; %let METRICSDS = metrics; %let RESULTSDS = results; %let SASREFSDS = better_sasreferences /* Set up the CST-FRAMEWORK standard. */ %cst_setstandardproperties( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _cstsubtype =initialize /* Set up the CDISC-SDTM standard. */ %cst_setstandardproperties( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _cstsubtype =initialize 2-17

34 2. Running SDTM Validation Checks /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &STUDYROOTPATH\sdtmfiles; %let _cstsasrefsname = &SASREFSDS; /* Allocate filerefs and librefs defined in sasreferences table */ %cstutil_allocatesasreferences; /* Run the validation checks. */ %sdtm_validate; Key points: Notice how macro variables set in the program are referenced in the sasreferences table. You can create generalized sasreferences tables that could be used with a number of studies. You can change the names of the metadata tables to use and the name of the validation checks table without having to edit the sasreferences table. Dynamically Generating Study Metadata Question: Why do I have to create study metadata tables? Can t CST figure it out at runtime? Answer: The current version of CST can t figure it out, but you can add your own macro to do it. The example below is a macro that creates metadata tables for a study. You can use this macro in a validation program to dynamically create metadata tables for a study if they do not exist. %* addon_create_study_metadata *; %* Purpose : Creates metadata tables for study domains *; %* Parameters: *; %* STUDYLIB Libref to use for study. *; %* STUDYID Study ID for prefixing metadata table names. *; %* STUDYDESC Study Description used in study metadata table. *; %* STUDYDATAPATH Path where study data is located. *; %* SDTMDATAPATH Path for writing metadata. *; %* DOMAINS blank or list of domains separated by a space. *; %* NEW blank or Y to re-create metadata tables if they *; %* already exist. *; %macro addon_create_study_metadata (STUDYLIB=, STUDYID=, STUDYDESC=, STUDYDATAPATH=, SDTMDATAPATH=, DOMAINS=, NEW= %let STANDARD = CDISC-SDTM; %let STDVERSION = 3.1.1; %let STUDYDS = &STUDYID._study; %let TABLESDS = &STUDYID._tables; %let COLUMNSDS = &STUDYID._columns; 2-18

35 Using the Clinical Standards Toolkit Version MAY10 %let WHERE = ; %* Point to where to write metadata tables ; libname sdtm "&SDTMDATAPATH"; %if &NEW= %then %do; %put ; %let rc=%sysfunc(exist(sdtm.&studyid._study,data) %if &rc=1 %then %put NOTE: sdtm.&studyid._study already exists.; %if &rc=0 %then %do; %let rc=%sysfunc(exist(sdtm.&studyid._tables,data) %if &rc=1 %then %put NOTE: sdtm.&studyid._tables already exists.; %end; %if &rc=0 %then %do; %let rc=%sysfunc(exist(sdtm.&studyid._columns,data) %if &rc=1 %then %put NOTE: sdtm.&studyid._columns already exists.; %end; %if &rc=1 %then %do; %put NOTE: At least one study metadata table exists. Not re-creating them.; %put ; %goto exit; %end; %end; %if %length(&domains)>0 %then %do; %let count=0; %let WHERE = and MEMNAME in(; %do %until(&table= %let count=%eval(&count + 1 %let table=%scan(&domains,%eval(&count) %if %length(&table)>0 %then %do; %if &count=1 %then %let WHERE = &WHERE "&table"; %if &count>1 %then %let WHERE = &WHERE,"&table"; %end; %end; %if %length(&where)>0 %then %let WHERE = &WHERE %str(%) %end; %* Point to where study data tables live ; libname &STUDYLIB "&STUDYDATAPATH"; proc sql; < See section Strategies for Creating Metadata Tables for remainder of code > 2-19

36 2. Running SDTM Validation Checks Example of the macro call: %addon_create_study_metadata( STUDYLIB = study, STUDYID = sascstdemodata, STUDYDESC = CST Study, STUDYDATAPATH = d:\cststudy\data, SDTMDATAPATH = d:\cststudy\sdtmfiles, DOMAINS = DM VS, NEW = Y More on Properties Properties are macro var = value pairs that are created at run-time. Each registered standard has an initialize.properties file. A standard may also have an action-related properties file like validation.properties. A property can be set using a %let statement, but it has to be located in the correct place in the SAS program. Properties files are processed by referencing the properties file in the sasreferences table with type=properties. calling the cst_setproperties macro. Most useful macro variable properties: CST Framework initialize properties Name Values Description _cstdebug 0 1 Sets system options for macro debugging. Default is 0. _cstdebugoptions mprint molgic symbolgen System options set when _cstdebug=1. mautolocdisplay _cstresultsds Name of results table. Default is work._cstresults _cstreallocatesasrefs 0 1 Reallocate sasrefs and filerefs defined in sareferences table. Default is 0. _cstfmtlibraries Alter format search path defined by sasreferences table. 2-20

37 Using the Clinical Standards Toolkit Version MAY10 CDISC-SDTM initialize properties Name Values Description _cstsasrefsloc Path that defines location of sasreferences table. Default is &studyroopath/control. _cstsasrefsname Name of sasreferences table to use. Other macros to be aware of Name Values Description _cstgroot Path that defines location of the cstgloballibrary folder. CST automatically sets this macro. studyrootpath Defines root path for study data and metadata. It is typically referenced in the sasreferences table. This is usually set in your validation program. Making Sense of the Results Table CST generates a SAS table that contains the results of validation checks. The table itself is not in a format is easy to interpret. Example of a results table from a CDISC-SDTM validation process: By using some simple reporting tools, you can create output that is easier to understand. Example of a report program for results table: proc sql; create table tmpresults as select *, scan(srcdata,-1) as tmpsrcdata length=10, case upcase(resultseverity) when "NOTE" then 1 when "WARNING" then 2 when "ERROR" then 3 else 0 end as numseverity from samples.checkresults order by tmpsrcdata, numseverity 2-21

38 2. Running SDTM Validation Checks quit; ; title "Messages Grouped by Domain and Severity"; options ls=120 ps=60 pageno=1 nodate; proc report data=tmpresults nowd ; where checkid ne "" ; column tmpsrcdata numseverity resultseverity message actual checkid; define tmpsrcdata / group "Source /======" format=$20.; define numseverity / group noprint ; define resultseverity / display "Severity/========" format=$8.; define message / "Message /=======" width=38 flow; define actual / "Details /=======" width=38 flow; define checkid / "Check ID/========" format=$8.; run; break after numseverity / skip; Report generated by the program: NOTE: CST provides a reporting macro for generating reports from metrics and results tables. This macro is covered Module 4, Overview of CST Macros, of this training course. 2-22

39 3 Creating a define.xml File Version MAY Objectives Creating CRT-DDS Data from a Study Understanding the Inputs and Outputs Input Files and the Program Metadata Tables Study Control Table sasreferences Table Program to Run the Conversion Process Understanding How the Pieces Fit Together Running the Conversion Process Validating the CRT-DDS Data Understanding the Inputs and Outputs Input Files and the Program Metadata Tables List of Validation Checks to Run sasreferences Table Program to Run the Validation Process Understanding How the Pieces Fit Together Running the CRT-DDS Validation Process Creating a define.xml File Understanding the Inputs and Outputs Input Files and the Program sasreferences Table Program to Run the define.xml Process Understanding How the Pieces Fit Together Running the Program Validating the define.xml File Inputs and the Program sasreferences Table Program to Validate the define.xml Running the Program Controlling Log Level of XML Messages

40 3. Creating a define.xml File 3.1 Objectives Understand that creating a define.xml file from your SDTM study metadata is a two-step process: 1. Create CRT-DDS SAS tables from your SDTM study metadata. 2. Create a define.xml file from CRT-DDS SAS tables. Understand the input files and tables required for creating SAS tables that represent a define.xml file. Know how to build input files for creating CRT-DDS tables. Know how to run CRT-DDS validation checks on the SAS tables. Know how to create a define.xml file from CRT-DDS tables. Know how to run validation checks on the define.xml file. 3-2

41 Using the Clinical Standards Toolkit Version MAY Creating CRT-DDS Data from a Study Understanding the Inputs and Outputs SDTM to CRT-DDS Processing Study Metadata SAS Tables that represent define.xml structure Run-time Control Files Convert Processing Results Table CRT-DDS Standards Key Points: You have to provide metadata tables for a study s tables and columns. A references table tells a CRT-DDS creation process where to find all the other files and tables needed for the process. You have to create a SAS program that calls appropriate CST macros to run the creation process. 3-3

42 3. Creating a define.xml File Input Files and the Program Metadata Tables The SDTM-to-CRTDDS conversion process requires three metadata tables: the metadata table with the list of domain tables (used for SDTM validation) the SDTM metadata table with the list of columns (used for SDTM validation) a metadata table that describes the study. NOTE: This is where it is important to have all columns in the metadata tables as complete as possible. Completeness of the columns will affect how the output tables are created and how the define.xml file is created. Study Control Table The study metadata table contains columns that provide information on each study. Multiple studies can be referenced in this table by using different values for the sasref column for each study. The screenshot below shows an example of a study control table needed by the SDTM-to-CRT- DDS conversion process. The program below is an example of how to create a table that contains metadata information about the study to be processed by the SDTM-to-CRT-DDS conversion. %let STUDYID = SASCSTDEMODATA; %let STDYLIB = STUDY; proc sql; create table &STUDYID._study ( Studyname char(128), DefineDocumentName char(64), SASRef char(8), ProtocolName char(20), StudyDescription char(100) 3-4

43 Using the Clinical Standards Toolkit Version MAY10 insert into &STUDYID._study set Studyname ="Test Study", DefineDocumentName="define", SASRef ="&STDYLIB", ProtocolName ="&STUDYID", StudyDescription ="CST Study" ; quit; sasreferences Table The sasreferences table provides a list of pointers to the tables and files needed to run a SDTM to CRT-DDS conversion process. The sasreferences table is basically a list of librefs, filerefs, folder paths, and files to use. The list of items in the sasreferences table varies depending on the standards and the type of process you are running. There should be a record in the sasreferences table that references itself. Example of a sasreferences table for SDTM to CRT-DDS conversion: Program to Run the Conversion Process Creation of CRT-DDS tables involves running a SAS program that initializes the CST standards to use, defines the location and name of the sasreferences table to use, and runs the appropriate CST macros to create the tables. Example of a SAS program that runs the conversion process: /* Set some macro vars */ %let STUDYROOTPATH = D:\CSTStudy; %let STUDYID = sascstdemodata; /* Librefs for pointing to input and output locations */ libname sdtm "&STUDYROOTPATH\sdtmFiles"; libname crtdds "&STUDYROOTPATH\crtddsdata"; 3-5

44 3. Creating a define.xml File /* Set up the CST-FRAMEWORK standard. */ %cst_setstandardproperties( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _cstsubtype =initialize /* Set up the CST_CRTDDS standard */ %cst_setstandardproperties( _cststandard =CDISC-CRTDDS, _cststandardversion=1.0, _cstsubtype =initialize /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &STUDYROOTPATH\crtddsFiles; %let _cstsasrefsname = sasreferences_convert; /* Define where to write results */ %let _cstresultsds = crtdds.results_convert; /* Allocate filerefs and librefs defined in sasreferences table */ %cstutil_allocatesasreferences; %crtdds_sdtm311todefine10( _cstoutlib =crtdds, _cstsourcetables =sdtm.&studyid._tables, _cstsourcecolumns=sdtm.&studyid._columns, _cstsourcestudy =sdtm.&studyid._study libname sdtm clear; libname crtdds clear; Key points about the program: At the beginning to the program, the macro variable studyrootpath is defined with a folder path. This macro variable is used in the sasreferences table to generalize the folder paths to study-specific folders. The CST framework and standards to use for the validation process are initialized before any other CST macros are run. Two CST macro variables, _cstsasrefloc and _cstsasrefname, are defined that identify the location and name of the sasreferences table to use, then a CST macro is called that allocates the librefs and filerefs defined in the sasreferences table. The CST macro variable _cstresultsds is defined to redirect the results table to a permanent location. The crtdds_sdtm311todefine10 macro is the last thing in the SAS program. 3-6

45 Using the Clinical Standards Toolkit Version MAY10 Understanding How the Pieces Fit Together Metadata tables describe the study, study tables, and study columns, and provide the input for the CRT-DDS tables. A sasreferences table identifies all the input files that a validation process should use. SDTM to CRT-DDS Processing Study source_study source_tables CRT-DDS Tables clitemdecodetranslatetext codelistitems codelists source_columns definedocument itemdefs sasreferences Conversion Processing itemgroupdefitemrefs itemgroupdefs metadataversion study Standards CRT-DDS results Running the Conversion Process The crtdds_sdtm31todefine10 macro reads the table and column metadata tables for a study and writes out this metadata in the form of SAS tables that represent components of a define.xml file. The nine basic tables: definedocument study metadataversion itemgroupdefs itemgroupdefsitemrefs itemdefs codelists codelistitems clitemdecodetranslatedtext NOTE: To create shells for all 39 CRT-DDS tables use the cst_createtablesfordatastandard macro. 3-7

46 3. Creating a define.xml File Example of how to create the 39 CRT-DDS table shells: %cst_createtablesfordatastandard( _cststandard= CDISC-CRTDDS,_cstStandardVersion= 1.0,_cstOutputLibrary= work The example below shows the skeleton parts of a define.xml file. You can see how the tags in the file correspond to one of the nine SAS tables above. <ODM > <Study> <GlobalVariables> <StudyName> CST Study </StudyName> <StudyDescription> Test Study </StudyDescription> <ProtocolName> cstsasdemodata </ProtocolName> </GlobalVariables> <MetaDataVersion Name="CDISC-SDTM 3.1.1" > <ItemGroupDef Name="CO" > <ItemRef ItemOID="COL1" > <ItemRef ItemOID="COL2" > </ItemGroupDef > <ItemDef OID="COL1" Name="CODTC" > <ItemDef OID="COL2" Name="COEVAL" > </MetaDataVersion> </Study> </ODM> The conversion process also generates a results table. The example below shows a results table generated by the previous SAS program. 3-8

47 Using the Clinical Standards Toolkit Version MAY Validating the CRT-DDS Data Understanding the Inputs and Outputs Validation of CRT-DDS data is similar to the validation of SDTM data: Metadata tables describing the CRT-DDS tables and columns are required. A series of validation checks are available and you can subset them to a manageable number. You run the validation check process by using a SAS program with CST macro calls. Beware of the differences between SDTM validation and CRT-DDS validation: The tables and columns metadata tables reference the CRT-DDS tables. The validation checks run against the CRT-DDS tables and columns metadata tables. They do not reference the actual study domain tables. CRT-DDS Validation Processing CRT-DDS Data CRT-DDS Metadata Run-time Control Files Standards Metadata Validation Processing Metrics and Results Tables 3-9

48 3. Creating a define.xml File Input Files and the Program Metadata Tables CRT-DDS validation checks need a list of CRT-DDS tables to check and the columns to check. This information is provided in the form of SAS tables. The metadata control tables for CRT-DDS validation checks contain the following information: List of Tables List of Columns provides the CRT-DDS tables to include in validation checks. You can use any table name that is a valid SAS name. provides the columns from the CRT-DDS tables to include in validation checks. You can use any table name that is a valid SAS name. Examples of CRT-DDS metadata tables: 3-10

49 Using the Clinical Standards Toolkit Version MAY10 The program below is an example of how to create the tables and columns metadata tables for CRT-DDS tables. %let STDYLIB = STUDY; %let STUDYID = sascstdemodata; %let STUDYROOTPATH = D:\CSTStudy; %let STANDARD = CDISC-CRTDDS; %let STDVERSION = 1.0; %let TABLESDS = &STUDYID._tables; %let COLUMNSDS = &STUDYID._columns; /* Point to where crt-dds data tables live */ libname &STDYLIB "&STUDYROOTPATH.\crtddsdata"; /* Point to where to write metadata tables */ libname crtdds "&STUDYROOTPATH\crtddsfiles"; proc sql; /* Create list of study tables for CRT-DDS metadata */ create table &TABLESDS as select libname as sasref, memname as table, memlabel as label, "&STANDARD" as standard length=20, "&STDVERSION" as standardversion length=20 from dictionary.tables where upcase(libname)="&stdylib" ; /* Create list of study columns for CRT-DDS metadata */ create table &COLUMNSDS (rename=(coltype=type)) as select libname as sasref, memname as table, name as column, label, case when type="char" then "C" when type="num" then "N" end as coltype length=1, length, varnum as order, "&STANDARD" as standard length=20, "&STDVERSION" as standardversion length=20 from dictionary.columns where libname="&stdylib"; /* Initialize the standard to use for macro calls below */ %cst_setstandardproperties( _cststandard =&STANDARD, _cststandardversion=&stdversion, _cstsubtype =initialize /* Create an empty sasreferences table */ %cst_createds( _cststandard =CST-FRAMEWORK, 3-11

50 3. Creating a define.xml File _cststandardversion=1.2, _csttype =control, _cstsubtype =reference, _cstoutputds =sasrefs /* Add 2 records for references tables and columns */ proc sql; insert into sasrefs set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='table', sasref ='work', reftype ='libref', memname ='reftables' set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='column', sasref ='work', reftype ='libref', memname ='refcolumns' ; quit; /* Get tables and columns defined to standards */ %cst_getstandardmetadata( _cstsasreferences =work.sasrefs, _cstresultsoverrideds=_cstresults /* Add extended metadata from standards to CRT-DDS metadata */ proc sql; create table crtdds.&tablesds as select mystudy.*, ref.keys, ref.standardref, ref.class, ref.xmlelementname, ref.qualifiers, ref.comment from &TABLESDS as mystudy left join work.reftables as ref on upcase(mystudy.table)=upcase(ref.table create table crtdds.&columnsds as select mystudy.*, ref.displayformat, ref.standardref, ref.core, ref.xmlcodelist, ref.qualifier, ref.comment from &COLUMNSDS as mystudy left join 3-12

51 Using the Clinical Standards Toolkit Version MAY10 work.refcolumns as ref on upcase(mystudy.table) =upcase(ref.table) and upcase(mystudy.column)=upcase(ref.column drop table reftables, refcolumns, sasrefs, &TABLESDS, &COLUMNSDS; quit; libname study clear; libname crtdds clear; List of Validation Checks to Run A validation process requires a list of validation checks to run. Key points: The validation_master table provides a list of the checks for the CRT-DDS 1.0 standards. It is located in cstgloballibrary\standards\crt-dds-1.0\validation\control. You can create the list of checks by copying and subsetting the master list that goes with a standard. Example of a table of validation checks for the CRT-DDS 1.0 standard in CST: There are several columns in the validation_master table that you can use for subsetting the validation checks. CheckType CheckID Tablescope provides categories of check types. Possible values are Content or Structural. provides a way to select specific checks based on the check ID. contains the name of the table to which a check applies. 3-13

52 3. Creating a define.xml File Example of a program to create a list of validation checks for the nine basic CRT-DDS tables: %let CRTDDSVAL = E:\cstGlobalLibrary\standards\cdisc-crtdds-1.0\validation\control; %let CRTDDSDATA = D:\CSTStudy\crtddsfiles; %let CHECKSDS = crtddschecks ; /* CheckType Values: Content Structural */ *%let WHERECHKTYPE = checktype in("content" %let WHERETABLESCOPE = upcase(tablescope) in("clitemdecodetranslatedtext", "CODELISTITEMS", "CODELISTS", "DEFINEDOCUMENT", "ITEMDEFS", "ITEMGROUPDEFITEMREFS","ITEMGROUPDEFS","METADATAVERSION","STUDY" *%let WHERECHKIDS = checkid in("crt0000", "CRT0001", "CRT0108", "CRT0109", "SDTM0113" libname crtdds "&CRTDDSDATA"; libname crtddsvl "&CRTDDSVAL"; proc sql; create table crtdds.&checksds as select * from crtddsvl.validation_master where &WHERETABLESCOPE /* and &WHERECHKTYPE */ /* and &WHERECHKIDS */ ; quit; sasreferences Table The sasreferences table provides a list of pointers to the tables and files needed to run a validation check process. The sasreferences table is basically a list of librefs, filerefs, folder paths, and files to use. The list of items in the sasreferences table varies depending on the standards and the type of process you are running. There should be a record in the sasreferences table that references itself. CST is not very forgiving and can be very picky if things don t match up exactly between what is in the sasreferences table and what is in the metadata tables. 3-14

53 Using the Clinical Standards Toolkit Version MAY10 Example of a sasreferences table for CRT-DDS validation: Program to Run the Validation Process Validation of CRT-DDS tables involves running a SAS program that initializes the CST standards to use, defines the location and name of the sasreferences table to use, and runs the appropriate CST macros to validate the tables. Example of a program to run a CRT-DDS table validation process: /* >>> Define macro vars that point to data and metadata */ %let STUDYID = SASCSTDEMODATA; %let STUDYROOTPATH = D:\CSTStudy; %let STDYLIB = STUDY; %let TABLESMETADS = &STUDYID._tables; %let COLUMNSMETADS = &STUDYID._columns; %let CHECKSDS = crtddschecks; %let METRICSDS = metrics_validate; %let RESULTSDS = results_validate; %let SASREFSDS = sasreferences_crtddsvalidation ; /* Set up the CST-FRAMEWORK standard. */ %cst_setstandardproperties( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _cstsubtype =initialize /* Set up the CDISC-SDTM standard. */ %cst_setstandardproperties( _cststandard =CDISC-CRTDDS, _cststandardversion=1.0, _cstsubtype =initialize /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &STUDYROOTPATH\crtddsfiles; %let _cstsasrefsname = &SASREFSDS; /* Allocate filerefs and librefs defined in sasreferences table */ %cstutil_allocatesasreferences; /* Run the validation checks. */ %crtdds_validate; 3-15

54 3. Creating a define.xml File Understanding How the Pieces Fit Together Metadata tables describe the CRT-DDS tables and provide the input for the CRT-DDS tables. A sasreferences table identifies all the input files that a validation process should use. A validation control table contains a list of checks to perform on the CRT-DDS data. CRT-DDS Validation Processing Details CRT-DDS Data clitemdecodetranslate te xt codelistitems codelists defined ocum ent item defs source_tables source_columns sasreferences Validation checks ite mgro updefitem refs itemgroupdefs m etadataversion stu dy CRT-DDS Standards reference_tables Validation Processing metrics results reference_columns SAS Formats 3-16

55 Using the Clinical Standards Toolkit Version MAY10 Running the CRT-DDS Validation Process Running an CRT-DDS validation process generates two output data tables. Metrics contains a summary of the number of records processed by various checks and a count of the number of notes, warning messages and error messages generated. Use this table to get an overview of what ran and how many validation issues were encountered. Results contains the detailed results of the validation checks. 3-17

56 3. Creating a define.xml File 3.4 Creating a define.xml File Understanding the Inputs and Outputs define.xml Creation CRT-DDS Data Run-time Control Files define.xml Processing Define.xml Results Table Key Points: You must have created the CRT-DDS SAS tables and use those as an input. A references table is used to find all the other files and tables needed for the process. You have to create a SAS program that calls appropriate CST macros to run the define.xml process. 3-18

57 Using the Clinical Standards Toolkit Version MAY10 Input Files and the Program sasreferences Table The sasreferences table provides a list of pointers to the tables and files needed to run a define.xml creation process. The sasreferences table is basically a list of librefs, filerefs, folder paths, and files to use. The list of items in the sasreferences table varies depending on the standards and the type of process you are running. There should be a record in the sasreferences table that references itself. CST is not very forgiving and can be very picky if things don t match up exactly between what is in sasreferences and what is in the metadata tables. Example of a sasreferences table for creating a define.xml file: Program to Run the define.xml Process Creating a define.xml file from CRT-DDS tables involves running a SAS program that initializes the CST standards to use, defines the location and name of the sasreferences table to use, and runs the appropriate CST macros to create the define.xml file. Example of a program to run a define.xml creation process: /* Define study root folder. This is used in the sasreferences_define table. */ %let studyrootpath= D:\CSTStudy; /* Define location for writing results table */ libname results "&studyrootpath\crtddsfiles"; /* Set up the CST-FRAMEWORK standard. */ %cst_setstandardproperties( _cststandard=cst-framework, _cststandardversion=1.2, _cstsubtype=initialize /* Set up the CST_CRTDDS standard. */ %cst_setstandardproperties( 3-19

58 3. Creating a define.xml File _cststandard=cdisc-crtdds, _cststandardversion=1.0, _cstsubtype=initialize /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &studyrootpath\crtddsfiles; %let _cstsasrefsname = sasreferences_define; /* Define where to write results */ %let _cstresultsds = results.results_define; /* Allocate filerefs and librefs defined in sasreferences table */ %cstutil_allocatesasreferences; /* Create the define.xml file */ %crtdds_write( _cstcreatedisplaystylesheet=1 libname results clear; Understanding How the Pieces Fit Together CRT-DDS tables and provide the input for the process. A sasreferences table identifies all the input files that a process should use. A stylesheet defines the layout of the define.xml. define.xml Creation Details CRT-DDS Data clitemdecodetranslatetext codelistitems codelists definedocum ent itemdefs itemgroupdefitem refs item groupdefs metadataversion study define.xml Processing define.xml Results sa sreferences stylesheet.xsl 3-20

59 Using the Clinical Standards Toolkit Version MAY10 Running the Program Running a define.xml process generates two files. Results is a SAS table that contains the detailed results of the define.xml process. Define.xml is the xml file with the study description. It can be viewed using a web browser. 3-21

60 3. Creating a define.xml File 3.5 Validating the define.xml File CST provides a macro to validate the structure of a define.xml file. You typically run define.xml validaton after creating a define.xml file through CST after editing a define.xml file. Inputs and the Program sasreferences Table You can use the same sasreferencs table that you used to create the define.xml file. Program to Validate the define.xml Example of a program to run a define.xml validation process: /* Define study root folder. This is used in the ssasreferences_define table. */ %let studyrootpath= D:\CSTStudy; /* Define location for writing results table */ libname results "&studyrootpath\crtddsfiles"; /* Set up the CST-FRAMEWORK standard. */ %cst_setstandardproperties( _cststandard=cst-framework, _cststandardversion=1.2, _cstsubtype=initialize /* Set up the CST_CRTDDS standard. */ %cst_setstandardproperties( _cststandard=cdisc-crtdds, _cststandardversion=1.0, _cstsubtype=initialize /* >>> Define location/name of sasreferences table to use */ %let _cstsasrefsloc = &studyrootpath\crtddsfiles; 3-22

61 Using the Clinical Standards Toolkit Version MAY10 %let _cstsasrefsname = sasreferences_define; /* Define where to write results */ %let _cstresultsds = results.results_definevalidate; /* Allocate filerefs and librefs defined in sasreferences table */ %cstutil_allocatesasreferences; /* Run chekcs to make sure define.xml file is correct */ %crtdds_xmlvalidate; libname results clear; Running the Program Running a define.xml validation process generates one file. Results contains the detailed results of the define.xml validation process. Controlling Log Level of XML Messages The crtdds_xmlvalidatet macro contains a parameter named _cstloglevel. _cstloglevel values can be Info, Warning, Error, Fatal Error and None. If you choose a level to report, that log level of messages is reported along with any more severe messages. Example: %crtdds_xmlvalidate(_cstloglevel=info 3-23

62 3. Creating a define.xml File 3-24

63 4 Using CST Macros Version MAY Overview and Objectives Macros for Managing Standards and Tables cst_createds cst_createstudyfromstandard cst_createtablesfordatastandard cst_getregisteredstandards cst_getstandardmetadata cst_getstandardsasreferences cst_registerstandard cst_setstandardversiondefault cst_unregisterstandard Macros for Running CST Processes cst_setproperties cst_setstandardproperties cstutil_allocatesasreferences cstutil_cleanupcstsession Macros for Validation Reporting cstutil_createreport Examples of Using CST Macros Example 1: Create Validation Reports Example 2: Get a List of Registered Standards Example 3: Create Metadata and Control Tables from a Standard Example 4: Get Copies of a Standard s Metadata Tables Example 5: Create Empty Domain Tables from a Standard Example 6: Building and Registering a Validation Standard What Happens when you Register a Standard Tables and Files you need to Edit for a New Standard Steps for Creating a New Standard Step 1: Create a Folder Structure for the Standard Step 2: Create Metadata Tables for the Standard s Tables and Columns Step 3: Create a Table of Validation Checks for the Standard Understanding Table Scope and Column Scope Step 4: Copy the Properties Files from an Existing Standard Step 5: Create a sasreferences Table for the Standard Step 6: Create a standards table for the Standard Step 7: Register the Standard to CST

64 4. Using CST Macros 4.1 Overview and Objectives The SAS Clinical Standards Toolkit (CST) is built around a set of macro libraries. This module divides CST macros into three groups: macros for managing standards macros for managing CST execution macros for reporting. The final section of this module provides short examples of using the macros. Upon completion of this module you should be familiar with the most commonly used macros that are provided as part of CST know how to use some of the CST macros. 4-2

65 Using the Clinical Standards Toolkit Version MAY Macros for Managing Standards and Tables The macros referenced in this section are located in the cstframework\sasmacro folder. to the Clinical Standards Toolkit documentation for a complete list of macros. Refer cst_createds The cst_createds macro creates an empty data table based on any table that is referenced by a standard. For example, you can use this macro to create empty tables modeled after a registered standard s reference_tables, and reference_columns, and validation_master tables. Syntax: %cst_createds( _cststandard = name of registered standard, _cststandardversion= version of registered standard, _csttype = input/output metadata type, _cstsubtype = data/metadata type within type, _cstoutputds = name of table to create, _cstresultsoverrideds =name of results table Example: %cst_createds( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _csttype =control, _cstsubtype =reference, _cstoutputds =sasrefs See Example

66 4. Using CST Macros cst_createstudyfromstandard The cst_createstudyfromstandard macro creates a folder structure for a study based on the folder structure of a standard. There is currently a defect in this macro so some macro variables must be defined before calling this macro. Syntax: %cst_createstudyfromstandard( _cststandard = name of registered standard, _cststandardversion= version of registered standard, _cststudyrootpath = root folder path Example: %cst_createstudyfromstandard( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _cststudyrootpath =d:\xyz_123, cst_createtablesfordatastandard The cst_createtablesfordatastandard macro creates an empty table for every domain table referenced by a standard. Syntax: %cst_createtablesfordatastandard( _cststandard = name of standard, <_cststandardversion = version,> _cstoutputlibrary = output library, <_cstresultsoverrideds = results table> Example: %cst_createtablesfordatastandard( _cststandard = CDISC-SDTM, _cststandardversion = 3.1.1, _cstoutputlibrary = work, _cstresultsoverrideds = my_results See Example

67 Using the Clinical Standards Toolkit Version MAY10 cst_getregisteredstandards The cst_getregisteredstandards macro creates a SAS table with a list of registered standards defined to CST. Syntax: %cst_getregisteredstandards( _cstoutputds = table name <, _cstresultsds = results table name> Example: %cst_getregisteredstandards( _cstoutputds = work.stdslist, _cstresultsds = my_results See Example 2. cst_getstandardmetadata The cst_getstandardmetadata macro creates a copy of a metadata table referenced by a standard. The metadata table to copy is defined by a record in a sasreferences table. The _cstsasreferences parameter defines the sasreferences table to use. Each record specifies the type of metadata tabel to return and the name of the table to create. You typically use this macro to get the metadata tables for a standard list of tables and list of columns make up a standard, for example. Syntax: %cst_getstandardmetadata( _cstsasreferences = references table <, _cstresultsoverrideds = results table> Example: %cst_getstandardmetadata( _cstsasreferences =work.temprefs, _cstresultsoverrideds=my_results See Example

68 4. Using CST Macros cst_getstandardsasreferences The cst_getstandardsasreferences macro creates a sasreferences table based on the references defined for a standard. Syntax: %cst_getstandardsasreferences( _cststandard = name of standard, <_cststandardversion = version,> _cstoutputds = output table, <_cstresultsoverrideds = results table> Example: %cst_getstandardsasreferences( _cststandard = CDISC-SDTM, _cststandardversion = 3.1.1, _cstoutputlibrary = work.mysasrefs, _cstresultsoverrideds = my_results 4-6

69 Using the Clinical Standards Toolkit Version MAY10 cst_registerstandard The cst_registerstandard macro is used to add a new standard to CST. Syntax: %cst_registerstandard( _cstrootpath = root folder path for standard, _cstcontrolsubpath = relative path to std s sasrefs table, _cststddsname = name of standard s registration table, _cststdsasrefsdsname = name of standard s sasrefs table <, _cstoutputds = results table> Example: %cst_registerstandard( _cstrootpath =%nrstr(&_cstgroot./standards/mynewstds), _cstcontrolsubpath = control, _cststddsname = newstd.my_stds, _cststdsasrefsdsname = newstd.mysasrefs, _cstoutputds = work.myresults Notes: The _cstrootpath parameter specifies the root folder for the new standards you are registering. See Example 6. The _cstcontrolsubpath parameter specifies the subfolder where your standards table and sasreferences table are located. This folder must be under the folder specified by the _cstrootpath parameter and it must be a relative path. The _cststddsname parameter specifies the name of the standards table for your standard. It contains a single record with column values that identify the standard you are registering. You can use any valid SAS name for this table, but it must be structured like a standards table. The _cststdsasreferencedsname parameter specifies the name of the sasreferences table for your standards. You can use any valid SAS name for this table, but it must be structured like a sasreferences table. For the sasreferences table specified by the _cststdsasreferencedsname parameter, make sure that you specify only relative paths on reference records. Otherwise, you will have to edit paths in the standardsasreferences table in the cstgloballibrary\metadata folder. You can use the cst_createds macro to create an empty standards table and an empty sasreferences table. 4-7

70 4. Using CST Macros cst_setstandardversiondefault The cst_setstandardversiondefault macro is used to set the default version for a standard when multiple versions of a standard exist. The default version is used when you do not provide a version of a standard in a sasreferences table or in macro calls that contain a version parameter. Syntax: %cst_setstandardversiondefault( _cststandard = standard name, _cststandardversion = standard version <, _cstresultsoverrideds= results table > Example: %cst_setstandardversiondefault( _cststandard = CDISC-STDM, _cststandardversion = 3.1.1, _cstresultsoverrideds= myresults cst_unregisterstandard The cst_unregisterstandard macro removes a standard from the list of standards known to CST. Syntax: %cst_unregisterstandard( _cststandard = standard name, _cststandardversion = standard version <, _cstresultsoverrideds = results table > Example: %cst_unregisterstandard( _cststandard = CDISC-SDTM, _cststandardversion = , _cstresultsoverrideds = myresults Notes: The list of registered standards is maintained in the standards table in the cstgloballibrary\metadata folder. 4-8

71 Using the Clinical Standards Toolkit Version MAY Macros for Running CST Processes The macros referenced in this section are located in the cstframework\sasmacro folder. Refer to the Clinical Standards Toolkit documentation for a complete list of macros. cst_setproperties The cst_setproperties macro provides the location and name of the properties file to load. The contents of the properties file contains name=value pairs for global macros. The properties file can be specified as a text file, a SAS fileref, or a SAS table. Syntax: %cst_setproperties( _cstpropertieslocation = property file name, _cstlocationtype = PATH FILENAME DATA <, _cstresultsoverrideds = results table name > Example: %cst_setproperties( _cstpropertieslocation = g:\mystudy\config\cst.props, _cstlocationtype = path, _cstresultsoverrideds = myresults Notes: The value of _cstpropertieslocation depends on the value of _cstlocationtype. cst_setstandardproperties The cst_setstandardproperties macro sets global macros that are supplied as part of a standard. These properties are usually stored in a.properties file within a standard s root folder and are referenced in the standard s sasreferences. Syntax: %cst_setstandardproperties( _cststandard = standard name, <_cststandardversion = standard version, > _cstsubtype = supbtype to use <, _cstresultsoverrideds = results table > Example: %cst_setstandardproperties( _cststandard =CST_FRAMEWORK, _cststandardversion=1.2, _cstsubtype =initialize See Example 1, 2, 3, and

72 4. Using CST Macros cstutil_allocatesasreferences The cstutil_allocatesasreferences macro allocates the SAS librefs and filerefs specified in a sasreferences table. It has no parameters. The sasferences table is specified by setting the global macro variables _cstsasrefsloc and _cstsasrefsname before calling this macro. Syntax: %cstutil_allocatesasreferences; Example: %let _cstsasrefsloc = e:\mystudy\config; %let _cstsasrefsname = mysasrefs; %cstutil_allocatesasreferences; cstutil_cleanupcstsession The cstutil_cleanupcstsession macro cleans up after a CST session. It performs tasks like removing any process-level SAS files and clearing the temporary SAS macro catalog. Syntax: %cstutil_cleanupcstsession( _cstclearcompiledmacros = 0 1, _cstclearlibrefs = 0 1, _cstresetsasautos = 0 1, _cstresetfmtsearch = 0 1, _cstresetsasoptions = 0 1, _cstdeletefiles = 0 1, _cstdeleteglobalmacrovars = 0 1 Notes: _cstclearcompiledmacros removes all compiled macros from the work.sasmacr catalog. 0 is default. _cstclearlibrefs deallocates all librefs and filerefs set based upon sasreferences content. 0 is default. _cstresetsasautos resets the autocall search path to its initial state. 0 is default. _cstresetfmtsearch resets the format search path to its initial state. 0 is default. _cstresetsasoptions resets SAS options to their initial state. 1 is default. _cstdeletefiles deletes all CST work files and catalogs. If global macro variable _cstdebug=1, files are NOT deleted even if _cstdeletefiles=1. 1 is default. _cstdeleteglobalmacrovars deletes all CST global macro variables set based upon property file name/value pairs. 0 is default. 4-10

73 Using the Clinical Standards Toolkit Version MAY Macros for Validation Reporting There is currently only one reporting macro. It is located in the cstframework\sasmacro folder. Refer to the Clinical Standards Toolkit documentation for a complete list of macros. cstutil_createreport The cstutil_createreport macro generates a report on metrics and results tables created from a validation process. Reports can be generated in the form of HTML, PDF, or RTF files. This macro uses PROC REPORT and ODS to create the report. In general, you should run this macro in the same CST session after you run a validation check. It relies on certain macro variables being defined that point to CST files and tables. There is currently a defect in this macro and it will not run as documented. The underlying macro code has been edited locally so that it will run. This macro is included here as an illustration of a possible reporting capabilties in a future release. Syntax: %cstutil_createreport( _cstsasreferencesdset = sasreferences table, _cstresultsdset = results table to use, _cstmetricsdset = metrics table to use, _cstrejporterrorsonly = N Y, _cstreportobs = 0 1, _cstreportbydomain = 0 1, _cstdomaindset = domain to report, _cstreportformat = HTML PDF RTF, _cstreportoutput = folder path for output, _summaryreport = N Y, _ioreport = N Y, _metricsreport = N Y, _generalresultsreport = N Y, _checkidresultsreport = N Y Notes: _cstsasreferencesdset libref.dataset of SASreferences data set used for a specific process. Optional. If specified, the _cstresultsdset and _cstmetricsdset parameters will be ignored. Either _cstsasreferencesdset or _cstresultsdset must be provided. _cstresultsdset libref.dataset of SAS Clinical Standards Toolkit process results data set. Optional. Either _cstsasreferencesdset or _cstresultsdset must be provided. Ignored if _cstsasreferencesdset specified. 4-11

74 4. Using CST Macros _cstmetricsdset libref.dataset of SAS Clinical Standards Toolkit process metrics data set. Optional. Ignored if _cstsasreferencesdset specified. _cstreporterrorsonly If N (default), report all records in the results data set, including informational and non-error results. If Y, report only records in error (where the results data set field resultflag=1). _cstreportobs If null (default), report all records in error (where results.resultflag=1) in the results data set. Otherwise, set to any integer value > 0, signifying the number of records to print per checkid (where results.checkid is non-null). If _cstreportobs >0 results in excluding any records, a footnote is printed, noting that not all records were printed. _cstreportbydomain If N (default), do not report results by domain (that is, run report 1). If Y, report results by domain (that is, run report 2). _cstdomaindset _cstreportformat _cstreportoutput Optionally produce report based on a specific domain, indicated by libref.data set. If blank or the keyword _ALL_ is specified, all domains are included in the report. Ignored if _cstreportbydomain=n. HTML (default). Allows specification of other ODS-supported formats (for example, PDF and RTF). The directory location for the generated report output. The value should not include the filename, which is currently hardcoded to either CSTCheckIDReport + the _cstreportformat (report 1) orcstdomainreport + the _cstreportformat (report 2). Required. Be sure to end the path with a slash or the last part of the path will become the prefix for the filename. _summaryreport _ioreport _metricsreport (Y/N) If set to Y, generate the report summary panel. Default=Y. (Y/N) If set to Y, generate the process inputs/outputs panel. Default=Y. (Y/N) If set to Y, generate the process metrics panel. Default=Y. This should be set to N for any non-validation reports and cases where metrics were not generated. _generalresultsreport (Y/N) If set to Y, generate the general process reporting panel. Default=Y. _checkidresultsreport (Y/N) If set to Y, generate the process results panel. Default=Y. See Example

75 Using the Clinical Standards Toolkit Version MAY Examples of Using CST Macros Example 1: Create Validation Reports Generate validation reports on a previous validation process run. In this example, assume that better_validate_sdtm.sas has just run in this session. /* Assumes that better_validate_sdtm.sas has run in this session */ /* Define this macro to make it work */ %let _cstrptcontrol = srcmeta.sdtmchecks; %cstutil_createreport( _cstsasreferencesdset = results.better_sasreferences, _cstresultsdset = results.results, _cstmetricsdset = results.metrics, _cstreportbydomain = N, _cstreportoutput = d:\cststudy\sampleoutput\ The following is partial output generated by the cstutil_createreport macro: 4-13

76 4. Using CST Macros 4-14

77 Using the Clinical Standards Toolkit Version MAY10 Example 2: Get a List of Registered Standards The list of registered standards are located in the folder cstgloballibrary\metadata in a SAS table named standards. You can use the cst_getregisteredstandards macro to get a list of registered standards. It creates a copy of the standards SAS table. %cst_setstandardproperties( _cststandard = CST-FRAMEWORK, _cststandardversion = 1.2, _cstsubtype = initialize %cst_getregisteredstandards( _cstoutputds = work.mystdslist 4-15

78 4. Using CST Macros Example 3: Create Metadata and Control Tables from a Standard The main table structures that you need for a standard are a list of domain tables with metadata definitions a list of columns in domain tables with metadata definitions a list of validation checks a sasreferences table to identify where the compoents of a standard are located. You can use the cst_createds macro to create empty tables for any table referenced by a record in the sasreferences table for a standard. The following program creates empty tables for the fours key tables used by the SDTM standard. It creates the tables in the WORK library. %cst_setstandardproperties( _cststandard = CST-FRAMEWORK, _cststandardversion = 1.2, _cstsubtype = initialize %cst_setstandardproperties( _cststandard = CDISC-SDTM, _cststandardversion = 3.1.1, _cstsubtype = initialize %cst_createds( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _csttype =control, _cstsubtype =reference, _cstoutputds =work.my_sasrefs %cst_createds( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _csttype =referencemetadata, _cstsubtype =table, _cstoutputds =work.my_tablesmetadata %cst_createds( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _csttype =referencemetadata, _cstsubtype =column, _cstoutputds =work.my_columnsmetadata 4-16

79 Using the Clinical Standards Toolkit Version MAY10 %cst_createds( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _csttype =referencecontrol, _cstsubtype =validation, _cstoutputds =work.my_validationchecks 4-17

80 4. Using CST Macros Example 4: Get Copies of a Standard s Metadata Tables You want to get copies of the list of tables and list of columns defined to a standard. The following program creates copies of the tables for the used by the SDTM standard. It creates the tables in the WORK library. %let STANDARD = CDISC-SDTM; %let STDVERSION = 3.1.1; /* Initialize the standard to use for macro calls below */ %cst_setstandardproperties( _cststandard =&STANDARD, _cststandardversion=&stdversion, _cstsubtype =initialize /* Create an empty sasreferences table */ %cst_createds( _cststandard =CST-FRAMEWORK, _cststandardversion=1.2, _csttype =control, _cstsubtype =reference, _cstoutputds =sasrefs /* Add 2 records for references tables and columns */ proc sql; insert into sasrefs set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='table', sasref ='work', reftype ='libref', memname ='reftables' ; quit; set standard ="&STANDARD", standardversion="&stdversion", type ='referencemetadata', subtype ='column', sasref ='work', reftype ='libref', memname ='refcolumns' /* Get tables and columns defined to standards */ %cst_getstandardmetadata( _cstsasreferences =work.sasrefs, _cstresultsoverrideds=_cstresults 4-18

81 Using the Clinical Standards Toolkit Version MAY10 Example 5: Create Empty Domain Tables from a Standard You can use CST to create a set of empty domain tables from a standard like SDTM. This gives you a way to create domain tables for a new study that are consistent with a standard. The cst_createtablesfordatastandards macro uses a standard s reference_tables and reference_columns tables to create an empty table for every table referenced in the standard. %cst_setstandardproperties( _cststandard = CST-FRAMEWORK, _cststandardversion = 1.2, _cstsubtype = initialize %cst_createtablesfordatastandard( _cststandard = CDISC-SDTM, _cststandardversion = 3.1.1, _cstoutputlibrary = work 4-19

82 4. Using CST Macros Example 6: Building and Registering a Validation Standard You can define your own standards and register them to CST. There are two strategies for defining standards, depending on your needs: Make a copy of the entire folder hierarchy of an existing standard, edit the tables and files in your new standard folder. Build your own folder structure for your standard. This requires a bit more editing, but it potentially allows for sharing of things like messages and macro libraries across standards. What Happens when you Register a Standard When you register a standard two master CST tables get updated in cstgloballibrary\metadata: The standards table contains one record for each registered standard. A new record is added when you register a new standard. The standardsasreferences table contains a record for each libref and fileref defined in the sasreferences table for the standard. Tables and Files you need to Edit for a New Standard You can use any names that you want for files and tables that define a standard. Typical Table Name reference_tables reference_columns standards standardsasreferences Description Metadata for tables in a study. Reasons to edit: If you need to define new domains, remove domains, or modify other metadata for tables. Metadata for columns in the study tables. Reasons to edit: If you need to define new domains, remove domains, or modify other metadata for columns in a table. Contains a single record that describes properties for a standard. Reasons to edit: Must be updated for a new standard. Contains the librefs and filerefs used to point to the tables and files used by a standard. Reasons to edit: Must be updated for new standards. Beware of defining paths in the Path column 4-20

83 Using the Clinical Standards Toolkit Version MAY10 Typical Table Name initialize.properties validation.properties validation_master macros folder Description Contain name=value pairs that are converted into SAS macro variables. Reasons to edit: You want to change the default properties for a standard. Contains the list of validation checks for a validation standard. Reasons to edit: You want to remove validation checks, add new validation checks, or potentially modify the behavior of a validation check. Contains a macro library specific to a standard. Reasons to edit: You want to add new standard-specific macros. Steps for Creating a New Standard Scenario: We want to create an SDTM Lite version of SDTM We want to include a small list of domains, add some new domains, and have a simple study folder structure. We want to use the existing macro library for SDTM Step 1: Create a Folder Structure for the Standard The goal is to have a very simple folder structure for the standard and use that same simple structure for studies based on SDTM Lite. The folder structure below is what is used for this standard. 4-21

84 4. Using CST Macros Step 2: Create Metadata Tables for the Standard s Tables and Columns Create the two metadata tables using the addon_create_study_meatadata macro. This will create the metadata tables from existing study domain tables and populate the extended metadata columns based on the SDTM standard. From there, you can edit the metadata tables to reflect customizations to your standard. %addon_create_study_metadata( STUDYLIB = stdref, STUDYID = sdtmlite, STUDYDESC = SDTM Lite, STUDYDATAPATH = d:\l040123\data, SDTMDATAPATH = &_cstgroot\standards\cdisc-sdtm-lite\metadata, DOMAINS = DM DU LB OM QS VS, NEW = Y Notes: There are two new domains (DU and OM) that are not found in the SDTM standards. The extended metadata for these domains will have to be added for the tables and column metadata tables. Step 3: Create a Table of Validation Checks for the Standard Copy the validation_master table from the SDTM standard to your standard. Edit the table to remove/add domain references relevant to the new standard. This is where it is useful to join the validation_master table and messages table so that you can see validation properties of each check along with a description of the check. You will have to edit the tablescope and columnscope columns for checks in order to get them to work correctly. 4-22

85 Using the Clinical Standards Toolkit Version MAY10 Understanding Table Scope and Column Scope The table with the list of validation checks for a standard contains two columns that control how checks are applied: Tablescope specifies the domains to be tested by a check. Value Format Example Description _ALL_ Applies to all domains domainname DM Applies to a single domain domainname + domainname LB + OM Applies to specified domains; plus sign is delimiter _ALL_ - domainname _ALL_ - DM Applies to all domains, excluding those specified; minus sign is delimiter sharedtablename** SUPP** Wildcard to include multiple domains CLASS:classValue CLASS:FINDINGS Uses the Class column in the tables metadata to determine which domain tables to include [comparegroup][comparegroup] [_ALL_ -DM][DM] Allows the comparison of a group of domain tables with another group of domain tables Columnscope specifies the columns to be tested by a check. Value Format Example Description _ALL_ Applies to all columns. You can also specify a blank value _NA_ Specifies that the check is not applicable to column-level checks columnname AGE Applies to a singlecolumn. May be specified as libref.domain.column or domain.column columnname + columnname LBTEST+LBTESTCD Applies to specified columns; plus sign is delimiter **sharedcolumnname **TESTCD Applies to all shared column **sharedcolumnnamecolumnname **TEST - OMTEST names across domains Applies to all shared column names across domains, excluding those specified; minus sign is delimiter domainname** AE** Applies to all columns in the specified domain [comparegroup][comparegroup] [**_STDTC][**ENDTC] Allows the comparison of a group of columns with another group of of columns 4-23

86 4. Using CST Macros Here is the edited table of validation checks for the SDTM Lite standard. Step 4: Copy the Properties Files from an Existing Standard Since the SDTM Lite standard is based on the SDTM standard, copy the files initialize.properties and validation.properties from the SDTM standard folder to the SDTM Lite standard folder. Remember, these two property files are referenced in the sasreferences table for the standard. Having separate property files for a standard gives you the ability to customize the properties without affecting other standards. Step 5: Create a sasreferences Table for the Standard You could use the cst_createds macro to create an empty sasreferences table, then add records for each libref and fileref that the SDTM Lite standard needs. %cst_createds( _cststandard =CDISC-SDTM, _cststandardversion=3.1.1, _csttype =control, _cstsubtype =reference, _cstoutputds =work.my_sasrefs 4-24

87 Using the Clinical Standards Toolkit Version MAY10 You could use the cst_getstandardsasreferences macro to create a copy of the sasreferences table for the SDTM standard, then edit the records to fit the SDTM Lite standard. %cst_setstandardproperties( _cststandard = CST-FRAMEWORK, _cststandardversion = 1.2, _cstsubtype = initialize %cst_getstandardsasreferences( _cststandard = CDISC-SDTM, _cststandardversion = 3.1.1, _cstoutputds = sdtmlite_sasrefs Here is the sdtmlite_sasrefs table for the SDTM Lite standard. Notice that it uses the same macro library, messages, and lookup table as the SDTM standard. Beware of defining paths in the Path column. The cst_registerstandard macro assumes that the Path column contains only relative paths for standard-specific files. Step 6: Create a standards table for the Standard The standards table is a single-record table that stores key identifiers about the standard version. The standards table for the SDTM Lite standard: 4-25

88 4. Using CST Macros Step 7: Register the Standard to CST %cst_setstandardproperties( _cststandard = CST-FRAMEWORK, _cststandardversion = 1.2, _cstsubtype = initialize %cst_registerstandard( _cstrootpath = %nrstr(&_cstgroot./standards/cdisc-sdtm-lite), _cstcontrolsubpath = stddefiles, _cststddsname = sdtmlite_standards, _cststdsasrefsdsname = sdtmlite_sasrefs Details on the program: The _cstrootpath parameter specifies the root folder for the new standards you are registering. The value assigned above includes the _cstgroot macro and assumes that your standards are located under the cstgloballibrary\standards folder. Notice the use of the &nrstr macro function. This does prevents the _cstgroot macro variable from being resolved at run time and inserts entire path as is. The _cstcontrolsubpath parameter specifies the subfolder where your standards table and sasreferences table are located. This folder must be under the folder specified by the _cstrootpath parameter. The _cststddsname parameter specifies the name of the standards table for your standards. The cststdsasreferencedsname parameter specifies the name of the sasreferences table for your standards. Once the program above runs, there should be a record in the standards table in cstgloballibrary/metadata for you newly registered standard. 4-26

89 Using the Clinical Standards Toolkit Version MAY10 Records have also been added to the standardsasreferences table in the cstgloballibrary/metadata folder. Notice the values in the Path column where the standard s sasreferences table had full path references. 4-27