Using SAS Metadata as a Key Performance Indicator!

Transcription

1 Using SAS Metadata as a Key Performance Indicator! Vincent J. Amoruccio, Alexion Pharmaceuticals, Inc., Cheshire, CT ABSTRACT Key Performance Indicators (KPIs) help organizations track progress towards goals. SAS Metadata can be used as a platform neutral KPI to help your company do more, faster and better! The concept of metadata is often misunderstood since the term metadata is ambiguous. However, data about our data can be used as KPIs to quantifiably track and measure the success of analytical projects. The purpose of this paper is to show how to use SAS objects as metadata and track almost every aspect of your analytical deliverables. It will specifically address metadata for code reviews, check logs, and validation compliance. Finally, it will explain how to read in data from operating systems and create a utility to understand the status of a programming project. INTRODUCTION Alexion is a global biopharmaceutical company that combines groundbreaking science with a steadfast commitment to meeting the needs of patients living with severe, life-threatening and often ultra-rare diseases. The goal of a biopharmaceutical is to obtain approval from regulatory agencies such as the FDA and to license their drugs as medications. The goal of Alexion is to submit applications to regulatory agencies and obtain approval to use their drug as a medication. The goal of the Clinical and Statistical Programming group at Alexion is to successfully develop the statistical analysis in the form of datasets, tables, listings, and figures required for a regulatory submission. We are a key stakeholder in Alexion s success and have identified tools that quantifiably measure progress towards our success. KEY PERFORMANCE INDICATORS Key performance indicators, also known as KPIs, are a quantifiable performance measurement tools that organizations use to evaluate success of a particular goal. The concept of KPIs is often misunderstood since they have a history of being overly complicated. It is important to predefine what performance measurements are useful to the success of a project and how these measurements can be simply identified and reported. KPIs for SAS programming can be classified into four categories: quantitative, qualitative, process, and output. Qualitative indicators are those that can t be measured and are not presented as a number. They measure distinctive characteristics, attributes, and qualities. Quantitative indicators are those that can be measured as a number. Process indicators represent the efficiency or the productivity of the process. Output indicators reflect the outcome or results of the process activities. METADATA The concept of metadata is often misunderstood since the term metadata is ambiguous. Bretheron & Singley distinguish between two distinct classes: structural/control metadata and guide metadata. According to them, structural metadata are used to describe the structure of computer systems such as tables, columns and indexes. Guide metadata are used to help users find specific items and are usually expressed as a set of keywords in a natural language. While metadata is purely data about data, the metadata used in this paper is a combination of both. We will use data to define the structure of other data, specifically the attributes of data sets, variables, and parameters; however, we will also use data to help end users find specific data. A platform is the underlying hardware or software for a computing system. A metadata platform is the underlying software or file type to store metadata. SAS is a powerful system of software products and has the ability to convert a wide array of data types into SAS data. SAS can easily read files from Microsoft Excel, Microsoft Access, dbase, Lotus, and other delimited files (tab, comma, etc.). Depending on how the metadata are used in SAS, there might be limitations from each platform. If metadata are going to be dynamically accessed by users, SAS, or both, file-access permissions must be taken into consideration. Microsoft Excel files and various delimited files, such as CSV and TXT files, will be problematic if they need to be accessed simultaneously by multiple users or programs. These file types are usually single-access files where only one user or program may access it at time. 1

2 Metadata, or data about data, is platform neutral as long as the data it stores can be clearly identified by rows and columns. If this is true, then the platform storing the data is irrelevant and the metadata should be easily hosted by any other platform other than the one it is stored in. The metadata system at Alexion was initially created using Microsoft Excel and was transformed into Microsoft Access since Microsoft Access is an easy to use relational database. The concepts of metadata described in this paper can be established using any platform which stores data in a row-by-column structure. I highly recommend a relational database such as Microsoft Access since it can be accessed by multiple users, remain open while being accessed by SAS, and use structured query language to join and query data. Using other platforms, such as Microsoft Excel or other delimited files (tab, comma, etc.), will suffice, however it will require more SAS programming code to join and query data. HARNESSING METADATA TO USE AS KPI The goal of Clinical and Statistical programming is to develop and validate programs that create datasets, tables, listings, and figures for a regulatory submission. The measure of success for each of these outputs is determined by the cleanliness of the log, the status of independent programming, and compliance with SOPs. Each of these are measurable quantitatively and qualitatively. VALIDATION METADATA AS A KPI The pharmaceutical industry is governed by various rules and regulations by regulatory agencies. The only requirement for validating SAS programs and output is that validation must occur. The most common method for validating SAS output is independent programming. Independent programming is an objective approach whereby a second program is written by an independent programmer using written specifications. We use independent programming as the primary approach to validating all analysis output at Alexion. The COMPARE procedure is used to compare the contents of two SAS data sets, selected variables in different data sets, or variables within the same data set. It identifies matching variables with different values, the number of observations in common, the number of variables in common, differences in attributes of common variables, and a comparison of the values of matching observations. Validation metadata can be easily created from the COMPARE procedure using the Output Delivery System, also known as ODS. ODS is most popular for its ability to include or exclude select components of any SAS procedure output and converts them into more condensed, understandable, and reportable data. ODS breaks procedures down into sections. Each section is a set of raw data defined by SAS. These data sets store the information that is displayed in the procedure output and can be used to summarize data. tracing is listed below: Figure 1 RESULTS window displaying the COMPARE output object. After a procedure is run, the RESULTS window will display a list of the output sections produced by the procedure. Each section is called an output object and can be transformed into a data set. The procedure s components will become the observations and variables in the data set. Figure 1 is a RESULTS window listing four output objects for the COMPARE procedure: Datasets, Details, Summary, and Differences. The listing in the RESULTS window will match the sections of output generated by the procedure. The RESULTS window can be hidden to increase the working space on your computer monitor; however, when using ODS for the first time, it might be helpful to leave the RESULTS window open. The best way to visualize the output objects generated by a procedure is to view the output in the RESULTS window. ODS identifies the output objects in a way similar to the RESULTS window through a process called tracing. The general syntax for ods trace on; procedure syntax; ods trace off; To use tracing to identify the output objects for the COMPARE procedure, we submit the code below on the adverse events dataset. 2

3 ods trace on; proc compare base = dev.adae comp = val.adae listall; ods trace off; Figure 2 Tracing results from a section of the log for the COMPARE procedure Output Added: Name: CompareDatasets Label: Datasets Data Name: BatchOutput Path: Compare.CompareDatasets Output Added: Name: CompareDetails Label: Details Data Name: BatchOutput Path: Compare.CompareDetails Output Added: Name: CompareSummary Label: Summary Data Name: BatchOutput Path: Compare.CompareSummary Output Added: Name: CompareDifferences Label: Differences As a result of the tracing, information about each of the output objects will be generated in the log. SAS will still generate a listing of the procedure s output objects in the RESULTS window. Information that is not captured in the RESULTS window will be found in the log. Figure 2 shows the results of tracing the COMPARE procedure from a section of the log. The tracing results provide the Name, Label, Data Name, and Path for the output objects. For now, we will use the name of the output object. This provides the information to turn the output objects from the COMPARE procedure into metadata about the validation status of a particular SAS output. The output objects we are primarily concerned with are the Data Set Summary ( comparedatasets ) and the Observation Summary ( comparesummary ). Data Name: BatchOutput The Data Set Summary output object Path: Compare.CompareDifferences provides four useful pieces of metadata: data set name, modification date, number of variables, and number of observations for both the BASE and COMPARISON dataset. We use this at Alexion to ensure the BASE library is not a temporary library and the number of variables and observations are identical between both BASE and COMPAR- ISON datasets. The Observation Summary output object provides two useful pieces of data: the number of observations in common and the number of observations unequal. A metadata set containing one record per output can easily be created to describe the validation status of each output. We begin with COMPARE procedure and ODS OUTPUT statements. The ODS output statements produce SAS datasets from output objects we specify. We previously identified the names of the output objects comparedatasets and comparesummary using the TRACE option. In both of these datasets we kept only those records with data (TYPE = d and BATCH ne ). proc compare base = dev.adae comp = val.adae listall; ods output comparedatasets = comparedatasets1 (where = (type = "d" and batch ne '')); ods output comparesummary = comparesummary1 (where = (type = "d" and batch ne '')); The Data Set Summary output object ( comparedatasets ) contains three records; one for the BASE dataset, one for the COMPARISON dataset, and one for the number of common variables between them. The first DATA step identifies three datasets for which variables will be outputted. We create a dataset called BASE which contains the base dataset name ( base ), the date and time the dataset was modified ( base_dttm ), the number of variables in the base dataset ( base_nvar ), and the number of observations in the base dataset ( base_nobs ). We do the same for the comparison dataset but change the prefixes of the variables to comp. Finally, we create the dataset COMMONVARS which derives a variable to report the number of variables in common between the two datasets. 3

4 data base (rename = (dat = base dttm = base_dttm nvar=base_nvar nobs = base_nobs)) comp (rename = (dat = comp dttm = comp_dttm nvar=comp_nvar nobs = comp_nobs)) commonvars; length dat $50.; set comparedatasets1; batch = cats(batch); Since we restricted the metadata to records with data, we use the automatic variable _N_ to help select the records to be outputted. This automatic variable is initially set to 1. Each time the DATA step loops past the DATA statement, the variable increases by 1. It represents the number of times the DATA step re-iterated, but for our purposes it is used to output the records into a metadata set. The first record contains data about the BASE dataset and the second record contains data about the COMPAR- ISON dataset. The variable batch captures the text from the procedure output. We use the text in this variable to create the variables we need using the SCAN function with space as a delimiter. if _n_ in (1, 2) then do; dat = cats(scan(batch, 1, '')); dttm = input(scan(batch, 2, ''),datetime20.); nvar = input(scan(batch, 4, ''),best.); nobs = input(scan(batch, 5, ''),best.); if _n_ = 1 then output base; if _n_ = 2 then output comp; The third record contains the Variable Summary output. We use the SCAN function again, however, a period is used as the delimiter. The period is striped before converting it to a numeric variable with the INPUT function. if _n_ = 3 then do; commonvars = input(compress(scan(batch, -1,''),'.'),best.); output commonvars; We use the same approach to convert the Data Set Summary output objects into metadata. The ODS output statement of the COMPARE procedure restricted the metadata to only keep records with data. We use the automatic variable _N_ again and only keep records 5 and 12. These records provide data about the number observations in common and the number of observations unequal, respectively. data commonnobs nobsuneq; set comparesummary1; if _n_ = 5 then do; commonnobs = input(compress(scan(batch, -1,''),'.'),best.); output commonnobs; if _n_ = 12 then do; nobsuneq = input(compress(scan(batch, -1,''),'.'),best.); output nobsuneq; After splitting the COMPARE procedure output objects into separate SAS datasets, we complete the validation metadata by combining the data together. data compare; merge base comp commonvars commonnobs nobsuneq; keep base: comp: common: nobsuneq; 4

5 Validation metadata are one record per SAS output. Validation metadata are a critical check to ensure all components of the submission package have been independently programmed and pass the validation procedures. This is a critical KPI of the success of a validated and SOP compliant deliverable. The TABULATE procedure displays descriptive statistics in tabular format and is one way to summarize data. proc tabulate data=compare missing; class nobsuneq base; tables nobsuneq*base, (n pctn)/ rts = 100; Figure 3 Validation Metadata Summary of COM- PARE Output by BASE Name No. Obs Unequal Base Dataset N % 0 SDTM.AE SDTM.CM SDTM.DA SDTM.DM SDTM.DV SDTM.EX SDTM.LB SDTM.PC SDTM.PE SDTM.TA SDTM.TE SDTM.MH SDTM.SUPPXR Figure 3 shows a sample of output from the TABULATE procedure above. The metadata provides information about each BASE dataset from a particular library. We can use these metadata to quickly assess which outputs have successfully passed validation. All datasets must have no observations unequal. Additionally, the number of variables and the number of observations in both BASE and COMPARISON datasets must be the same. At Alexion, we create derived variables to categorize these findings further and continually use validation metadata as a KPI. LOG METADATA AS A KPI A SAS log ( log ) is a record of the execution of each statement in a SAS program. Original program statements are identified by line numbers along with messages from SAS about errors, warnings, or other useful notes. SAS performs error processing during both the compilation and the execution phases. An error is a clear indication that something in the SAS program or the data is incorrect. Errors must not be allowed in the logs of any SAS programs. A majority within the SAS programming community agree that avoidable warnings or notes in the log are not acceptable. These messages are confusing and often lead to errors. If any message in the log is avoidable programmers should program around it. Alexion Standard Operating Procedures ( SOPs ) prohibit errors, and avoidable warnings and notes. Our SOP specifically identifies 10 notes which are avoidable and degrade the integrity of the program. The cleanliness of our logs is determined by the presence of errors, warning, and the prohibited notes. It is converted into metadata and used as a KPI. Log metadata can be created in one simple DATA step to report the number of errors, warnings, and nonpermissible note. The first step is to identify the file path where the logs are stored using a FILENAME statement. The FILENAME statement associates a SAS file reference ( FILEREF ) with an external file or a logical file device. It is a powerful statement which can be used to convert any SAS file type such as logs (.log ), lists (.lst ), and programs (.sas ) into searchable data. Wild cards, such as the asterisk and question mark can be used in either the external filename or file extension for matching input filenames. The code below specifies a FILEREF and uses the asterisk as a wild card to identify all log files in a particular directory. filename logs "C:\Users\amorv1982\NESUG\Programs\ANALYSIS\Tables\*.log"; The first step uses the INFILE statement to read in the logs. The INFILE statement specifies an external file to read with and then uses an INPUT statement to bring in the log files. The filename variable is used to determine the file being read. The binary variable eov is created by SAS and is set to 1 on the first record of a file being set in through a series of other files. The INPUT statement describes the arrangement of values in the input data record and assigns input values to the corresponding SAS variables. 5

6 data chklog; length filename $256 progname $32; retain progname; infile logs filename=filename eov=eov length=l; input logline $varying256. l; Next, we use the filename of the first record to create a variable to store the program name. if _n_ eq 1 or eov then do; progname = scan(filename,-2,'.\'); eov = 0; The log will have program statements as well as SAS messages. We use the SCAN function to pick off only those log lines containing the SAS messages. The SCAN function returns the n th word from a character string. Using a positive counter we scan the log line from left to right using colon as the delimiter. if index(logline,':') gt 0 then type = scan(logline, 1, ':'); else if index(logline,':') =0 then type = ''; The sample code below shows a few SAS NOTES we check for at Alexion using the INDEX function. This function searches a character expression for a string of characters, and returns the position of the string s first character for the first occurrence of the string. We then use the CATX function to remove leading and trailing blanks, insert a semicolon as a delimiter and create a concatenated variable discrep categorizing the NOTE messages. if type eq "NOTE" then do; if index(upcase(logline),'uninitialized') gt 0 then r1= "Uninitialized"; if index(upcase(logline),'converted') gt 0 then r2= "Converted"; if index(upcase(logline),'invalid') gt 0 then r3 = "Invalid"; if index(upcase(logline),'missing') gt 0 then r4 = "Missing"; discrep = catx(";", r3, r4); The final step in this data set is to keep only ERROR and WARNING messages or those NOTES identified using the INDEX function. if type in ("ERROR", "WARNING") or (type eq "NOTE" and discrep ne ''); The data about the logs, also known as log metadata, can be summarized and presented as a KPI about the cleanliness of the logs through the TABULATE procedure. proc tabulate data=chklog missing; class progname type discrep; tables progname*type*discrep, (n pctn)/ rts = 100; Figure 4 shows a sample of the output from the TABULATE procedure above. These metadata provide information about each programs log in a particular submission. At Alexion, we do not allow ERRORs in the log and we do not allow WARNINGS in the logs unless they are unavoidable such as those from statistical procedures. Additionally, we have a select list of NOTES which we do not allow unless they are proven to be unavoidable. Log metadata are used as a KPI to provide an indication of the cleanliness of the logs. We anticipate no findings as the best-case scenario and document any deviations from this. 6

7 Figure 4 Log Metadata Summary of Log Notes by Program Name Program Name Category Discrepancy N % eff-chg-tto WARNING t6mwt WARNING tdexa WARNING texpext ERROR texpext NOTE Stopped texpext WARNING thhd WARNING tpd24 WARNING tpdvis NOTE Missing tpdvis WARNING tpft WARNING tstatlabchgbase NOTE Missing tstatlabchgbase WARNING PROGRAM METADATA AS A KPI A SAS Program is a group of DATA steps or procedure steps, used in any combination, to guide SAS through a series of processes to read, transform, and output data. The SAS program can be easily turned into metadata similar to how we turned the SAS Log into metadata. Alexion has an SOP requiring good programming standards, not practices. These standards make our programs manageable and maintainable. We convert our SAS programs into metadata and check them for compliance with our SOPs through a process known as code review. In the example below, we convert SAS programs into searchable text and look for two items from our good programming standards. The results of this search are then converted into metadata and used as KPI about code review. In the first step, we use the FILENAME statement to associate a SAS file reference ( FILEREF ) with SAS programs. We will again use a wild card statement to read in all SAS programs found in a particular location. filename progs "C:\Users\amorv1982\NESUG2013\Programs\*.sas"; Once the FILEREF has been created, the first step is to convert the programs into searchable text using the IN- FILE and INPUT statement. The next piece of code is to create a variable to identify the program name using the filename of the first record from the program s FILEREF. data codervw; length filename $256 progname $32; retain progname; infile progs filename=filename eov=eov length=l; input progline $varying256. l; if _n_ eq 1 or eov then do; progname = scan(filename,-2,'.\'); eov = 0; The two conditional if/then loops provide a straight forward example of two items from Alexion s code review program however, our programming standards describe many more conventions to check for. This sample code is intended to demonstrate how your SAS program can be converted into searchable text data. In this first straight forward example, we are searching for the use of a library reference, also known as a LIBNAME. Our SOP only allows LIBNAMES to be used in initialization programs. Next, we search the end of macros and flag cases where the macro name is not specified on the MEND statement. Providing the name of the macro on the MEND statement makes the program more manageable. if substr(left(upcase(progline)),1,7)='libname' then do; 7

8 violation="library references found."; output; if substr(left(upcase(compress(progline))),1,6)='%mend;' then do; violation="mend Statement must include macro name."; output; From this, data about programming conventions, also known as code review metadata, can be summarized through the TABULATE procedure and presented as a KPI about the level of compliance with the SOP. proc tabulate data = codervw out = test; class violation progname; tables violation*progname, (n pctn) / rts =100; Figure 5 Code Review Metadata Summary of SOP Violations SOP Violation Program N % Library references found. compare MEND Statement must include macro name. chklog compare from this. Figure 5 shows a sample of the output from the TABULATE procedure above. These metadata provide information about each program s compliance with programming standards in a particular submission. We anticipate no findings as the bestcase scenario and document any deviations OPERATING SYSTEM METADATA AS A KPI An operating system ( O/S ) is a collection of software that supports a computer s basic functions. Alexion uses a Microsoft Windows 2008 R2 O/S and SAS software. Information about the SAS files is stored in a file manager application that is provided by Microsoft and referred to as Windows Explorer. Windows explorer provides the name, location, modification date/time, and size of all files stored on it. Figure 6 Results of PIPE Command Directory of C:\Users\amorv1982\NESUG2013\Programs 07/06/ :43 PM <DIR>. 07/06/ :43 PM <DIR>.. 07/03/ :00 PM <DIR> ANALYSIS 07/03/ :31 AM 63,184 chklog.log 07/03/ :31 AM 3,370 chklog.lst 07/05/ :03 PM 2,514 chklog.sas 07/06/ :25 PM 1,329 codervw.sas 07/05/ :43 PM 1,806 compare.sas 07/06/ :43 PM 1,329 pipe.sas 07/03/ :00 PM <DIR> SDTM Directory of C:\Users\amorv1982\NESUG2013\Programs 6 File(s) 73,532 bytes 4 Dir(s) 495,282,667,520 bytes free We begin again with a FILENAME statement but now include a PIPE command. PIPE commands enable you to capture data from a program external to SAS without creating an intermediate data file. Initializing this command and reading the file into a dataset, creates metadata about the directory specified in the FILENAME statement. Figure 6 shows the crude results of using the FILENAME statement with the PIPE command. The metadata set contains one text variable which can be searched to create other metadata. Since we are only concerned with pulling in metadata about our SAS programs, we use the INDEX function to only keep those records with.sas as a file extension. 8

9 filename path1 pipe "dir q ""C:\Users\amorv1982\NESUG2013\Programs"" 2>&1'"; data pipe1; infile path1 firstobs = 5 truncover; grepline $300.; if index(grepline, ".SAS") > 0; We use the sub-strand function, known as SUBSTR, to convert the date and time of the file into a single variable. The sub-strand function returns a substring of a text variable. We then use SCAN function to create a variable to store the file name. dttm =input(substr(grepline,1,20),mdyampm25.2); flnme = upcase(scan(substr(grepline,40),1,".")); format dttm mdyampm25.2; These metadata are not a powerful KPI alone, but can be used in conjunction with other metadata to enhance its value as a KPI. THE KPI UTILITY The goal of Clinical and Statistical Programming is to develop and validate the programs that create datasets, tables, listings, and figures for a regulatory submission. The measure of success for each of these outputs comes from a cleanliness of the SAS log, the status and independent programming results of COMPARE procedure output, and compliance with our SOPs. Each of these are quantifiable and can be measured through our validation metadata, log metadata, and code review metadata. Combining them with our O/S metadata, we can build a single dataset by merging the metadata sets together. A simple REPORT procedure can then be used to provide the overall status of a group of a programs used for a submission. To enhance the process of identifying records which are ready to be submitted, TRAFFIC LIGHTING is used. This is a way of using data values conditionally to visually format your report. ods tagsets.excelxp path=" C:\Users\amorv1982\NESUG2013\Output file="kpi.xml" style=xlsansprinter; options pageno=1 number orientation = landscape; ods tagsets.excelxp options( sheet_interval="none" sheet_name="kpi" width_fudge='1' default_column_width = "20" autofilter='all' autofilter_table = '1' frozen_headers = '1' frozen_rowheaders = '2' ); proc report data=out.kpi nowd headline headskip spacing=3 split='*' missing ls=161 ps=42 style(column) = [just=center font_face = courier] style(header) = [just=center font_face = courier font_weight = bold]; column _all_; define Pass / display; compute Pass; if upcase(pass) = "NO" then call define (_row_, 'style', 'style={background = light red}'); if upcase(pass) = "YES" then call define (_row_, 'style', 'style={background = light green}'); 9

10 endcomp; ods tagsets.excelxp close; ods listing; Figure 7 is a simple example of the output from our KPI Utility. In this example, we can see the programs on records two and four are not ready for submission. The program on record two is not ready because the O/S metadata discovered the development output was updated but not validated. The program on record four is not ready because it problems were identified in the log metadata, code review metadata, and the validation metadata. Figure 7 KPI Utility CONCLUSIONS There are many ways for us to extract information directly from our SAS objects and summarize the information. Key Performance Indicators (KPIs) help organizations track progress towards their goals by allowing them to summarize information about processes and objects. Each SAS programmer has a different goal depending on their organization. Data about their SAS logs, lists, programs and output can be easily extracted to summarize and track their progress towards their goal. REFERENCES: Amoruccio, Vincent. "Using Procedure-Based ODS Data Components in Statistical Reporting." Paper Print. < Amoruccio, Vincent. "The Y2K17 Bug! Using Metadata to Respond to PDUFA V Requirements." DS12Print. < Amrouccio, Vincent. "Could have, Would have, Should have! Adopting Good Programming Standards, Not Practices, to Survive An Audit." IB04Print. < IB04.pdf>. "Base SAS Glossary." N.p.. Web. 6 Jul < DelGobbo, Vincent. "Traffic Lighting Your Multi-Sheet Microsoft Excel Workbooks the easy way with SAS." Paper 153. Print. Howard, Neil. "Beyond Debugging: Program Validation." Paper 58-28Print. < 10

11 Newhouse, Rusell. "Validation and SAS Programming: Benefits of Using The System Life Cycle Method." n.d. n. page. Print. < Truong, Sy. "Effective Ways of Validating SAS Programs." Paper HW07Print. < ACKNOWLEDGMENTS I would like to thank Daniel Amoruccio for his unbounded support in making this paper possible. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc. in the USA and other countries. indicates USA registration. Other brand and product names are registered trademarks or trademarks of their respective companies. Acknowledgments like the two above go after your references. You should acknowledge the trademarks of the products discussed in your paper. You can also use this section to thank any individuals or organizations that assisted you in a substantive manner. CONTACT INFORMATION Your comments and questions are valued and encouraged. Contact the author at: Vincent J. Amoruccio Alexion Pharmaceuticals 352 Knotter Drive Cheshire, CT (203) AmoruccioV@alxn.com 11