NIST Risk Assessment for Part 11 Compliance: Evaluation of a GXP Case Study

NIST Risk Assessment for Part 11 Compliance: Evaluation of a GXP Case Study Monica Fanjoy* 109 Fairground Road, Holly Springs, NC 27540, USA Summary Current guidance for compliance with 21 Code of Federal Regulations (CFR) Part 11 requires a risk-based approach that is justified, documented, and addresses potential product quality and safety risks. Based on this guidance, different risk assessment methods can be used to comply with Part 11. This study presents a risk assessment of a Part 11-regulated computerized system using the techniques presented in the first public draft of the National Institute of Standards and Technology s (NIST) Risk Management Guide for Information Technology Systems. Results of this case study evaluation suggest that GXP-regulated industries might strengthen their overall risk management and compliance activities by adopting the NIST framework; however, they would need to address these three challenges: terminology differences, unclear direction with regards to product safety risks, and cumbersome documentation in the regulated environment. Copyright 2004 John Wiley & Sons, Ltd. Key Words computer security; computerized system; data integrity; GXP; NIST; Part 11; risk assessment *Correspondence to: Monica Fanjoy, 109 Fairground Road, Holly Springs, NC 27540, USA. E-mail: mofanjoy@yahoo.com This study was based on the First Public Exposure Draft of the reference noted as Reference [1]. The draft document is: National Institute of Standards and Technology. Risk Management Guide for Computer Security. NIST Special Publication 800-30. First Public Exposure Draft, June 2001. Introduction Current thinking for compliance with 21 Code of Federal Regulations (CFR) Part 11 (Part 11) requires a risk-based approach that is justified and documented and that addresses potential product quality and safety risks. Different risk assessment methods can be used to comply with Part 11. This study presents a risk assessment of a Part 11-regulated computerized system using the techniques presented in the first public draft of the National Institute of Standards and Technology s (NIST s) Risk Management Guide for Information Technology Systems [1]. This study compares and contrasts the NIST framework with the GXP- and Part 11- regulated environments. This paper presents a direct comparison of the terminology and frameworks used in the two environments. We selected a simple hypothetical GXP case study as a tool to evaluate the effects of the NIST approach in the regulated environment. We applied the NIST approach as outlined in the NIST Risk Management Guide for Information Technology Systems to the case study. We also discuss the implications of using the NIST guidance on risk assessment for computerized systems when used as a tool for Part 11 compliance in a GXP-regulated environment. Comparison of Frameworks and Terminology Table 1 compares major points regarding the framework of the two different approaches. Part 11-GXP requirements are regulatory driven requirements that focus on data integrity. In contrast, NIST is a voluntary program designed to DOI: 10.1002/qaj.291

248 M. Fanjoy help a broad range of industries improve computer security. The different focus is also apparent in the terminology employed within the two frameworks. Table 2 compares specific jargon found in NIST to terminology commonly used in the regulated environment. One of the most distinct differences is the use of the term computer security to mean meeting security goals in NIST and the use of the term security in the GXP environment to mean assuring data integrity. Table 1. Comparison of framework for NIST [1] and 21 CFR Part 11 for risk assessment of computerized systems NIST Draft Guidance 21 CFR Part 11-GXP Requirements Authority Voluntary Regulatory Scope Any industry GXP-regulated industries only Intent Computerized Data integrity system security Critical points No emphasis given Data integrity ensures health/safety Case Study Scenario description and controls The computerized system envisioned for this analysis is owned and operated by a small familyowned company in California that performs GXP-regulated studies. The company has a low turnover rate (less than 5% of the 40 employees turnover annually) and no track record of disgruntled employees. The system includes the server and three wired office client systems networked through a secure domain on the company local area network (LAN). The system supports a database that contains regulated scientific information. Database and network server are commonly used, commercially purchased, and marketed as Part 11 compliant. Controls for the system include those that are described below. Existing controls Existing controls that are intended for use as a simplified example for the case study are listed below. The list is not intended to be comprehensive because many topics (open systems, data retention, etc.) are not addressed. Table 2. Language intersection: summary of terminology differences between NIST and Part 11-GXP risk assessments of computerized systems NIST Draft Guidance GXPs and 21 CFR Part 11 Accountability Trace actions of entity GXPs require signature and date; Part 11 requires user-specific change control Assurance Confidence that goals for Ensuring compliance with GXP accountability, confidentiality, regulatory requirements availability, and integrity are met Availability Availability of data that can lead to Data integrity concerns, record unauthorized use or change retention, and availability for inspection Confidentiality Protection from unauthorized reading Business concern; No GXP regulatory or viewing of information requirement Integrity Protection against unauthorized Data integrity, protection against violation of system or data unauthorized changes Security Having characteristics and mechanisms With regard to GXP and Part 11, the that meet security goals term security refers to data integrity

NIST Risk Assessment for Part 11 Compliance 249 Physical and logical controls: Hardware physically secured by limited building access. Terminals and server are located away from window. Equipment is protected from power surges. Firewall. Anti-virus software. Encryption. Access restrictions. Individual-level user accountability: Unique combination of identification code and password for network and software. Periodic checking and rotation of password. Training on good password management. Policy to modify user account immediately following change of responsibility level, employment status, etc. Detection and reporting of unauthorized entry by locking user ID after repeated unsuccessful attempts. Software features include built-in audit trail that provides user accountability. System allows role-specific user capability authorization. Results Likelihood, Impacts, and Risk Risk determination is based on the severity of impacts and the likelihood of occurrence. According to NIST, moderate risk means that the potential problem results in discernable but recoverable unavailability, modification, disclosure, destruction of data or other system assets or loss of system services, resulting in transitory, yet important impact no personal injury. In contrast, the loss of data integrity is critical for GXP systems because it may cause injury to people through inappropriate approval or manufacture. Likelihood analysis Many risks were rated at low or moderate likelihood based on this particular scenario or the previously existing controls indicated in this case study. Rating the likelihood of significant risks from insiders as Low (see Table 3), for instance, is unrealistic in most cases, but management believes it appropriate in this case because this hypothetical scenario describes an exceptionally low personnel turnover rate of less than 5%. Impact assessment The impact assessment categorizes the risks and assigns a valuation that is used in conjunction with the likelihood for the risk determination (see Table 4). Risk determination Risk determination is based on the severity of impacts and the likelihood of occurrence (see Table 3. Likelihood of significant risks Scenario Rating Rationale Hacker Moderate Have experienced viruses etc. Criminal Low Unprocessed scientific data effortful for trained audience, little use to general audience. Data corporate sensitive only, eventually published in public Insider Low Based on low staff turnover rate and high employee satisfaction Chemical spill Low Rare occurrence due to controls Electrical storm Moderate Several systems were affected by storms previously Earthquake Low Rare occurrence on annual basis

250 M. Fanjoy Table 4. Impact Assessment, Description, and Categorical Analysis Scenario Impact Categories* Description Valuation Hacker I A C A A Results in discernable but recoverable unavailability, Moderate v c modification, disclosure, destruction of data or other system assets or loss of system services, resulting in transitory, yet important impact no personal injury Criminal I A C A A Same as above Moderate v c Insider I A C A A Unauthorized insider access would have High to v c largest impact such as destruction of data, moderate disclosure, or modification that may include loss of integrity, availability, and confidentiality. Discernability resulting from required audit trails takes this impact from a high to moderate level; if a toggle to turn off the audit trail capability exists and is disabled it could be considered a higher risk. For regulated, studies an insider would have a moderate risk in this situation Chemical spill / Access from several locations minimizes Low, I C A electrical storm / potential impacts. May incur replacement except for earthquake and validation costs. Damage to server undetecmight cause loss of integrity or availability. ted loss of Food/chemicals are prohibited in room function housing server and location is protected. The data are regularly archived on tapes and stored at another location, minimizing potential data loss. An undetected loss of hardware functionality may result in loss of integrity, assurance, and confidentiality *I, Integrity; Av, availability; C, confidentiality; A, assurance; Ac, accountability. Table 5. Risk determination based on combined results of likelihood and impact analysis Scenario Likelihood Impact Risk Determination Hacker Moderate Moderate Moderate Criminal Low Moderate Low Insider Low Moderate Low Chemical spill Low Low Low Electrical storm Moderate Low Low Earthquake Low Low Low Table 5). The risk determination was based on the risk determination construct provided in the NIST guidelines. The construct designates a determination based on the likelihood and impact. Analysis of current controls As suggested by NIST, the existing controls were analyzed by category and type to ensure that a

NIST Risk Assessment for Part 11 Compliance 251 Table 6. Categorical control analysis matrix for selected requirements Requirement Category Type Control description Individual-level T D, P Unique combination of identification code and user password for network and software Accountability T, O P Periodic checking and rotation of password M, O P Training on good password management O, M P Policy to modify user account immediately following change of responsibility level, employment status, etc. T, M D Detection and reporting of unauthorized entry by locking user ID after repeated unsuccessful attempts T D Software features include built-in audit trail that provides user accountability T P System allows role-specific user capability authorization T, technical; O, operational; M, management; P, preventive techniques; D, detection and monitoring. full range of controls was used (see Table 6). Based on this review, regulatory requirements are initially satisfied. For simplicity, this analysis assumes that omitted requirements are met. Implementation involves a tiered approach to compliance. Also, a combination of categories (prevention, detection, and monitoring) and types (technical, operational, and management) of control methods were applied. Categories of controls are technical (T), such as access control and antivirus software; operational (O), including training and procedures; and management (M), which prevent or manage risks. Types of controls are preventive techniques (P) and detection and monitoring (D). Residual risk analysis Existing controls were reviewed for any gaps. These residual risks were communicated to responsible management. Table 7 shows the identified gap or risk, provides the compliance recommendation, documents management decision, and provides justification. Implications with good practices The detailed and extensive documentation process proposed by NIST may prove cumbersome in a regulated environment; however, the systematic process reveals inconsistencies that the GXP compliance practices could potentially overlook. In conclusion, GXP-regulated industries might strengthen their overall risk management and compliance activities by adopting the NIST framework; however, they would need to address these three challenges: terminology differences, unclear direction with regards to product safety risks, and cumbersome documentation in the regulated environment. Based on this case study, a system-driven risk assessment such as the NIST framework can provide a comprehensive analysis that reveals discrepancies and gaps within the assurance process and the NIST framework is flexible enough to manage regulatory and business risks in combination. Documentation that synthesizes pertinent information across various documents and procedures at various levels also ensures comprehensive assurance. Integrated and systematic assurance makes sense for high-risk systems because it increases the likelihood of comprehensive assurance; however, the NIST process is extensive and detailed. As a result, the NIST framework may potentially generate a large volume of supporting documentation that would prove cumbersome in a regulated environment. Specifically, creating and archiving documents such as the threat identification, likelihood analysis, impact assessment, risk determination, and control analysis might significantly increase labor time involved in the

252 M. Fanjoy Table 7. Simulation of management decisions on residual risk Residual risks Recommendation Management decision Rationale Undetected loss of Use routine system Implement automated Budget does not allow functionality may checks to routinely system integrity check for manual checks, so result in loss of data assess integrity that comes with use automated checks integrity, accountability, Implement a disaster software. Test after on a limited basis. At and assurance recovery policy that occurrence of specified the present, periodic assures functionality events as required by surveillance testing is Add periodic disaster planning also too costly reverification testing policy to preventive surveillance and maintenance The two users Document background Implement policy Both employees have authorized to disable check as preventive regarding audit trail demonstrated the audit trail measure. Implement a use. Background check reliability by 20+ present a high risk policy regarding use is unnecessary expense years with the and disengagement of company audit trail Office client in Move or implement Move to secure Relocating the client shared office space local physical security location is less costly than is not secure from measures in shared remodeling facility employees of office space Company B No indication to Implement policy that Agree Employees are aware employees that equates electronic and of this; however, it is electronic and handwritten appropriate to handwritten signatures document this in a signatures are equal written policy compliance process. Moreover, the NIST framework provides little direction on data integrity and health/safety risks that are inherent in GXPregulated industries and contains language that is inconsistent with terminology set forth in Part 11 s GXP predicates. References 1. National Institute of Standards and Technology (NIST). Risk Management Guide for Information Technology Systems. NIST Special Publication SP800-30. http://csrs.nist.gov/publications/ nistpubs/800-30/sp800-30.pdf