Joseph B. Baugh, PhD, PMP, CISA, CISSP, CRISC, CISM Senior Compliance Auditor Cyber Security WECC: Vancouver WA Office

Size: px

Start display at page:

Download "Joseph B. Baugh, PhD, PMP, CISA, CISSP, CRISC, CISM Senior Compliance Auditor Cyber Security WECC: Vancouver WA Office"

Paula Strickland
5 years ago
Views:

1 Joseph B. Baugh, PhD, PMP, CISA, CISSP, CRISC, CISM Senior Compliance Auditor Cyber Security WECC: Vancouver WA Office CIP-101: CIP-002 v3 to v5 Transition WECC Office: Salt Lake City UT September 24-25, Speaker Introduction Dr. Joseph B. Baugh o 40 years Electrical Industry Experience Transmission Lineman NERC Certified System Operator Information Infrastructure Design & Implementation IT Manager & Power Operations Manager 20 years Information Technology & Security Experience Project Manager & IT Program Manager PMP, CISSP, CISA, CRISC, CISM, NSA-IAM/IEM certifications o 17 years Teaching Experience (Multiple Schools) Degrees: PhD, MBA, BS-Computer Science Information Technology and IT Security courses Business Strategy, Leadership, & Management courses PMP, CISSP, CISA, CISM, ITIL, & Cisco certification prep courses Project Management courses (c) 2013 Dr. Joseph B. Baugh 1

WECC CIP-101 Disclaimer The WECC Cyber Security team has created a mythical Registered Entity, Billiam Power Company (BILL) and fabricated evidence to illustrate key points in the CIP audit processes.

2 WECC CIP-101 Disclaimer The WECC Cyber Security team has created a mythical Registered Entity, Billiam Power Company (BILL) and fabricated evidence to illustrate key points in the CIP audit processes. Any resemblance of BILL to any actual Registered Entity is purely coincidental. All evidence presented, auditor comments, and findings made in regard to BILL during this presentation and the mock audit are fictitious, but are representative of audit team activities during an actual audit. 3 Agenda Class Introductions o Name, Title, Organization, Interest in CIP-002 CIP Mock Audit Overview Review CIP Transition Guidance Review CIP Requirements Review CIP-002 Team audit approach The BILL Mock Audit Questions 4 (c) 2013 Dr. Joseph B. Baugh 2

3 CIP-101 Mock Audit Overview BILL has identified and documented a list of Critical Assets through an application of the CIP Impact Rating Criteria (IRC) per the recent NERC v5 Transition Guidance BILL has identified associated Critical Cyber Assets BILL requires a full Compliance audit on CIP through CIP o First week: Discovery phase at WECC offices o Second week: Compliance audit at BILL office 5 CIP-101 Mock Audit Overview Mock Audit squeezes 2 weeks of audit activities into a few hours. Sample DR s Mock Interview Site Visits Use the RSAW as the guiding document Present and review evidence for each requirement What do YOU think is the appropriate finding for each requirement? 6 (c) 2013 Dr. Joseph B. Baugh 3

4 CIP Overview CIP is the first step in the CIP Compliance trail All Registered Entities who perform the BA, GO, GOP, LSE, TO, TOP, and/or TSP registered functions are required to be compliant with CIP CIP replaces LSE with the DP function, TSP function drops out. o However, for this mock audit, we are only using the CIP R1 and accompanying Attachment 1 IRC to identify and document a list of Critical Assets and remain compliant with CIP-00x-3, so the v3 functions are still valid. Some entities find they are only required to be compliant with CIP & CIP R2. o Typically requires a reduced scope audit that is conducted at WECC offices or other locations as necessary. 7 Current CIP Requirements: R1 R1: Identify and document a risk-based assessment methodology (the RBAM). o Include procedures and evaluation criteria (R1.1) o Consider all BES Assets, pay close attention to those assets listed in R1.2 (see R R1.2.7). 8 (c) 2013 Dr. Joseph B. Baugh 4

5 CIP-002-3: R1, R1.1, R1.2 9 CIP-002-3: R Use these asset types as represented by your inventory of BES Assets in your application of the CIP IRC during the transition period. 10 (c) 2013 Dr. Joseph B. Baugh 5

6 CIP Requirements: R2 Apply the RBAM to a list of your BES Assets to identify and document a list of Critical Assets. Review the list of Critical Assets at least annually and update as necessary. 11 CIP-002-3: R2 12 (c) 2013 Dr. Joseph B. Baugh 6

2) BILL chooses Option 2 to identify and document a list of Critical Assets from its inventory of BES

7 CIP v5 Transition Guidance Cyber Security Standards Transition Guidance (NERC, 2013 Sept 5, p. 2) 13 CIP-00x-5 Transition Guidance Cyber Security Standards Transition Guidance (NERC, 2013 Sept 5, p. 2) BILL chooses Option 2 to identify and document a list of Critical Assets from its inventory of BES Assets. The CIP Senior Manager documents this choice prior to implementation. 14 (c) 2013 Dr. Joseph B. Baugh 7

BILL Documents Its CAID Choice 16 CIP-002-5 Transition Changes Each Responsible Entity shall implement a process that considers each of the following assets for purposes of parts 1.1 through 1.

8 BILL Documents Its CAID Choice 16 CIP Transition Changes Each Responsible Entity shall implement a process that considers each of the following assets for purposes of parts 1.1 through 1.3: [Violation Risk Factor: High][Time Horizon: Operations Planning] o i. Control Centers and backup Control Centers; o ii. Transmission stations and substations; o iii. Generation resources; o iv. Systems and facilities critical to system restoration, including Blackstart Resources and Cranking Paths and initial switching requirements; o v. Special Protection Systems that support the reliable operation of the Bulk Electric System; and o vi. For Distribution Providers, Protection Systems specified in Applicability section above. (Not applicable for transition) Ensure all asset types described in CIP R1.2.1 through R1.2.7 are included in the above categories. If not, add them to the evaluation process (per slide 10). (c) 2013 Dr. Joseph B. Baugh 8

9 17 CIP Transition Changes Each Responsible Entity shall implement a process that considers each of the following assets for purposes of parts 1.1 through 1.3: o 1.1. Identify each of the high impact BES Cyber Systems according to Attachment 1, Section 1, if any, at each asset; o 1.2. Identify each of the medium impact BES Cyber Systems according to Attachment 1, Section 2, if any, at each asset; and o 1.3. Identify each asset that contains a low impact BES Cyber System according to Attachment 1, Section 3, if any (a discrete list of low impact BES Cyber Systems is not required). CIP R1.1-R1.3 are not applicable for the transition period. 18 CIP R3 After identifying and documenting a list of Critical Assets by applying the IRC to BILL s inventory of BES Assets, the process reverts to the current mandatory and enforceable CIP R3 processes. Use the list of Critical Assets (CA s) developed by applying CIP R1 and the IRC to develop a list of Cyber Assets associated with each Critical Asset and apply the current Critical Cyber Asset Identification [CCAID] methodology to determine if any Cyber Assets are essential to the operation of the Critical Asset. (c) 2013 Dr. Joseph B. Baugh 9

CIP-002-3: R3 19 20 CIP-002-3 Requirements: R3 For each such Cyber Asset that is deemed essential, consider: o R3.1: Does it use a routable protocol to communicate outside the ESP? or o R3.

10 CIP-002-3: R CIP Requirements: R3 For each such Cyber Asset that is deemed essential, consider: o R3.1: Does it use a routable protocol to communicate outside the ESP? or o R3.2: Does it use a routable protocol within a control center? or o R3.3: Is it dial-up accessible? o If any of the above are true, the Cyber Asset is a CCA. Review the list of CCAs at least annually and update as necessary. (c) 2013 Dr. Joseph B. Baugh 10

CIP-002-3 Requirements: R4 The senior manager or delegate (as defined in CIP-003-3 R2) must approve at least annually: o The RBAM (not applicable under Option 1 or 2) o The list of Critical Assets o

11 CIP Requirements: R4 The senior manager or delegate (as defined in CIP R2) must approve at least annually: o The RBAM (not applicable under Option 1 or 2) o The list of Critical Assets o The list of CCAs, even if such list is null. The entity may determine it has no Critical Assets or associated CCAs The entity must maintain signed and dated records of the approvals listed above. 21 CIP-002-3: R4 22 (c) 2013 Dr. Joseph B. Baugh 11

12 CIP Audit Team Approach Review the application of the IRC to identify and document a list of Critical Assets Audit to the Standard. Review the Evidence: o Current RBAM o Current list of Critical Assets o Current list of CCAs, even if such list is null. o Records of current and prior approved versions of the above documents (the Bookends) DR for additional information, as needed. 23 WECC Audit Team Approach Use a methodical approach to deliver consistent results across all entities Use the RSAW supplied by the entity as working papers to document the audit and findings Review Initial Evidence package supplied by the entity o Attachment G 24 (c) 2013 Dr. Joseph B. Baugh 12

13 Prior to the selection of an option, provide all versions of the RBAM in force during the audit period up to the date of selection. After a transition option is declared, entities should attach a copy of the CIP Senior Manager statement and the annual application of either the BLC or the IRC depending on choice in lieu of the RBAM. Initial Evidence: Attachment G 25 WECC Audit Team Approach Submit Data Requests (DR s) for any additional information that will support the entity s compliance efforts, e.g.: o One-line diagrams (we ll see the BILL one-line later) o Prior documentation to provide bookends o Initial list of Cyber Assets at each Critical Asset identified in R2. o Address any questions or concerns 26 (c) 2013 Dr. Joseph B. Baugh 13

14 WECC Audit Team Approach Review the RBAM or application of the IRC (R1), list of CA s (R2), lists of CCA s, even if such lists are null (R3) If full Compliance audit: o Hold interviews with the entity s CIP SMEs o Site visits (Trust, but Verify) Validate annual approval documentation (R4) Submit DR s, if needed, to clarify compliance Determine findings (NF, PV, or OEA) Discuss findings with entire Cyber Security Team Complete RSAW Prepare CIP audit report (ATL & CPC) 27 CIP-101 Mock Audit Walk through audit process in more detail Explain the differences between a reduced scope off-site audit and a full Compliance audit The Mock Audit simulates a Compliance audit of Billiam Power Company [BILL] BILL is registered with NERC as a BA, GO, GOP, LSE, TO, TOP, TP, and TSP. 28 (c) 2013 Dr. Joseph B. Baugh 14

15 Review Initial Evidence Received from the entity in the initial evidence package Response to data requests in Attachment G Information contained in entity response to the RSAWs Sets the stage for the initial audit review o Discovery phase at the WECC offices Followed up by additional Data Requests as needed 29 The BILL System (from entity report) Billiam Power Company s (hereafter referred to by its NERC acronym, BILL) Balancing Authority (BA) area is effectively within the boundaries of the three counties on the western edge of Some State, bordered by Another State on the north and the Almost Mountains on the East and South. These three counties occupy about 15% of the land area of the state and contain about 20% of the state's population. BILL is registered as a BA, DP, GO, GOP, LSE, TO, TOP, TSP 30 (c) 2013 Dr. Joseph B. Baugh 15

16 31 The BILL System (continued) BILL s primary generation station is located in eastern Whatchamacallit County. The BILL generation station has two 1,000 MW fossil fuel generating units. The output of these units supports BILL s native load and any available excess energy is marketed throughout the WECC Interconnection. BILL owns and operates nine Combustion Turbines (averaging 30 MWs each) located near various consumer load centers throughout the service territory. These CT s are primarily used as peaking units and for voltage and frequency support during the summer months. BILL also owns and operates the BILL-3 Hydroelectric plant on the Sweet William River. BILL-3 has a nameplate rating of 100 MW. This hydro unit is Blackstart capable and is connected to the BILL Generation Station through a dedicated 115 kv line that runs 87 miles from Sub3 to Sub1. Total BILL generation capacity is 2,380 MWs. The BILL System (continued) There are two synchronous 345 kv interties with adjacent BA s that define the BILL BA area. These ties are with XXXX Electrical Utility and YYYY Federal Power District at Sub1, which is adjacent to the BILL Generation Station. The BES portion of BILL's BA area, its 345 kv, 230 kv, and 115 kv facilities, include 190 miles of 345 kv transmission lines, 450 miles of 230 kv lines, and 973 miles of 115 kv lines. BILL owns and operates two 345kV substations, kv substations, and kv substations throughout its service territory. BILL serves its native residential and commercial load through its 115 kv and 230 kv transmission facilities. The Generation and Transmission facilities are monitored and managed from the Primary Control Center (PCC) located at the corporate headquarters in Big Bill City. BILL also maintains a hot stand-by Back-up Control Center (BUCC) located in its operations center in Little Bill City, which is approximately 50 miles from the PCC. BILL is a summer peaking BA and BILL's BA all-time area peak load was recorded on July 20, 2010 at 2,482 MWs. 32 (c) 2013 Dr. Joseph B. Baugh 16

17 BILL One-Line Diagram BILL s Critical Asset Identification The first step in a normal CIP audit is to review the RBAM. The second step is to review the Critical Asset Identification Methodology [CAID]. The CAID is typically included as part of the RBAM, but the audit team will review the application of the IRC under this scenario o Starts with an overall list of entity BES Assets. o Uses the IRC to identify and document a list of Critical Assets. Review BILL s 2013 list of Critical Assets derived from the IRC and compare it to the previous lists derived from the RBAM. Were applicable BES Assets evaluated relative to IRC criteria or 2.8? [If Option 1 selected, then 1.3, 1.8, 1.9, 1.10] o Did BILL demonstrate coordination with the applicable registered function(s)? o If not, should we submit a data request? (c) 2013 Dr. Joseph B. Baugh 17

BILL BES Assets: 2012 Control Centers 35 BILL BES

23 Critical Assets Net Changes 45 Control Centers o No change Substations o Add 4 (Subs 4, 7, 8, 11) o Drop 1 (Sub 3, related to blackstart) Generation Units o Drop blackstart unit Special Protection Systems o No change 46 R2: Critical Asset Review Questions Did BILL apply the IRC appropriately? Does BILL need to confer with its RC, PA, or TP to consider any Critical Assets relative to Criteria 2.3, 2.6, or 2.8? Did BILL review its list of Critical Assets at least annually? Did BILL update the list as necessary? Application Questions o Did BILL consider all BES Assets in R1.i through R1.vi? o Did BILL review and evaluate all BES Assets through the IRC? o Did BILL clearly identify and document all Critical Assets? Is any additional information necessary? o If so, do we submit a DR? (c) 2013 Dr. Joseph B. Baugh 23

BILL s Critical Cyber Asset Identification 47 The third step in a CIP-002-3 audit is to review the Critical Cyber Asset Identification Methodology [CCAID].

24 BILL s Critical Cyber Asset Identification 47 The third step in a CIP audit is to review the Critical Cyber Asset Identification Methodology [CCAID]. Under this scenario, the CCAID should be maintained as a discrete document. Starts with the identified list of Critical Assets. Uses the CCAID procedures and evaluation criteria to identify and document a list of Critical Cyber Assets, even if such list is null. Review the BILL Critical Cyber Asset Identification Methodology Review List of Critical Cyber Assets 2012 CCAs: Primary Control Center 48 (c) 2013 Dr. Joseph B. Baugh 24

2013 CCAs: Primary Control Center 49 2012 CCAs:

2013 Null Lists CCAs: Generation & Subs 55 56 R3: Critical Cyber Asset Review Questions Did BILL use the Critical Asset list developed in R2 to identify Critical Cyber Assets?

Did BILL review the list at least annually and update the list as necessary? Application Questions o Did BILL consider all Cyber Assets located at its Critical Assets for evaluation through the CCAID?

28 2013 Null Lists CCAs: Generation & Subs R3: Critical Cyber Asset Review Questions Did BILL use the Critical Asset list developed in R2 to identify Critical Cyber Assets? Did BILL apply its Critical Cyber Asset Identification Methodology [CCAID] appropriately to consider all Cyber Assets supporting the reliability function of the Critical Asset? Did BILL review the list at least annually and update the list as necessary? Application Questions o Did BILL consider all Cyber Assets located at its Critical Assets for evaluation through the CCAID? o Did BILL consider R3.1-R3.3 for all Cyber Assets considered essential to the operation of the Critical Asset o Did BILL clearly identify and document all Critical Cyber Assets? Are any DR s necessary? o If so, what additional information is required? (c) 2013 Dr. Joseph B. Baugh 28

29 BILL s Annual Approvals The fourth step in a CIP audit is to review the annual approvals of the RBAM, the list of Critical Assets, and the lists of Critical Cyber Assets, even if such lists are null. Review the BILL 2012 Annual Approvals Review the BILL 2013 Annual Approvals 57 R4: Annual Approval Review Questions Did the BILL CIP Senior Manager or delegate approve at least annually the RBAM, the list of Critical Assets, and the lists of Critical Cyber Assets, even if such lists are null? Application Questions o Did BILL provide evidence of annual reviews and approvals? Are any DR s necessary? o If so, what additional information is required? 58 (c) 2013 Dr. Joseph B. Baugh 29

30 On-Site Activities: The Interview Set up through an interview DR the prior week Typically held on Monday of the on-site week immediately after the opening presentation Examines the entity s understanding of and approach to R1-R4 Cover any areas of concern raised through the initial evidence review Schedule follow-up interview(s), if needed, after the site visits 59 On-site activities: Mock Interview Need four volunteers o You are BILL SMEs o No, you don t get to practice We will ask a series of questions that we generally ask all CIP-002 SMEs Also ask questions of concern, if indicated by the initial review of the evidence The Interview Question Set 60 (c) 2013 Dr. Joseph B. Baugh 30

31 On-site activities: Mock Interview What did we learn from the interview? What was the key issue from an audit perspective? Should we find a PV for this issue? Why or why not? 61 On-Site Activities: Site Visit Set up through a site visit DR the prior week Itinerary determined through review of the initial evidence Trust, but verify. Why? Depending on entity size, 100% validation or a statistical sampling Where? o Control Centers o Generation Facilities o Transmission Facilities 62 (c) 2013 Dr. Joseph B. Baugh 31

32 On-Site Activities: Site Visit 63 Who? o CIP Sub-Team Validates lists of CCAs, even if such lists are NULL Works in conjunction with CIP-005-3a sub-team o CIP-005-3a Sub-Team Validates Electronic Access Points [EAPs] and Electronic Access Control and Monitoring devices [EACMs]. Confirms ESP boundaries o CIP-006-3c Sub-Team Validates PSPs and Physical Access Controls, such as PACS, cameras, logs, etc. My colleague, Wally Magda, provided an overview on CIP-006-3c audit activities earlier. 64 On-Site Activities: CIP Site Visit What? o Validate lists of CCAs o Validate null lists of CCAs o Look for aberrations from the lists o Hold informal interviews with entity SMEs When? o Sometimes during the off-site week. o Typically on Tuesday of the on-site audit o May also be on Wednesday depending on sites visited, distances traveled, etc. (c) 2013 Dr. Joseph B. Baugh 32

33 On-Site Activities: BILL Site Visits Visit the Primary and Backup Control Centers o 100% validation of CCAs in both locations o Talk to Operators & SMEs Visit the BILL Generation Station, SUB1, SUB2, SUB4, SUB7, SUB8, and SUB11. o Validate the Null Lists of CCAs o Talk with entity SMEs Site Visit Questions o Why validate all CCAs at a given site? o Why validate Null lists of CCAs? o Why ask questions of entity SMEs? 65 BILL Site Visits: Control Centers Visited the Primary Control Center o 100% validation of CCAs o Found nothing out of the ordinary. Visited the Backup Control Center o 100% validation of CCAs o Found nothing out of the ordinary. 66 (c) 2013 Dr. Joseph B. Baugh 33

34 Site Visits: Generation Units Visited BILL Generation Station o Validated Null list of CCAs o Found nothing out of the ordinary Site Visits: Substations Visited Sub1 o 100% validation of CCAs o Found nothing out of the ordinary. Visited Sub2 o Validated Null list of CCAs o Noticed something strange here. Visited Subs 4, 7, 8, & 11 o Validated Null list of CCAs o Noticed something strange at each of these substations too. (c) 2013 Dr. Joseph B. Baugh 34

What was the main concern with the unexpected devices? Should we DR for additional information?

35 Site Visits: What Did We See? What is this device and what is it doing here in the subs? 69 On-Site Activities: Site Visit What did we learn from the site visit? Why do we validate Null lists of CCAs? What was the main concern with the unexpected devices? Should we DR for additional information? o Tour Notes DR Would another interview be more effective? Does this situation call for an R3 PV finding? Why or why not? 70 (c) 2013 Dr. Joseph B. Baugh 35

36 71 Discussing the Findings Discuss with whole Cyber Security Team Is there a PV for the undocumented devices? o R2: Undeclared Critical Assets The Combustion Turbines Does the entity have documentation from its TP or PA/PC that exempts the CTs from Criterion 2.3? o R3: Undeclared Critical Cyber Assets The Substation Modems Determine the scope of a PV o How do we do this? Complete the CIP Findings Table in RSAW Submit to the ATL and CPC for the Closeout Presentation Value-Added Activity: Feedback WECC Audit Teams never Prescribe Solutions, but we do describe: o Brief entities on findings o Encourage good security practices o Discuss examples of industry best practices o Identify areas of concern, which may not be violations, but which could stand improvements o Provide suggestions, when appropriate Support development of a sustainable compliance culture 72 (c) 2013 Dr. Joseph B. Baugh 36

37 Audit Documentation: The RSAW An auditor is judged by the quality of his or her working papers. o Complete the RSAW o Document findings o DR for any final needed information 73 Audit Documentation Auditors review evidence, find facts, and report findings o Turn PVs over to the Enforcement team o Enforcement team depends heavily on the quality of auditor documentation Be Literate, be Concise, but above all else, Be Accurate. If it s not written down, it didn t happen. 74 (c) 2013 Dr. Joseph B. Baugh 37

38 Post-Audit Auditor Activities The Audit Report o Work with ATL & CPC o Verify findings and other information related to audited standard(s) Document findings in webcdms o PV & OEA findings only Work with WECC Enforcement personnel to support Investigations as SME for audit processes and findings 75 Post-Audit Auditor Activities Participate in entity Outreach activities, such as this event and CIPUG meetings Be available to address entity questions/ comments Work at National level o CCWG o Drafting teams o Comment on new Standards, CANs, etc. o Attend and present at conferences 76 (c) 2013 Dr. Joseph B. Baugh 38

39 Summary Audit to the Standard Provide useful feedback to the entity Prepare a valid report Be available to CIP personnel at the entities Work at National level 77 Remember the Auditor s Mission Just the facts, Ma am, Just the facts! 78 (c) 2013 Dr. Joseph B. Baugh 39

References NERC. (2013 September 5). Cyber Security Standards Transition Guidance (Revised). Retrieved from http://www.nerc.

40 References NERC. (2013 September 5). Cyber Security Standards Transition Guidance (Revised). Retrieved from ResourcesDL/Cyber%20Security %20Standards%20Transition%20Guidance %20(Revised).pdf 79 Questions? Joseph B. Baugh, Ph.D., PMP, CISA, CISSP, CRISC, CISM Senior Compliance Auditor - Cyber Security Western Electricity Coordinating Council (WECC) 7400 NE 41st Street, Suite 160 Vancouver, WA jbaugh (at) wecc (dot) biz (C) (O) (c) 2013 Dr. Joseph B. Baugh 40

Iden%fying & Audi%ng Low Impact BES Assets: A Mock Audit BC Outreach Webinar: Session 2 Salt Lake City UT January 9, 2018

Iden%fying & Audi%ng Low Impact BES Assets: A Mock Audit BC Outreach Webinar: Session 2 Salt Lake City UT January 9, 2018 Joseph B. Baugh, PhD Senior Compliance Auditor Cyber Security Western Electricity