Annual Service Report. Performance Year 2016/17

Performance Year 2016/17 Submitted: 31/08/2017

Contents 1 Context... 4 2 Introduction... 5 3 Performance Year 2016/17 Context... 6 3.1 Key Events... 6 4 DCC Service Delivery Performance... 8 4.1 Operational Readiness... 8 4.2 Operating the DCC Service Desk... 8 4.3 Maintenance Windows... 9 4.4 Engagement... 9 4.5 SMKI Developments during... 10 4.6 DCC s Assurance of Smart Metering Wireless Access Network (SM WAN) Coverage by the CSPs... 10 4.7 SEC Code Performance Measures... 12 4.8 Performance of Services Provided to SEC Parties... 15 5 DCC Programme Performance... 30 5.1 SMETS2 Programme... 30 5.1.1 Overall Performance on SMETS2 Programme... 30 5.1.2 Release 1.2 DCC Live... 30 5.1.3 Release 1.3 (R1.3)... 37 5.1.4 Performance against Milestones... 39 5.2 Planned Future Releases... 41 5.3 Enrolment and Adoption... 42 5.3.1 Initial Enrolment Project Feasibility Report (IEPFR)... 43 5.3.2 Enrolment Option Testing (EOT)... 44 5.4 Centralised Registration Service (Switching Programme)... 45 6 Performance of Change Management... 46 6.1 Overview of Change Management... 46 6.2 SEC Modifications... 48 6.3 Performance of Commercial Management on Major Contracts... 49 6.1 Future Incentives... 50 6.1.1 The Operational Performance Regime... 50 7 External Service Provider Performance... 51 7.1 Arqiva Smart Metering... 60 7.1.1 Overview... 60 7.1.2 Delivery of SMETS 2 Releases... 61 DCC Public Page 2 of 118

7.1.3 Update on PY 2015/16 Remediation Plan... 62 7.1.4 Performance against Contractual Milestones... 62 7.1.5 Operational Performance... 62 7.2 British Telecommunications Plc... 63 7.2.1 Overall Performance... 63 7.2.2 Milestone Analysis... 63 7.2.3 Operational Performance... 64 7.3 Capita Business Services... 64 7.3.1 Overview... 64 7.3.2 Billing System Service Provider... 64 7.3.3 Office Space / Facilities, Resources & Consultancy Services Provider... 65 7.3.4 Capita IT Enterprise Services... 66 7.3.5 Overview... 66 7.3.6 Delivery during... 66 7.4 CGI IT UK... 68 7.4.1 Overview... 68 7.4.2 Delivery of SMETS2 Releases... 69 7.4.3 Update on PY 2015/16 Remediation Plans... 70 7.4.4 Performance against Contractual Milestones... 70 7.4.5 Operational Delivery... 71 7.5 Critical Software Technologies... 72 7.5.1 Overview... 72 7.5.2 Parse and Correlate... 73 7.5.3 GBCS Interface Testing... 73 7.5.4 File Signing Utility... 74 7.5.5 Enrolment Options Testing... 74 7.6 Telefonica UK... 74 7.6.1 Overview... 74 7.6.2 Delivery of SMETS 2 Releases... 75 7.6.3 Remediation Plans... 76 7.6.4 Milestone Analysis... 77 7.6.5 Operational Performance... 77 Appendix A External Service Provider Feedback... 78 Appendix B Contractual Milestones (Arqiva, CGI, Telefonica)... 109 Appendix C Appendix C - R1.2 Test Phases... 117 DCC Public Page 3 of 118

1 Context 1. In November 2016 DCC launched the data and communications infrastructure to enable the nationwide rollout of smart meters. The service will provide a single national communications infrastructure to allow suppliers to install meters in homes and small businesses across Great Britain. This national infrastructure is expected to play a central role in the transition to a low carbon, flexible UK energy industry. 2. The DCC network offers a secure, consistent service for all energy suppliers and avoids the complexity and duplicated costs of energy suppliers procuring their own networks. It will provide the information that will enable the industry to develop innovative new services and products in the future. 3. DCC is a skilled delivery body that procures and delivers services on behalf of the energy sector. In doing so, DCC manages a complex supply chain of core delivery partners. The diagram below shows a simplified version of DCC s current supply chain. 4. The purpose of this report is to provide an assessment of the service performed by DCC and its External Service Providers 1 during the preceding Performance Year (PY). The PY for this ran from 1 April 2016 to 31 March 2017. 1 A list of External Service Providers and their feedback on the report is included in Appendix A. DCC Public Page 4 of 118

2 Introduction 5. The has been developed in accordance with Condition 34 of the Smart Meter Communication Licence (the Licence). In accordance with Condition 34.9 of the Licence, External Service Providers were provided with a copy of the Report 2 in draft and were invited to comment on the relevant sections. DCC has completed a review of these comments and has outlined to what extent they have affected the drafting of the in Appendix A. If a factual error or omission had been made the text was updated accordingly. 6. The includes: Summary of overall DCC performance, split into: Performance of Service Delivery Performance of Programme Delivery Reporting on the context for the 2016/17 Performance Year A description of the role of each External Service Provider with an assessment of performance, which may include an additional summary of: Overall performance Change management Performance against contractual milestones (where applicable) Remediation plans (where applicable) Milestones (where applicable) Operational performance (where applicable). 2 External Service Providers received sections 1-6 and the part of section 7 pertaining to their organisation. DCC Public Page 5 of 118

3 Performance Year 2016/17 Context 7. The activities delivered by the DCC in PY2016/17 included the delivery of DCC Live for the SMETS2 Service in November 2016 through Release 1.2 (R1.2), and the DCC operation to support the scaling up of the live service. In addition, significant progress was made towards Release 1.3 (R1.3), which will be delivered in PY 2017/18, and will complete the core SMETS2 smart meter communication service as prescribed in the Smart Energy Code (SEC). 3 8. DCC has also made good progress towards developing solutions to support the implementation of Smart Metering through two additional projects: Enrolment and Adoption of SMETS1 meters and Dual Band Communications Hubs. 9. Finally, DCC has established a dedicated team of specialists to contribute to Ofgem s Switching Programme, and has developed a project to design and procure a new Centralised Registration Service to enable a faster and more reliable service for energy consumers to switch energy suppliers. Following consultation, DCC s Switching Business Case was baselined in March 2017. 3.1 Key Events 10. Key events during included: Delivery of DCC Live through R1.2 in the Central and South Regions on 8 November 2016 and in the North Region on 25 November 2016. This release provided the core functionality of the smart meter communications service for SMETS2 meters. R1.2 functionality allowed for SMETS2 meters to be installed for credit customers. Launching several new services to Industry, including: Communications Hubs ordering and delivery, pre-user Entry Process Testing (pre-uept), UEPT, End to End (E2E) Testing, and a number of forecasting services. Approval by the Secretary of State for Business, Energy, and Industrial Strategy (BEIS) for DCC s proposed delivery plan for R1.3. This release will be delivered in 2017/18. Following engagement with industry and analysing the options for providing a SMETS1 Service, DCC issued a feasibility report the Initial Enrolment Project Feasibility Report (IEPFR) for SMETS1 Meters for consultation in November 2016. The consultation period closed in January 2017. Achieving independent security assessments by the Competent Independent Organisation, and planned accreditations (ISO27001:2013 and tscheme, for the SMKI service). Providing briefings to industry representatives through two Industry Days, three Independent Suppliers Days, two Meter Operator forums and various design forums. Assessing a large volume of complex change to the design of the SMETS2 solution, with 72 contractual change requests and 33 SEC Modification Proposals for consideration. 3 https://www.smartenergycodecompany.co.uk/sec/sec-and-guidance-documents DCC Public Page 6 of 118

Three live releases of Parse and Correlate software to Industry to align with the latest versions of GBCS. DCC running 18 public consultations. DCC Public Page 7 of 118

4 DCC Service Delivery Performance 11. The purpose of this section is to report on various developments within the DCC Service offering in PY2016/17. The full range of DCC Services is set out in section 4.8. 12. During there was a significant uplift in the number of DCC Services (as defined by the SEC) being offered to customers, following DCC Live on 8 November 2016. 4.1 Operational Readiness 13. Operational readiness is the process which DCC followed to achieve the smooth migration of Releases from Programme to Live Operation. This process is set out in Figure 1 below. It ensures that all activities and artefacts required to support the live service are identified and completed. This included ensuring that: All testing, including Operational Acceptance Testing (OAT) and Business Acceptance Testing (BAT), had been completed All operational processes, user guides, training and work instructions had been identified, created, formally accepted and handed over in operations. Service Design Business Acceptance Testing (BAT) Transition Operational Acceptance Board (OAB) Live Figure 1 Transition to Live Operations 14. The full set of services available to customers is presented on the Self Service Interface (SSI). All forms, templates and guides are available on SharePoint for Parties who do not have access to the SSI. 4.2 Operating the DCC Service Desk 15. Based in Ruddington, Nottinghamshire, the Service Desk, provided by Capita ITES, is a vital component of DCC Service. There were no major issues identified with the service during. However, it is worth noting that the volume of activity in the early life phase has been very low as Users have been testing their integration with DCC and their end-to-end processes, with some awaiting the release of the prepayment functionality through Release 1.3. 16. The team of analysts are responsible for several key functions, notably: Logging and resolving SEC Party incidents, requests for support and general enquiries Operating as the primary interface to Users for all DCC live services Communicating to Parties on new services, updates and other information as required; and Logging nominated contacts and managing access to the Live Operations SharePoint Site, which is necessary to be able to make use of a number of DCC Services. DCC Public Page 8 of 118

17. If an incident cannot be resolved by the first line Service Desk, it is passed to the relevant DCC Service Provider the second line Service Desk. 18. The Service Desk comprised of 15 team members at the end of March 2016. As the year progressed, the team grew in preparation for DCC Live. On 12 September 2016, prior to DCC Live, 24 x 7 hour working was implemented. To match demand and to manage the live service, there were 32 Service Desk staff at the end of March 2017. 19. In advance of an enduring DCC Service Management System (DSMS) being deployed, an interim system of Incident and Request ticketing and email connectivity was used by the Service Desk. This platform during provided 99.99% system availability. The Enduring DSMS solution went live on 8 November 2016. The Interim Solution was left in place as a back-up, and was withdrawn and decommissioned on 31 December 2016. 20. Service Desk metrics from 1 April 2016 to 31 October 2016: Metric emails Calls Amount c. 1,100 (per week) 1,229 (for the period) 21. Service Desk metrics from DCC Live to March 31 2017: Metric emails Calls Amount c. 1,500 (per week) 1,442 (for the period) 4.3 Maintenance Windows 22. As agreed by the SEC Panel on 14 October 2016, planned maintenance windows for the Live service are available for use by DCC every Tuesday and Friday night time, 00:01 06:00. From DCC Live up to 31 March 2017, 12 planned maintenance windows were used. 4.4 Engagement 23. DCC continued its commitment to being open and transparent and sharing relevant information with its External Service Providers, BEIS, Ofgem, the SEC Panel and industry stakeholders. Industry engagement has included: DCC Industry Days two events were held during with a total of 258 participants. Feedback on the events showed that 78% of attendees rated the events as good / very good / excellent In addition to the Industry Days held in the prior year the need was identified to offer events targeted at independent suppliers. DCC ran three events, (two in London and one in Leicester), as well as two Meter Operator Forums DCC Forums 46 forums were held during PY 201617, topics included: Service Management Design, Design Release, Communications Hub & SM WAN and DCC Public Page 9 of 118

Testing. These events typically lasted half a day with the average of 20 30 attendees Quarterly finance webinars to explain movements in the charging statements Participation in transitional and enduring SEC committees and sub-committees Consultation briefings and contact with all stakeholders disagreeing with or having questions on the proposals set out in consultation drafts Frequent and structured reporting to Parties on testing activities, including progress through Systems Integration Testing and end stage reports On-going customer engagement through DCC Industry Partnership Managers, who directly engaged (via face to face meetings) with 56 individual SEC Party organisations during. 4.5 SMKI Developments during 24. The Smart Metering Key Infrastructure (SMKI) is one of the primary mechanisms to secure communications between Users and smart metering devices such as smart meters and communications hubs. SMKI Certificates used are to sign messages between SEC parties and devices. They are also used to sign DUIS commands sent to the Data Service Provider. SMKI service went live on 9 March 2016 so during this PY the efforts were concentrated on maintaining and expanding this service. 25. Key developments during the year: The SMKI Service has been certified by an independent auditor (LRQA) under tscheme standard at stages 0, 1 and 2 during the year, followed by routine surveillance audits. The DCC have worked closely with the SMKI Policy Management Authority (a SEC Panel Sub-Committee) throughout the year to ensure that the SMKI Service and systems are technically and operationally secure in line with risk assessments and specifications. This has included working with the National Cyber Security Centre and the BEIS Security team. 4.6 DCC s Assurance of Smart Metering Wireless Access Network (SM WAN) Coverage by the CSPs 26. In accordance with DCC s SM WAN Assurance Strategy, DCC was required to assure that: the required connectivity to 80% of GB premises was ready for DCC Live; and this connectivity will reach over 99% of GB premises by the end of 2020. 27. DCC s approach to assuring that the CSPs meet their connectivity and coverage service commitments was detailed in its SM WAN DCC Assurance Strategy 4 of March 2016. In accordance with this strategy, the focus in was to mobilise resources, equipment and arrangements to enable field based testing (planning for the gaining and 4 https://www.smartdcc.co.uk/media/355368/dcc_sm-wan_assurance_approach_march_2016_v1.0_2016-03-07_final.pdf DCC Public Page 10 of 118

procurement of equipment and competencies required was undertaken in PY 2015/16) to become a complimentary part of the signoff process for the CSP milestone. 28. In addition, external audits of the CSPs testing arrangements were set up which allowed DCC to study these data sets in confidence without incurring the unnecessary expense of fully repeating such activities. This enabled DCC in RY 2016/17 DCC to fully complete its transition from the initial document-based assurance regime to include rich empirical-based SMWAN delivery evidence. In recognition of the use of Mesh technology at scale for the first time in UK Smart Metering: DCC assured and signed off the transition from 80% to 88% for CSP(N) and from 80% to 97.75% for CSP(S&C). The detail of the CSP SM WAN coverage milestones are available in the DCC Statement of Service Exemptions 5 which also shows the achievement and signoff dates; and DCC arranged an open Industry event on 22 September 2016 at the Buildings Research Establishment to demonstrate the implementation by CSP(S&C) of a proven Mesh technology in a variety of building types. 5 https://www.smartdcc.co.uk/media/430075/dcc_statement_of_service_exemptions_mar_2017_v1.0.pdf DCC Public Page 11 of 118

4.7 SEC Code Performance Measures 29. The Smart Energy Code (SEC) sets out the operational Service Levels (performance targets) which DCC needs to meet when providing Services to Users. This section provides a summary of the Service Levels achieved in respect of the Code Performance Measures set out in Section H13.1 and L8.6 of the SEC. The Service Levels achieved are calculated in accordance with the DCC Performance Measurement Methodology 6. Pursuant to Section H13.4 of the SEC, a monthly report providing greater detail is shared with SEC Parties, SEC Panel, Ofgem and the Secretary of State. 30. As DCC is now operating a live service, it reports performance against SEC Performance Measures as part of the DCC Service Delivery Performance. The Performance Measures are the focus for DCC providing service in line with customer expectation. These measures are reported monthly to all stakeholders. A Target Service Level and Minimum Service Level apply to each of the Code Performance Measures. Table 1 below summarises DCC s performance against the SEC Code Performance Measures. 31. DCC is required to report on each Performance Measure from the point at which the relevant DCC Service becomes live: From April to October 2016, only Code Performance Measure 7 was reported following go live of the SMKI Certificates Service. The Service Levels achieved has been above the Target Service Level each month. DCC go live was in November 2016 this activated reporting against Code Performance Measures 1 6. There is a Secretary of State Variation to the SEC in place which has deferred the reporting of Code Performance Measure 8 until DCC Release 1.3. 32. The Service Levels achieved for Incident Resolution (Code Performance Measures 4 and 5) were below the Target Levels in the initial period following DCC Live. Generally the Incidents recorded have been low level technical or access issues for production services, none prevented Testing Participants progressing through Testing. Remedial actions have been taken and have resulted in a gradual improvement of the Service Levels achieved in 2017. Table 1 Key Green Amber Red Met Target Service Level Met Minimum Service Level Did not meet Minimum Service Level 6 Pursuant to Section H13.6 of the SEC the DCC Performance Measurement Methodology documents the methodology for calculating performance against each Code Performance Measure. DCC Public Page 12 of 118

SEC Ref. 1 2 3 4 SEC Code Performance Measure Percentage of On-Demand Service Responses delivered within the applicable Target Response Time. Percentage of Future-Dated Service Responses delivered within the applicable Target Response Time. Percentage of Alerts delivered within the applicable Target Response Time. Percentage of Incidents which the DCC is responsible for resolving and which fall within Incident Category 1 or 2 that are resolved in accordance with the Incident Management Policy within the Target Resolution Time. Target Service Level Minimum Service Level Service Level November 2016 December 2016 January 2017 February 2017 March 2017 99% 96% No data 7 No data No data No data No data 99% 96% No data No data No data No data No data 99% 96% No data No data No data No data No data 100% 85% 67% 100% 50% 100% 100% 7 There is no data for Code Performance Measures 1, 2 and 3 because no Communications Hubs have been installed by Users and therefore there is no Service Request activity to report. DCC Public Page 13 of 118

SEC Ref. 5 6 7 8 SEC Code Performance Measure Percentage of Incidents which the DCC is responsible for resolving and which fall within Incident Category 3, 4 or 5 that are resolved in accordance with the Incident Management Policy within the Target Resolution Time. Percentage of time (in minutes) when the Self-Service Interface is available to be accessed by all Users during the Target Availability Period. Percentage of Certificates delivered within the applicable Target Response Time for the SMKI Services. Percentage of documents stored on the SMKI Repository delivered within the applicable Target Response Time for the SMKI Repository Service. Target Service Level Minimum Service Level Service Level November 2016 December 2016 January 2017 February 2017 March 2017 90% 80% 61% 64% 81% 86% 94% 99.5% 98% 100% 100% 100% 100% 100% 99% 96% 100% 100% 100% 100% 100% 99% 96% N/A N/A N/A N/A N/A SEC Code Performance Measures DCC Public Page 14 of 118

4.8 Performance of Services Provided to SEC Parties 33. The following services were in operation during. The services are grouped appropriately. Where performance of a service is described as satisfactory this means it has met or exceeded DCC expectations and delivered a professional service within agreed timescales and quality standards. Where the service had improvement required, issues arose during the year, but remediation plans or other required actions were agreed and delivered. Where the service was unsatisfactory, issues that required action were identified but not addressed. DCC Gateway Connections Service DCC Gateway Connections Gateway Connection Configurations Provider of Service DSP DSP Description of Service Performance for The installation of these Connections is a three stage process, with the first two stages being covered by this service and the third stage covered the Gateway Connection Configurations service (described below): 1) Laying of the physical cabling 2) Installation of the equipment at the end of the connection 3) Configuration of the connection to connect to the required services. SEC reference: sections H15 and E3. Part of the DCC Gateway Connections process, Parties and RDPs are able to use this service to request configuration of their installed DCC Gateway Connections. Once the physical connection has been supplied using the DCC Gateway Connection process, the gateway connections then need to be configured so that they recognise Users confidential URL and IP DCC Public Page 15 of 118 Satisfactory performance. The service was provided to the expected timescales and quality. The installation issues faced in 2015/16 were overcome during 2016/17. Volume of installations during the for this service include: 8 High Volume Connections and 4 Low Volume Connections Cumulative volume of installations up to 31 March 2017 include: 44 High Volume Connections and 16 Low Volume Connections Satisfactory performance. Delivered according to published timescales. Service Desk received 512 requests to configure DCC Gateway Connections during 2016/17. Date Available to Parties Jan 2015 Aug 2015

Service Provider of Service Description of Service Performance for address data and enable communication to DCC. Configuration is a remote activity carried out by the DSP. Multiple configurations will be required for each connection, for the various environments, services and DCC User IDs. SEC reference: sections H15 and E3. Date Available to Parties Gateway Connection Performance Reports DSP/DCC The Gateway Connection Performance Reports service is within the suite of DCC Gateway Connection Services. The service allows Parties to opt for DCC to provide a monthly report on the performance of each of its Gateway Connections. SEC reference: section H15. Satisfactory performance. Gateway performance reports were issued to Parties each month during once the service was live. The Reports were published within expected timeframes. Feb 2016 Communications Hubs Forecasting and Ordering Services Service Provider of Service Description of Service Performance for Date Available to Parties Communications Hubs Forecasting CSPs DCC is required to combine all forecasts into a single prediction for each External Service Provider for each month and each Region. Parties will then refine these forecasts to improve accuracy on a month by month basis. Subsequent orders must be within +/-50% of the volume forecasted 10 months before the delivery month and +/-20% of the volume forecasted 7 months before the delivery month. Satisfactory performance. Both CSPs have had workarounds in place for OMS as the process of advanced shipment notices are unable to be completed via OMS and are done manually, delivered to Parties through SharePoint. 25 Parties forecasted during. Feb/Apr 2016 DCC Public Page 16 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties The Order Management System (OMS) developed by Telefonica for the Central and South Regions went live in February 2016, with the OMS for the North region provided by Arqiva going live in April 2016. All Communications Hub forecasts are now submitted via OMS for each CSP. In order to allow forecasting to take place a full 10 months ahead of the Initial Delivery Month, SEC Transitional Arrangements allowed forecasts to be submitted via DCC s SharePoint service as the enduring solution was still in development at the time forecasts were required. SEC reference: section F5. Communications Hubs Ordering CSPs This Service allows Parties to submit Communications Hubs orders to the DCC. It encompasses processes for the submission, receipt, validation, analysis, consolidation, aggregation and forwarding to the respective External Service Providers of monthly orders for Communications Hubs. All Communications Hubs Orders are now submitted via OMS for each CSP. SEC reference: sections F5. Satisfactory performance. The service was available for Parties to order. 18 Parties ordered during. Due to the delays to R1.2 and R1.3 all 18 Users cancelled some Communications Hubs orders as they reforecast their roll out timescales. Apr 2016 DCC Public Page 17 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Communications Hubs Recall CSPs/DCC If DCC suspects a fault in Communications Hubs delivered to Parties, it can request to recall them. Communications Hubs must then be returned by Parties to DCC. SEC reference: F8. Satisfactory performance. The process ran efficiently and all Communications Hubs that required a recall were recalled. 896 Live Telefonica Toshiba Communications Hubs were recalled for two (2) Parties and 338 Test Communications Hubs were recalled for 16 Parties. See section 7.6.5 for recall explanation. Nov 2016 Communications Hub Delivery and Rejection CSPs A Communications Hubs delivery may be rejected in whole, or part, by the recipient. Parties inform DCC of the reason for any rejection via the OMS. SEC reference: F8. Satisfactory performance. Process of delivery has been smooth but Parties have rejected Communications Hubs because the versions of firmware installed are not the versions required for Live operation. Workarounds have been required for process of rejections via OMS. Sep 2016 Communications Hubs In-Life Returns DCC Once installed and commissioned, or if a fault is identified as part of installation, Parties can return Communications Hubs to DCC. This process is managed through SharePoint. SEC reference: F8. Service available but not used as no Communications Hubs were attempted to be installed by Parties in the PY. Sep 2016 DCC Public Page 18 of 118

Quarterly Forecasting Service Provider of Service Description of Service Performance for Date Available to Parties SMKI Certificate Signing Request (CSR) & Repository Forecasting DCC SMKI CSR & Repository forecasts are currently being submitted by SMKI Users and prospective SMKI Users. These are completed on an Excel spreadsheet and manually uploaded to SharePoint. DCC creates a consolidation file of all forecasts received. Satisfactory performance. The process was available for Parties to forecast. At the end of there were 19 Parties forecasting. Apr 2016 Service Request Forecasting DCC Service Request forecasts are currently being submitted by Users and prospective Users. These are completed on an Excel spreadsheet and manually uploaded to SharePoint. DCC creates a consolidation file of all forecasts received. Satisfactory performance. The process was available for Parties to forecast. At the end of there were 28 Parties forecasting. Apr 2016 Smart Meter/ Communications Hubs/Service Requests Forecasting DCC Smart Meter forecasts are currently being submitted by Parties. These are completed on an Excel spreadsheet and manually uploaded to SharePoint. DCC creates a consolidation file of all forecasts received. Satisfactory performance. The process was available for Parties to forecast. At the end of there were 12 Parties forecasting. Apr 2016 DCC Public Page 19 of 118

Testing Services Service Provider of Service Description of Service Performance for Date Available to Parties GBCS Interface Testing for Industry (GFI) DCC/ Critical GBCS Interface Testing for Industry (GFI) is a software tool, developed by Critical Software, to share DCC s interpretation of GBCS for SMETS 2 Devices with SEC and non-sec Parties. The tool allows Test Participants to emulate the sending and receipt of GBCS messages and help identify any potential differences with Test Participant s interpretation of the specifications. Satisfactory performance. There continues to be take-up from device manufacturers wishing to test their interpretation and implementation of GBCS. Other users of the service include software houses wishing to test their software and energy suppliers testing the meters they plan to install. Jun 2015 SEC reference: sections H14 and X3. SMKI and Repository Entry Process Testing (SREPT) TSP/DSP Demonstrate a Party s systems, personnel and processes ability to successfully use the SMKI and SMKI Repository interfaces according to the specifications set out in the SMKI Repository Interface Design Specification and the SMKI Interface Design Specification. Parties are issued with SMKI Certificates for use in Live Operations. Satisfactory performance. Service delivered efficiently. Nov 2015 SEC reference: sections H14. Early User Connectivity Testing DSP Validates a Party's connection to DCC's User Integration Testing (UIT) environment. SEC reference: sections H14. Satisfactory performance. Efficient service, with any configuration issues easily rectified. Apr 2016 DCC Public Page 20 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Pre-User Entry Process Testing (Pre-UEPT) DSP Enable Parties to test implementation of their User Systems and improve confidence in completing UEPT successfully. This service is voluntary. SEC reference: section X9. Satisfactory performance. Satisfactory support provided. There has been decreasing demand for this service since UEPT and E2E became available. Jun 2016 User Entry Process Testing (UEPT) DSP/ CSPs To demonstrate a Party's ability to communicate with meters across a number of roles (by means of executing Service Requests) and accessing the Self Service Interface (SSI) via DCC. This service is mandatory to become a User. SEC reference: section H14. Unsatisfactory performance. The start date of this service was delayed due to a number of factors experienced during SIT execution stage. Once testing commenced, Participants experienced a number of issues with the Install and Commission Process. Sep 2016 End to End Testing DSP/ CSPs Opportunity for Parties to conduct Device and User System Tests against the DCC i.e. to test the interoperability of meters and back office systems against the DCC. Parties bring their own devices to test. Testing is undertaken either in DCC Test Labs or via Remote Test Laboratory function. This service is a voluntary option. Unsatisfactory performance. Same as UEPT. Sep 2016 SEC reference: section H14. Ordering of Remote Test Lab facilities DCC/ CSPs Parties, including Energy Suppliers and other Parties such as Device Manufacturers and Test Houses can undertake Device and User System testing on a voluntary basis. This testing may be carried out in an External Service Provider Test Lab or by using a remote testing service. Improvement required. There were delays in getting Remote Test Service (RTS) active. The Wide Area Network (WAN) and Home Area Network (HAN) faced initial instability, with new firmware versions improving the issues. Sep 2016 DCC Public Page 21 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties The remote testing option enables a Testing Participant to install its own devices and Test Communications Hubs into its own test lab (at a location of its own choice). This service enables a Testing Participant to request a Remote Test Lab connection and will include a quotation for the Remote Test Lab set up and the monthly support charges applicable. Testing Participants can order Test Communications Hubs to support Device and User System testing in the End-to- End Testing stage. Over the Air Firmware upgrades have not worked consistently for CSP (C&S). The Service Desk received 14 applications for Remote Test Lab facilities during the PY 2016/17. The Service Desk received 805 applications for Test Communications Hubs during the PY 2016/17. SEC reference: sections F5 and H14. Test Reporting DSP The DSP SI team consolidates the reporting on testing services provided by the DSP and CSPs. The reports are made available to Parties twice a week. SEC reference: section H14. Improvement required. The DSP did not provide reports to assist Parties in the early stages. The reports are now being produced satisfactorily. Jan 2017 Smart Metering Key Infrastructure (SMKI) Services Service SMKI Registration Provider of Service DCC Description of Service Performance for In order for any Party or RDP to use the SMKI Service they will need to register their organisation and named Responsible Officers. Once their registration is verified the Satisfactory performance. The registration system performed in line with the Service Level Agreement (SLA). Date Available to Parties Sep 2015 DCC Public Page 22 of 118

Service SMKI Certificate Signing Requests Provider of Service Trusted Service Provider (TSP) Description of Service Performance for Authorised Responsible Officers (AROs) will receive credentials to authenticate to the SMKI Interfaces. This service also enables Parties to register test SMKI (AROs) and gain credentials to access the test interfaces. SMKI Credentials are required by Parties as a pre-requisite to the commencement of SMKI Repository Entry Process Testing (SREPT). The SMKI Registration service will also provide access to the SMKI Repository interfaces using a combination of username and password for the Portal and SFTP interfaces and API Key for the web service interface. SEC reference: 5.7 Appendix D. SMKI Certificates secure messages between Parties and Devices by providing assurance of the identity of each entity. They are also used to sign DCC User Interface Specification (DUIS) commands sent to the DCC. A Certificate Signing Request (CSR) is a message sent from an applicant to a Certificate Authority in order to apply for a unique digital identity certificate. For the SMKI Service, CSRs will be sent to the DCC s Trusted Service Provider (BT) to apply for unique digital identities for Organisations, and Devices. It is possible for Authorised Subscribers to submit Batched Device CSRs for up to 50,000 individual Devices at once. Organisation CSRs can only be sent to the DCC via the SMKI Portal, however, Device CSRs can be sent via the SMKI Portal or the Web Service Interface. Once CSRs have been validated by the DCC SMKI Registration Authority team, the Organisation Certificate Authority or Device Certificate Authority will generate and DCC Public Page 23 of 118 At 31 March 2017, 48 Parties had been supported through SREPT and received test completion certificates. 26 SEC Parties had obtained organisation certificates in production and 6 SEC parties have obtained production Device Certificates. 420 individuals have been verified by the SMKI Registration Authority Managers. Satisfactory performance. However, as with SMKI registration, Parties experiences were impacted by the SMKI Repository not always being available when Parties required. As at 31 March 2017 the number of valid Organisation Certificates was 161. The total number of Device Certificates issued was 1,087,362. The total number of valid Infrastructure Key Infrastructure (IKI) certificates issued was 485. Date Available to Parties Nov 2015 to support Testing Services/ Mar 2016 for Live services

Service Provider of Service Description of Service Performance for log the relevant corresponding certificates in the SMKI Repository, and make them available for download on the SMKI Portal. Date Available to Parties This service is also used to enable Parties to gain SMKI Certificates that could be used for testing purposes. SMKI Repository, Management and Interim Recovery Services DSP SEC reference: section L11. The SMKI Repository, Management and Interim Recovery Services cover the processes around the mainstream SMKI activities that are required to manage the performance and delivery of the core SMKI services of Registration and the Production of SMKI Certificates. This includes the Interim Recovery measures that are required during the initial stage of Live SMKI, before the DCC Service was in live operation. SEC reference: section L6 Improvement required. In the first two months after go live the SMKI repository service performed below the SLA. This was due to issues between the SMKI service and the SMKI Repository service resulting in certificates not being published due to timeout issues. This issue was resolved with the assistance of the two CSPs, who were early adopters and the only users at the time. Since this time the service has performed in line with the SLA. Mar 2016 Other Services Service Provider of Service Description of Service Performance for Date Available to Parties Reporting DCC DCC is required under the SEC to produce a number of regulatory reports. DCC has implemented new business processes and tools to ensure the reports are produced to Satisfactory performance. Reports were produced as required. Various dates DCC Public Page 24 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Performance Standards and Reporting Smart Metering Wireless Access Network (SM WAN) DCC CSPs the standard and timescales required i.e. DCC introduced a catalogue to manage all regulatory reports and a catalogue reference to enable DCC, its Service providers and its customers to uniquely identify a report. DCC is required to produce a monthly Performance Measurement report. The report is shared with SEC Parties, SEC Panel, Ofgem and the Secretary of State. It provides details of the Service Levels achieved by DCC in respect of the Code Performance Measures and Service Provider Performance Measures set out in the SEC. SEC reference: H13. The CSPs maintained coverage information regarding availability of the Services at the Communications Hub Installation Point in Consumer Premises in each postcode in their respective Regions. This information can be used by Energy Suppliers carrying out strategic planning activities regarding the roll out of Smart Metering Systems and their preparation for forecasting their Communications Hub orders. DCC Public Page 25 of 118 Satisfactory performance. In accordance with Section H13 of the SEC monthly Performance Measurement reports were published 25 Working Days following the completion of the measurement period (calendar month). Reports for the November 2016 and March 2017 measurement periods were issued by the 27th Working Day. Additional time was taken to undertake final assurance of the Service Levels being reported. In response to stakeholder feedback greater detail is now provided in the report. It is hoped that the increased transparency has provided assurance to stakeholders that any issues with Services are being managed effectively. Satisfactory performance. SM WAN Coverage information was made available to Parties throughout 2016/17. through PY 2016/17 Feb 2016 Aug 2015 (Interim) Nov 2016 (SSI)

Service Provider of Service Description of Service Performance for Date Available to Parties Coverage Service The interim SM WAN Coverage files support upload of coverage data from the External Service Providers, receipt and verification of that data, and the publication of that data on the DCC SharePoint system where it can be accessed by Parties. Since DCC Live (R1.2) this information is now available to Users via the Self Service Interface (SSI), Web Graphical User Interface (GUI) or via web transaction (DUIS) as a query to the Coverage Checker Database (CCDB). The interim solution has been extended indefinitely in order that those SEC Parties who are not yet Users (and therefore cannot yet access the SSI nor send DUIS commands to query the CCDB) can still avail themselves of this coverage information. CSPs met all SM WAN delivery milestones on time, as documented formally in the DCC Statement of Service Exemptions. 8 DCCKI Registration Service DSP SEC reference: sections H8 and X3. The DCC Key Infrastructure (DCCKI) is the means by which communications between Users and DCC over the DCC User Interface Specification (DUIS), Self Service Interface (SSI) and Registration Data Interface (REGI) interfaces are secured. DCCKI Registration Service provides formal arrangements under which a Party or RDP registers their Organisation via a Nominating Officer, nominate a DCCKI Senior Responsible Officer (SRO) and a DCCKI Authorised Satisfactory performance. Initial issues with DCCKI registration forms have now been rectified with the introduction of a simpler form. In March 2017 for Live services there were 60 active SROs and 84 Active AROs. In March 2017 for Testing Services there were 130 active SROs and 230 active AROs. Mar 2016 8 https://www.smartdcc.co.uk/media/430075/dcc_statement_of_service_exemptions_mar_2017_v1.0.pdf DCC Public Page 26 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Responsible Officer (ARO). The service allows a Party or RDP to become a DCCKI Authorised Subscriber. The DCC and its External Service Providers will also need to go through the DCCKI Registration process. SEC reference: section L13. DCCKI Certificate Signing Request (CSR) DSP DCCKI ARO s can send DCCKI Certificate Signing Requests to the DCCKI Registration Authority via secured electronic means to authenticate and secure access to DCC interfaces. There are two types of certificates available to Parties through DCCKI: Satisfactory performance. There were initial server issues which were fixed. For more detail see CGI Section 7.4.5. Jul 2016 Infrastructure Certificates used to establish secure channels to DCC Interfaces e.g. Registration Data Interface, DCC User Interface, etc. User Personnel Certificates obtained via an automated interface to authenticate SEC Party personnel to the Self-Service Interface. SEC Reference: L13. Parse and Correlate Service Critical All critical Service Requests that are sent across the DCC User Gateway will be converted from xml into GBCS format. The Parse and Correlate software is software produced by Critical Software that DCC is making available to interested Parties to carry out checking that the xml/gbcs conversion has not changed the underlying request. Satisfactory performance. New versions have been successfully released on time with identified defects resolved. The live software continues to support Systems Integration Testing (SIT) and respond to compatibility issues raised by DCC and Service Provider's. May 2015 DCC Public Page 27 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Parties are not obliged to use the Parse and Correlate software; instead they can develop the functionality to do this themselves. The software is issued on a No Liability basis and is available to both SEC and non SEC Parties. There have been 3 new live versions of the software released during. SEC reference: Section H11. Incident Management Service Desk/DSP The Incident Management service manages the logging of incidents and service requests from nominated contacts of Parties and the resolution of these incidents and requests. The process also covers the logging and management of contact queries from the wider SEC Party community. To support this process, the SEC requires the DCC to provide a live Service Desk and to publish its contact details on the DCC website. The enduring Incident Management services went live on 8 November 2016 in the Central and South Regions and 25 November 2016 in the North Region. The enduring solution is delivered as part of the wider DCC Service Management System (DSMS) made up of the BMC Remedy Application, the Self-Service Management Interface (SSMI) and the Self Service Interface (SSI). See section 4.2 for more information on the Service Desk. SEC reference: section H9. Improvement required. Service Desk Between 1 April 2016 and 11 September 2016, the Service Desk operated from 08:00 to 18:00 hrs Monday to Friday. On 12 September 2016, a 24/7 service was implemented. The Service Desk was 99.99% available, in line with DSMS availability. DSMS At the end of there were on-going performance issues with the DSMS. For further detail see section 7.4.5. The DSMS incident management system was 99.99% available. Jan 2015 Interim Nov 2016 Enduring DCC Public Page 28 of 118

Service Provider of Service Description of Service Performance for Date Available to Parties Nominated Contact Management Smart DCC Parties are required to provide a list of nominated contacts whom DCC will recognise as being authorised to raise Incidents and service requests on their Party s behalf. This service covers the collection and storage of Party nominated contacts into a spreadsheet that can be used for validation in other services, including the Incident Management Process. SEC reference: section H9. Satisfactory performance. Systems available throughout the year. Since January 2015, up to 31 March 2017, 165 SEC Parties provided details of 1,011 individual nominated contacts. DCC has confirmed email connectivity with each nominee. Jan 2015 DCC Public Page 29 of 118

5 DCC Programme Performance 34. DCC currently runs three parallel programmes. This section summarises: the progress on SMETS2 that DCC achieved during the year, including DCC Live, R1.3 and future releases; SMETS1 progress (also known as Enrolment and Adoption); and DCC s work on Switching and the Centralised Registration Service. 5.1 SMETS2 Programme 5.1.1 Overall Performance on SMETS2 Programme 35. In November 2016 DCC launched the data and communications infrastructure to enable the nationwide rollout of smart meters. This is a major milestone that delivers a first-of-a-kind, coherent and highly secure national communication infrastructure, allowing energy suppliers to install meters in homes and small businesses across Great Britain. 36. During, DCC focused on delivering its core SMETS2 solution. Whilst significant progress was made by both DCC and its Service Providers, overall performance in PY 2016/17 needs to be considered in the context of delays to delivery of both R1.2 and R1.3 from their originally directed dates. There were a number of related issues but overall causes of delays to delivery can be summarised as: The number and complexity of defects identified and resolved during UEPT and SIT for R1.2. This moved DCC Live from July 2016 to November 2016; The subsequent knock on effect on R1.3, resulting on this release being replanned from September 2016 to April 2017 9 ; and Complexity of the solution required late addition of test scope to provide additional assurance. 5.1.2 Release 1.2 DCC Live 37. In order to deliver the core communication infrastructure DCC has integrated numerous structural components into a complex and secure system. In so doing, DCC has managed an extensive supply chain to resolve delivery challenges. 38. DCC achieved live operational status in the Central and South Regions on 8 November 2016 and in the North Region on 25 November 2016, through the delivery of R1.2. 39. The scope of R1.2 was designed to deliver the functionality required by any DCC User to: (i) (ii) perform User Entry Process Testing (UEPT) in line with the Smart Metering Design Group s 'must have' business priorities; and commence the deployment of gas and electricity SMETS2 smart meters for credit customers. 9 R1.3 went live on 20 July 2017. DCC Public Page 30 of 118

40. Functionality was defined as 63 Service Requests / 84 GBCS Use Cases. This marked the point at which DCC was operationally ready and able to manage messages to and from meters on our production platform. Revisions to the Original Delivery Plan 41. During PY 2015/16 DCC, working with its Service Providers, developed a delivery plan, which was approved by the Secretary of State (SoS) on 18 December 2015, setting out a path to go live on the 20 July 2016, with SoS holding one month of contingency. This plan, with the dates the milestones achieved, is included in Table 2 below. Milestone Name Noncontingent plan R1.2 Plan with contingency applied Date Milestone Achieved C/S Region Date milestone achieved N Region Pre-UIT commences (Environment Available) 18 April 2016 16 May 2016 15 April 2016 15 April 2016 SIT Solution Test Commences SIT Solution Test Complete Interface Testing Commence 6 May 2016 3 June 2016 6 May 2016 6 May 2016 3 June 2016 1 July 2016 23 October 2016 SIT Exit for Go Live 4 November 2016 SIT Stream 4 and overall SIT Complete 4 November 2016 6 June 2016 4 July 2016 11 August 2016 11 August 2016 UEPT Commence 13 June 2016 11 July 2016 25 August 2016 25 August 2016 E2E Test Environments available 4 July 2016 1 August 2016 28 September 2016 28 September 2016 DCC Public Page 31 of 118

Milestone Name Noncontingent plan R1.2 Plan with contingency applied Date Milestone Achieved C/S Region Date milestone achieved N Region Service Desk ready to support live installations 10 6 July 2016 3 August 2016 22 October 2016 22 October 2016 Production Communications Hubs available 20 July 2016 17 August 2016 8 November 2016 25 November 2016 Interface Testing complete 20 July 2016 17 August 2016 4 November 2016 17 November 2016 DCC Live 20 July 2016 17 August 2016 8 November 2016 25 November 2016 18 December 2015 approved Department of Energy and Climate Change (DECC) plan with dates achieved Revisions to the Original Plan 42. The date planned at the beginning of was not met, with final go live being moved to November 2016. The timeline of movements in planned dates, as shared with the Parties and the SEC Panel, is included in Table 3 below: Date Event Original Live Date Revised Live Date 20 May 2016 SIT Progress Review Control Point 20 July 2016 20 July 2016 23 May 2016 Call down of Contingency for R1.2 20 July 2016 17 August 2016 21 July 2016 Revised Plan Proposed by DCC 17 August 2016 30 September 2016 12 Aug 2016 Acceptance of replan proposal by DECC SRO 10 renamed as DCC ready to support Live Installations on 7 June DCC Public Page 32 of 118

21 September Proposal for 4 SIT Streams 11. Delays to SIT identified 30 September 2016 31 October 2016 Timeline of R1.2 go live date movements Delay Factors and Mitigations Undertaken 43. There were a number of factors that contributed to the delay in DCC Live. DCC strived to mitigate these, where possible. The following section covers the main factors and mitigations. Late PIT Complete 44. FACTOR: By late March 2016 a bottom up planning assessment had identified that the four-week SIT Solution Testing window in the plan for completion by 6 May could not be delivered. During April, it became clear that there would need to be multiple drops of code from PIT into SIT to maintain the SIT start date as 6 May. DCC was able to meet its SIT entry milestone with a reduced scope. The late delivery of sub-system components by Service Providers into SIT inevitably delayed the point at which integration defects were found. 45. MITIGATION: DCC allowed a staggered entry of functionality into SIT (i.e. multiple drops of code). This allowed SIT Solution Testing to commence on the planned date. Subsequent drops of code were introduced with associated regression testing. Level of Complexity 46. FACTOR: Release 1.2 was a new-build solution and hence not previously integrated. The level of complexity, especially regarding security requirements, led to a larger number of defects than had been anticipated. There were a number of specification defects / ambiguities / inter-operability issues which delayed test execution. These issues in some cases prevented testing, or in other cases caused HAN reliability issues while trying to commence testing. 47. The security model is complex and is made more complicated by elements of the solution having different non-functional capabilities for the same system function (e.g. Communications Hubs) in different Regions. 48. MITIGATION: DCC directed prioritisation and the deployment of DCC technical expertise, and the introduction of six day working. DCC agreed with BEIS and Industry (via SMDG) a subset of R1.2 functionality ( SIT Stream 4 ) that was not required at go live. Full set of Streams were as follows: Stream Roles within Scope SIT Complete forecast Pre / Post Go Live 1 Import Supplier and Gas Supplier roles (required for UEPT to enable IT Exit) 17 October Pre 11 SIT originally ran as a single Stream. DCC Public Page 33 of 118

2 Electricity Supplier, Gas Transporter and Electricity Distributor roles (to enable UEPT) 13 October Pre 3 Core functionality (TADP, CPL, DSMS, SSI, SSMI, ESI) 7 October Pre 4a Registered Supplier Agent and Other User roles 11 November (post go live) Post 4b Telefonica Mesh / Mesh with extended antenna Comms Hubs 11 November (post go live) Post Streams within SIT Speed of Defect Remediation 49. FACTOR: It took time for DCC to learn how to categorise and resolve defects more quickly as the release progressed. 50. MITIGATION: Once common defect themes had emerged, DCC-led taskforces were introduced to identify root cause and prioritise triaging and remediation activities. Additional resource was brought in to support the collaboration between DCC and External Service Providers to ensure faster progress of defect remediation in SIT. Meter Availability for SIT 51. FACTOR: DCC had a dependency on the availability of suitable meters for SIT, however the meters selected through the Device Selection Methodology were not fit for purpose for SIT. In addition, the meter manufacturers were constrained in their ability to uplift firmware versions quickly enough. 52. MITIGATION: SIT needed to be undertaken on a mix of emulators and meters. This however added to the integration burden in a fixed capacity environment. 53. Following the delivery of DCC Live, DCC completed a lessons learnt review to update management of future releases. DCC Delivery Approach 54. This section outlines the processes DCC undertook to assure the quality of the R1.2 delivery. Gating Approach 55. Quality Gates and Programme Gates, with defined entry and/or exit criteria, were used to ensure quality criteria were met before proceeding into the next stage or phase of the release. DCC created a 9 gate process from initiation of change to operational acceptance. Quality Gates DCC Public Page 34 of 118

56. The requirement for Quality Gates is set out in the Joint Test Strategy. 12 Quality Gate Reviews were held between Test Stages to confirm that the Exit Criteria of the preceding Test Stage and the Entry Criteria of the upcoming Test Stage had been met (see Section 9.4 of the Joint Test Strategy for details of the Exit and Entry Criteria). Programme Gates and Assurance Points 57. Release R1.2 was delivered using gates governing the entry by Service Providers to the next stage or phase of the Release. This started at the Design Complete gate and progressed through to Go Live. As part of the gating, artefact lists were defined for each gate and evidence of completion logged. To pass through Programme Gate required Programme Director approval. To pass through an Assurance Point required Release Manager approval. External Go Live Governance 58. DCC s own assessment of Readiness to go live was subject to internal scrutiny by the DCC Executive Board and the DCC Board. 59. In addition, DCC submitted the documentation to SEC Panel and Secretary of State provided assurance that the DCC Live Services Criteria were satisfied for the following: 1. Enrolment and Communication Services Ready 2. Incident and Problem Management Capabilities Ready 3. Service Management, Self Service Interface and Service Desk Ready 4. Business Continuity Arrangements Ready 5. Parse and Correlate Software Available 6. Installation, maintenance and removal of Communications Hubs ready 7. Pre-Existing Services Remain Stable. 60. This report was also provided to Parties. Systems Integration 61. CGI is the Systems Integrator (SI), with responsibility for orchestrating the Service Provider elements of the technology solution within the integration environment and ensuring that the systems integrate effectively. For R1.2, DCC was required to provide extensive assistance to the DSP in order to ensure effective SI activity. The performance of the DSP SI Service is covered in section 7.4.2. 62. Slippage to the R1.2 go live in relation to the Systems Integration was due to: Incremental availability of system components into integration; Integration complexity of the overall solution; Instability in test environments; Lack of end to end design assurance; and Time to fix integration defects. 63. Lessons have been identified from R1.2 and applied to R1.3. Due to the inherent restrictions on the DSP being able to operate as a successful SI, DCC has performed a significant amount of the SI role and assisted with technical integration expertise. 12 https://www.smartdcc.co.uk/media/1213/dt_0006_joint_test_strategy_v2_3.pdf DCC Public Page 35 of 118

Testing Delivery for R1.2 Testing Stages R1.2 64. The following diagram illustrates the full set and sequence of Test Phases and constituent Test Stages for the programme, covering DCC Readiness Testing and Informal Testing. Light blue boxes: Informal test phases, not part of DCC Readiness Testing. More information on R1.2 testing phases could be found in Appendix C. Delivery of testing for R1.2 65. In October 2016, DCC completed testing for R1.2 go live. This comprised Pre-Integration Testing (PIT), Systems Integration Testing (SIT), and Operational Acceptance Testing (OAT) conducted by Service Providers and assured by DCC. DCC also provided UEPT in the Interface Testing Stage (specific to R1.2) and End to End Testing Stage for its Testing Participant s (TP s). 66. Testing progress was slower than expected and finished beyond the date that was provided by Users as an estimate for both SIT and Interface Testing (IT). The number of issues discovered during testing was higher and the time taken to resolve issues was slower than expected, partly due to the complexity of testing with both a number of different meter types and emulators and the need to integrate complex system components for the first time. To mitigate against this the scope of SIT was narrowed and DCC exited using one meter type and emulators. 67. Additional testing was undertaken in the form of Operational Confidence Testing (OCT), which tested real life and negative scenarios for DCC go live. DCC Public Page 36 of 118