Market Trials Review Group. May 2, 2012

Market Trials Review Group May 2, 2012

Section 1 WELCOME & INTRODUCTIONS 4

Agenda 08:00 08:30 Welcome and Introductions Review of Today s Agenda 08:30 09:00 MTRG Objectives and Process 09:00 10:00 High Level Timeline & Test Stages - Connectivity, Structured, Unstructured, Parallel Operations 10:00 12:00 Connectivity Testing 12:00 12:30 LUNCH 12:30 02:30 TCR Testing 02:30 03:30 Parking Lot and Next Steps Optional topics if time permits: Brainstorm and prioritize test scenarios Discuss planning for and handling contingencies 5

Section 2 MTRG OBJECTIVES AND PROCESS 6

Agenda Group Overview (Why, What, Who) Review/Discuss Process (How) Timeline (When) Q&A 7

Why? Objectives Successful Market Trials Member engagement Transparency Success of this group means Well planned Market Trials Test Phases Participant understanding support of approach Smoother execution and go-live 8

What? Ad hoc group reporting to the CWG Members, vendors, SPP staff coordination Provide input into planning for Market Trials Test Phases Discussion(s) focused on individual Test Phase prior to development of Approach Provide feedback on Market Trials Test Phase Approaches prior to official publishing Meeting logistics TBD Group Scope Statement 9

Who? Market Trials Review Group Representatives Jessica Collins, XCEL Terry Gates, AEP Jim Jacoby, AEP Dave Erickson, AEP Tony Delacluyse, PCI Tyler Wolford, TEA Kaneisha Jones, XCEL Andrea Reordan, Tenaska 10

How? MTRG Input to Test Phase Approach Test Phase Approach Development (SPP) SPP publishes to MTRG MTRG Review/ Comment SPP Compiles Comments Comment Review/ Reconciliation Meeting SPP Updates Approach SPP publishes to MPs via CWG 11

When? MTRG review approach TBD aholbert: I m not sure how to best schedule as cutting time out of development window for approach documents does not seem feasible; may have to have approach review in parallel to publication or provide pseudodraft version for review. PIRG schedule follows this pattern: publish specs to PIRG 1 week for group comment 1 week for SPP compilation and addressing of comments 1-2 weeks for updates per comments before publication of updated version 12

When? All dates tentative. To be confirmed in Market Trials Connectivity Approach planning. 13

Market Participant Milestones 14

Section 3 HIGH LEVEL TIMELINE & TEST STAGES 15

High Level Timeline Gantt format 16

High Level Timeline Rod Format 17

Market Trials Testing Objectives Confirm SPP / Participant systems and business processes are ready to support the Integrated Marketplace by: Validating the technical connectivity for each Participant Exercising each Marketplace business function through a series of controlled tests Allowing SPP and Participants the flexibility to further verify systems and business functionality through a collaborative unstructured test Establishing confidence in the Marketplace systems in what will become the Production environment Allow for cooperative feedback, discussion, and education on the operational performance of Integrated Marketplace 18

Market Trials Connectivity Testing Connectivity Testing will verify technical connectivity between SPP and Participants including digital certificates, XML listeners, and firewall access Connectivity Testing will exercise the programmatic interfaces between SPP and the Market Participant as part of functional operations Connectivity Testing will also verify Participant access to the external user interfaces for SPP systems. Some user interfaces may not be available until later in Connectivity Testing Successful completion of Connectivity Testing will provide a measure of both SPP and Participant readiness for Structured Market Trials 19

Market Trials Connectivity Testing ICCP testing is considered a separate effort from the Market Trials Connectivity Testing SPP plans to publish the Market Trials Connectivity Test Approach by July 9, 2012 SPP reviews and informational sessions with Participants will start in July 2012 Market Trials Connectivity Testing is scheduled to start on January 7, 2013 All dates tentative. To be confirmed in Market Trials Connectivity Approach planning. 20

Market Trials Structured Testing Market Trials Structured Testing will cover both normal business processes and exception scenarios Market Trials Structured Testing will be performed as a controlled scenario test, which means that SPP will provide the test data that is to be submitted by the Participants To ensure that the expected outcomes are achieved, SPP will execute the operational functions central to each test using the test data provided to the Participants 21

Market Trials Structured Testing SPP plans to publish the Market Trials Structured / Unstructured Test Approach by December 10, 2012. This will be a single Test Approach covering both test phases SPP reviews and informational sessions with Participants will start in January 2013. These reviews will include a detailed walk-through of each scenario and associated test data Market Trials Structured Testing is scheduled to start on June 3, 2013 All dates tentative. To be confirmed in Market Trials Connectivity Approach planning. 22

Market Trials Unstructured Testing As we validate business functionality through Structured Testing, SPP will enable Participants to further validate each function through the Market Trials Unstructured Testing phase Test data will not be provided by SPP, rather Participants will be able to submit data as desired to exercise and validate various interactions between the Participant and SPP This approach should enable a controlled approach to enabling Participants to interact with broader Marketplace systems leading into Market Trials Parallel Operations 23

Market Trials Unstructured Testing SPP plans to publish the Market Trials Structured / Unstructured Test Approach by December 10, 2012. This will be a single Test Approach covering both test phases SPP reviews and informational sessions with Participants will start in January 2013 Market Trials Unstructured Testing is scheduled to start on July 8, 2013 All dates tentative. To be confirmed in Market Trials Connectivity Approach planning. 24

Market Trials Parallel Operations Test data will not be provided by SPP, rather Participants will be able to submit data as desired to exercise and validate various interactions between the Participant and SPP It is expected that the Market Participants will be submitting production or production like data on a daily basis SPP will support Parallel Operations as if in production, and will be staffed at go-live levels Integrated Deployment Testing, a test within Parallel Operations, will be coordinated across the entire SPP CBA footprint to demonstrate the capability of the Integrated Marketplace to successfully dispatch units, manage congestion, and balance the system 25

Market Trials Parallel Operations SPP plans to publish the Market Trials Parallel Operations Test Approach by May 27, 2013 SPP reviews and informational sessions with Participants will start in June 2013 Market Trials Parallel Operations Testing is scheduled to start on November 11, 2013 All dates tentative. To be confirmed in Market Trials Connectivity Approach planning. 26

Communications Plan Test Stage Kickoff Teleconference Bi-Weekly Teleconference (frequency to vary by Test Stage) Market Trials Liaison email distribution list SPP Market Trials Email Distribution List (Marketplace_MarketTrials@spp.org) Spp.org CWG project folder Communication with individual Market Participants Your Customer Service Rep Emails and Phonecalls to Market Trials Liaison GlobalScape 27

Issue Management MPs will not need to access SPP s Defect Mgmt. Tool Market Participants will submit Issues using a distribution list (do we want one specifically for this purpose?) SPP will triage all Issues and determine the root cause If determined to be a defect, SPP will enter the defect in the Defect Mgmt. Tool (SilkCentral) and prioritize for resolution SPP will maintain a Known Issues spreadsheet Only issues affecting current phase of market trials will be included, the upcoming phase will be added as it approaches New and resolved issues discussed on bi-weekly teleconference 28

Section 4 CONNECTIVITY TESTING 29

Market Trials Connectivity Testing Connectivity Testing will verify technical connectivity between SPP and Participants including digital certificates, XML listeners, and firewall access Connectivity Testing will exercise the programmatic interfaces between SPP and the Market Participant as part of functional operations Connectivity Testing will also verify Participant access to the external user interfaces for SPP systems. Some user interfaces may not be available until later in Connectivity Testing Successful completion of Connectivity Testing will provide a measure of both SPP and Participant readiness for Structured Market Trials 30

High Level Timeline Gantt format 31

Connectivity Testing Basics Connectivity Testing will verify technical connectivity between SPP and Participants including digital certificates, XML listeners, and firewall access Marketplace Production environment Security Digital certificates Firewall Static Commercial Model version December 2012 model 32

Connectivity Testing Scope of Programmatic Interface Testing Connectivity Testing will exercise the programmatic interfaces between SPP and the Market Participant as part of functional operations All member facing applications Both Inbound and Outbound All programmatic interface types API, FTP, etc. Critical interfaces Business Rule Validation will not be tested* 33

Connectivity Testing User Interfaces Connectivity Testing will also verify Participant access to the external user interfaces for SPP systems. Some user interfaces may not be available until later in Connectivity Testing Access to Marketplace UIs only Simple UI Interactions 34

Connectivity Testing - Logistics Test Stage Kickoff Teleconference Bi-Weekly Teleconference Testing cadence By Market Participant vs. By Functional Area Programmatic vs. User Interfaces By appointment Tue Thu 8:00 5:00 Mitigation planning Known Defects, Stage Fixes 35

Connectivity Testing Readiness Metrics Successful completion of Connectivity Testing will provide a measure of both SPP and Participant readiness for Structured Market Trials Participation of Market Participant Scheduled appointment Executed connectivity tests Confirmation by data point/ UI 36

Section 5 TCR TESTING 37

Agenda Status update on TCR Workstream TCR Market Trial details Potential Challenges 38

Section 5.1 STATUS UPDATE TCR WORKSTREAM 39

Current TCR Activities Mock TCR auction Using offline Hedge program 40+ members participating Data interface through GlobalScape Software (i-hedge) testing Three Pre-FAT demos of i-hedge completed 4 th Demo is underway 40

TCR Activities What s Next i-hedge Testing FAT: Scheduled in June 2012 SAT: Scheduled to start in July 2012 Overall TCR Workstream Timeline Market Protocols 1Q 2011 Mock TCR Runs 1 Q 2012 Design/ Customization 2 Q 2012 FAT/SAT/ Testing 3Q -4Q 2012 Market Trial Runs 1Q 2013-3Q 2013 Ready to Go Live 4 Q 2013 41

Section 5.2 TCR MARKET TRIALS PHASE 1 & 2 42

Market Trial : Objective First member facing testing using i-hedge system MUI Portal Interface with Oasis Transfer Tool, CMS and CMT Mimic Production (Go-Live) Plan for first year Periods offered: Phase I : March 1 st, 2014 through May 31 st, 2014 Phase II: June 1 st, 2014 through May 31 st, 2015 43

Su Mo Tu We Th Fr Sa Su Mo Tu We Th 1 2 3 4 5 1 2 3 6 7 8 9 10 11 12 6 7 8 9 10 13 14 15 16 17 18 19 13 14 15 16 17 Market Trials 20 21 2013 22 23 24 25 26 20 21 22 23 24 Phase 1 27 28 29 Phase 30 312 27 28 29 30 31 January February January February Su Mo Tu We Th Fr Sa Su SuMoMoTu TuWeWeThThFr FrSa Sa Su Mo Tu We Th 1 2 3 4 5 1 2 3 1 4 2 5 6 7 8 9 10 11 12 3 6 4 7 5 8 6 9 7 10 8 11 9 12 3 4 5 6 7 13 14 15 16 17 18 19 10 1311 1412 1513 1614 1715 1816 19 10 11 12 13 14 20 21 22 23 24 25 26 17 2018 2119 2220 2321 2422 2523 26 17 18 19 20 21 27 28 29 30 31 24 2725 2826 2927 3028 31 24 25 26 27 28 February March February March Su Mo Tu We Th Fr Sa Su SuMoMoTu TuWeWeThThFr FrSa Sa Su Mo Tu We Th 1 2 1 1 2 2 3 4 5 6 7 8 9 3 3 4 4 5 5 6 6 7 7 8 8 9 9 3 4 5 6 7 10 11 12 13 14 15 16 10 1011 1112 1213 1314 1415 1516 16 10 11 12 13 14 17 18 19 20 21 22 23 17 1718 1819 1920 2021 2122 2223 23 17 18 19 20 21 24 25 26 27 28 24 2418 2519 2620 2721 2822 30 24 25 26 27 28 March 31 March 31 Su Mo Tu We Th Fr Sa Su Mo Tu April We Th Fr Sa April 1 2 Su Mo Tu We Th Fr 1 Sa 2 Su Mo Tu We Th 3 4 5 6 7 8 9 3 1 4 2 5 3 6 4 7 5 8 6 9 1 2 3 44 4 10 11 12 13 14 15 16 7 10 8 11 9 1210 1311 1412 1513 16 7 8 9 10 11 TCR Market Trials Phase 1 Timeline (2013)

Phase 1 (Interim) ARR/TCR Objective of this phase is to represent first three months of market (i.e. March 1 st 2014 through May 31 st 2014) March and Spring periods On-peak and off-peak products March and Spring ARR: 100% system capacity March TCR: 100% system capacity Spring TCR: 60% system capacity 45

Phase 1 Incremental ARR/Monthly TCR Conducted prior to April and May e.g. process executed in March offers April TCR s On-peak and off-peak products Incremental ARR: 100% system capacity Following Month 2-round TCR o +50% remaining system capacity o 100% system capacity 46

June February 30 1 2 Su Mo Tu We Th Fr1 Sa2 Su Mo Tu We Th Fr Sa Sa Su Mo Tu We Th Fr Sa Su Mo Tu We Th Fr Sa July 3 4 5 6 7 8 9 3 14 25 36 47 58 69 2 1 2 1 2 3 4 5 6 Su Mo Tu We Th Fr Sa 10 11 12 13 14 15 16 710 811 912 1013 9 TCR 3 4 Market 5 6 7 8 Trials 9 Phase 2111431215 2 4 1316 Timeline 5 6 7 8 9 10 11 12 13 (2013) 1 2 3 4 5 6 16 17 10 18 11 19 12 20 13 21 14 22 15 23 16 1417 1518 1619 1720 918 21 1019 22 1120 23 12 13 14 14 15 15 16 17 18 19 7 8 9 10 11 12 13 23 2417 18 18 19 19 20 20 21 21 22 22Market 3023 21 Trials 24 1825 2013 1926 202716 2128172229 182730 19 20 21 21 22 23 14 15 24 16 25 17 26 18 27 19 20 3124 25 26 27 28 Phase 2831 2 Cont. 29 30 23 24 25 26 27 28 29 30 21 22 23 24 25 26 27 April March June May April 30 28 May 29 30 31 Sa Su SuMo MoTuTuWe WeThTh Fr Fr Sa Sa Su Mo Tu Su We Su Mo Th Mo Fr Tu TuSa We We Th Th Fr Fr SaJuly Su Mo Tu We ThAugust Fr Sa 2 1 2 1 2 3 4 5 6 1 2 113 Su2 4Mo3 5Tu 4We 6 Th Fr Sa Su Mo 1 Tu2 We3 Th 4Fr Sa 9 3 4 5 6 7 8 9 7 8 9 10 11 122 133 4 557 68 7 9 8810 911110122 11133 4 5 6 7 8 9 101 112 3 16 10 11 12 13 14 15 16 23 1417 15181619172018211922 9 2010 23 11 12 12 14 1315 14 16151517 71618817199 1820 10 11 12 13 14 4 15 5 616 7 17 8 189 10 30 2124 18251926202721282216 292717 30 18 19 21 2022 21 2322224 14 2325152426 162527 17 18 19 20 21 11 12 22 1323 142415 2516 17 Market 2831 29 Trials 30 2013 23 24 25 26 28 2729 28 302929 21 30 2231 23 24 25 26 27 28 18 19 29 2030 213122 23 24 Phase 2 Cont. May April 30 May28 29 30 31 25 26 27 28 29 30 31 Sa Su SuMo Mo June Tu TuWe WeThTh Fr Fr Sa Sa July Su Mo Tu We Th Fr August Sa September 6 Su Mo Tu 1We 2Th 1 3Fr Sa 2 4 3Su 5 4Mo 6 Tu We Th Fr Sa 1 Su 2Mo 3Tu We 4 Th Fr Sa Su Mo Tu We Th Fr Sa 13 7 8 9 10 11 1 12 13 5 6 7 8 9 10 111 2 35 46 5 7 6 8 9 10 11 1 2 3 1 2 3 4 5 6 7 20 2 314 4 155 166 17 7 18 8 19 20 12 13 14 15 16 177 188 9 10 12 1113 121413 15 4 165 176 18 7 8 9 10 8 9 10 11 12 13 14 27 9 1021 112212 2313 24 14 15 25 26 27 16 19 17 20 21 22 23 2414 2515 16 17 19 1820 192120 22 23 24 25 28 182919 3020 21 22 11 12 13 14 15 16 17 15 16 17 18 19 20 21 23 26 24 25 27 2628 2729 May 28 3029 3121 22 23 24 26 2527 262827 29 183019 3120 21 22 23 24 22 23 24 25 26 27 28 30 28 29 30 31 25 26 27 28 29 30 31 29 30 Sa Su Mo Tu We Th Fr Sa July 4 1 2 3 4 August September Su Mo Tu We Th Fr Sa 11 5 6 7 8 9 Su 10 Mo 11 Tu We Th Fr Sa Su Mo Tu We Th Fr Sa 1 2 3 4 5 6 18 12 13 14 15 16 17 18 1 2 3 1 2 3 4 5 6 7 7 8 9 10 11 12 13 25 19 20 21 22 23 24 4 525 6 7 8 9 10 8 9 10 11 12 13 14 14 15 16 17 18 19 20 26 27 28 29 30 31 11 12 13 14 15 16 17 21 22 23 24 25 26 27 15 16 17 18 19 20 21 28 29 30 31 18 19 20 21 22 23 24 22 23 24 25 26 27 28 47 August 25 26 27 28 29 30 31 29 30 Su Mo Tu We Th Fr Sa September

Phase 2 Annual ARR/TCR Objective of this phase is to represent 1 st calendar year for TCR process (i.e. June 1 st 2014 through May 31 st 2015) June, July, August, September, Fall, Winter, Spring 2015 periods On-peak and off-peak products All ARR: 100% system capacity June TCR: 100% System capacity July, August, September: 90% system capacity Fall, Winter, Spring: 60% system capacity 48

Phase 2 Incremental ARR/Monthly TCR Conducted prior to each month starting July e.g. process executed in June offers July TCR s On-peak and off-peak products Incremental ARR: 100% system capacity July, August, September TCR: 100% system capacity October TCR o +50% remaining system capacity o 100% system capacity No Monthly TCR auction beyond October 2013 49

Section 5.3 POTENTIAL CHALLENGES 50

Potential Challenges: What if Scenarios External system to TCR not completely ready Market Trails begins January 14, 2013 External system expected to be ready on/before Jan 1, 2013 TCR participating members not ready to participate MUI, Portal, TCR Strategy etc. MP participate in TCR market but decides not to participate in Day Ahead Market OR TCR market goes live on October 2013 but the rest of the market delays How to undo awarded TCRs? 51

Mak Nagle Manager, Congestion Hedging MarketplaceTCR@spp.org

Section 6 PARKING LOT & NEXT STEPS 53