Data Interchange Framework and Glossary 1.01 Final September 2018

Transcription

1 Data Interchange Framework and Glossary 1.01 Final September 2018 Explains Data Interchange terms, abbreviations, and provides important upgrade information

2 Data Interchange Framework and Glossary 2018 AEMO i

3 Important Notice PURPOSE This Data Interchange Framework and Glossary, prepared by the Australian Energy Market Operator (AEMO), provides guidance for Data Interchange under the National NER or NGR (Rules). NO RELIANCE OR WARRANTY This document does not constitute legal or business advice, and should not be relied on as a substitute for obtaining detailed advice about the National Gas or Electricity Law, the Rules or any other applicable laws, procedures or policies. While AEMO has made every effort to ensure the quality of the information in this Guide, neither AEMO, nor any of its employees, agents and consultants make any representation or warranty as to the accuracy, reliability, completeness, currency or suitability for particular purposes of that information. LIMITATION OF LIABILITY To the maximum extent permitted by law, AEMO and its advisers, consultants and other contributors to this Guide (or their respective associated companies, businesses, partners, directors, officers or employees) are not liable (whether by reason of negligence or otherwise) for any errors, omissions, defects or misrepresentations in this document, or for any loss or damage suffered by persons who use or rely on the information in it. TRADEMARK NOTICES Microsoft is a trademark of Microsoft Corporation in the United States and/or other countries. Oracle and Java are registered trademarks of Sun Microsystems, Inc. in the U.S. and other countries. UNIX is a registered trademark of The Open Group in the US and other countries. DISTRIBUTION Available to the public. DOCUMENT IDENTIFICATION Business custodian: Specialist, IT Architecture IT custodian: IT Development Title: Data Interchange Framework and Glossary Prepared by: Technology, Technical Writers VERSION HISTORY Version 1.01 Updated for pdrmonitor v Last update: Friday, 28 September :48 DOCUMENTS MADE OBSOLETE The release of this document changes any previous versions of Data Interchange Framework and Glossary. FURTHER INFORMATION For further information, please visit AEMO s website or contact: AEMO's Support HubPhone: 1300 AEMO 00 ( ), supporthub@aemo.com.au FEEDBACK Your feedback is important and helps us improve our services and products. To suggest improvements, please contact AEMO's Support Hub AEMO ii

4 Contents Chapter 1 Introduction 1 Purpose 1 Audience 1 How to use this guide 1 What s in this guide 2 Chapter 2 About Data Interchange 3 What Data Interchange is for 3 Who can use Data Interchange 3 How do you use Data Interchange 4 User rights access 4 Set participant 5 Set participant 5 System requirements 5 Supported web browsers 6 Chapter 3 Components 7 Data Delivery 8 Data Interchange bundles 8 Data Subscription 8 Electricity and Gas Data Models 8 Data 9 MarketNet 9 Participant's DBMS 10 Participant Data Replication Batcher (pdrbatcher) 10 Participant Data Replication Loader (pdrloader) 10 Participant Data Replication Monitor (pdrmonitor) 10 Participant File Server 11 Performance Monitor 11 Chapter 4 Baseline Assumptions 12 Compatible software 12 Data Interchange versions 12 Database management systems Australian Energy Market Operator Limited. The material in this publication may be used in accordance with the copyright permissions on AEMO s website.

5 Chapter 5 Implementation 14 About Legacy Files 14 Important upgrade information 15 Upgrade implications 15 Transition 16 Chapter 6 Data Interchange Resources 17 Data Interchange software 17 Data Interchange guides 17 Needing Help 20 AEMO's Support Hub 20 Feedback 21 Glossary 22 Index AEMO iv

6 Chapter 1 Introduction Purpose 1 Audience 1 How to use this guide 1 What s in this guide 2 Purpose This guide explains Data Interchange components, terms, abbreviations, and provides important upgrade information. Read this guide in conjunction with other guides in the Data Interchange set and the release schedules and technical specifications relevant to Data Interchange and the Data Model. Audience This guide is for Registered Participants : Who use Data Interchange. Technical and software development staff, responsible for systems implementation. How to use this guide This guide is written in plain language for easy reading. Where there is a discrepancy between the Rules, and information or a term in this document, the Rules take precedence. Text in this format indicates a resource on AEMO s website. Glossary terms are capitalised and have the meanings listed against them. Italicised terms are defined in the NER or NGR. Any rules terms not in this format still have the same meaning. Actions to complete in interfaces are bold and dark grey AEMO 1

7 What s in this guide What s in this guide Chapter 2 About Data Interchange on page 3 provides an overview of the software, who can use it, how you use it, system requirements, supported web browsers, and user rights access. Chapter 3 Components on page 7 describes the components that make up a Data Interchange environment. Chapter 4 Baseline Assumptions on page 12 provides information about the minimum supported configuration of software and systems participants need to run Data Interchange. Chapter 5 Implementation on page 14 explains Legacy Files, provides important information about upgrading to a new Data Model version. Read this section in conjunction with Data Model release schedules and technical specifications. Chapter 6 Data Interchange Resources on page 17 explains where to find the Data Interchange software and explains the resources available to assist with installation, upgrades, and troubleshooting. Needing Help on page 20 provides some troubleshooting, a list of related resources, information about contacting AEMO's Support Hub, and how to provide feedback. Glossary on page 22 provides a list of Data Interchange terms and acronyms used throughout this guide. Terms included in the glossary are capitalised AEMO 2

8 Chapter 2 About Data Interchange What Data Interchange is for 3 Who can use Data Interchange 3 How do you use Data Interchange 4 User rights access 4 Set participant 5 Set participant 5 System requirements 5 Supported web browsers 6 This chapter provides a brief overview of Data Interchange. For more details and help setting up an environment, see Concise Guide to Data Interchange. What Data Interchange is for Data Interchange is a set of cooperating applications to replicate, manage, and monitor data between AEMO s Wholesale Energy Market Systems and participants' local DBMS conforming to the Electricity or Gas Data Models. Data Interchange is predominantly one way from AEMO to participants. Data Interchange has two core functions: 1. AEMO-side reporting applications that generate structured.csv files into the participant file server. 2. Participant-side software to replicate and monitor data from the Participant File Server to participants' local DBMS. Who can use Data Interchange Data Interchange is for Registered Participants requiring software to replicate data between their systems and AEMO s AEMO 3

9 How do you use Data Interchange How do you use Data Interchange AEMO recommends participants set up a standard configuration of Data Interchange, which is a single data feed, from one Participant ID folders on the Participant File Server, to a single participant database. For example, you can have your preproduction environment connected to the Participant File Server pre-production environment and your production environment connected to the Participant File Server production environment. Under no circumstance point your pre-production environment to the AEMO production environment or vice versa. AEMO provides the software to participants and each participant is responsible for setting up their own systems and satisfying their data requirements. Participants should decide if the complexity of using Data Interchange is a better business decision than using the simpler web portal applications that require only a web browser. The main Data Interchange steps are: 1. Participants decide what data they wish to receive using the Data Subscription web application in the Markets Portal. For help, see Guide to Data Subscription. 2. AEMO generates the data in structured CSV files, placed onto the Participant File Server. For help, see Connecting to AEMO's IT systems. 3. The pdrbatcher and pdrloader software on the participant-side replicates the data from the Participant File Server to participants' local DBMS. For help, see Guide to pdrbatcher and Guide to pdrloader. 4. Participants manage their local DBMS and the data feed. For help, see Concise Guide to Data Interchange. User rights access To access Data Interchange, Participant Users must have the appropriate user rights access. The access right determines the functionalities and transactions you can use to access the web portal, batch interfaces, FTP, and API services. Your company's Participant Administrator (PA) provides access to Data Interchange for Participant Users in the MSATS>Administration>Maintain Entities menu. The initial PA is set up by AEMO during the registration process. If you don't know who your company's PA is, contact AEMO's Support Hub AEMO 4

10 Data Subscription The entities required for access to each component are listed below. Where a participant user has user rights assigned by more than one participant, they interactively choose the participant they represent, using the Set Participant option. Data Subscription EMMS - Data Subscription - Maintain and View Data EMMS - Data Subscription - View Data For more details about participant administration and user rights access, see Guide to User Rights Management. Data Delivery EMMS - Data Interchange Performance Monitor EMMS - Data Interchange - Performance Monitor Set participant Set participant Where a participant user has user rights assigned for more than one participant ID, they interactively choose the participant they represent, using the Set Participant option. System requirements Using Data Interchange (DI) requires: Access to MarketNet; if your company is a registered participant, you probably already have access because it is set up during participant onboarding, for more details, see Guide to Information Systems AEMO 5

11 Supported web browsers A Participant ID and password, provided by your company's participant administrator (PA). PAs are set up during Registration, if you don't know who your company's PA is, contact AEMO's Support Hub. For help, see Guide to User Rights Management. A DBMS supporting the Data Model, either Oracle and Microsoft SQL Server. For the latest DBMS versions, see the latest Technical Specification on AEMO's website: A Java runtime engine suitable for the target DBMS. The distribution file from AEMO contains supported JDBC drivers for Oracle and SQL Server.DI components run on both Windows and Unix-like operating systems. Good IT skills, including networking, database management, batch file management, and disaster recovery. For help, see Concise Guide to Data Interchange. Supported web browsers To access the Data Interchange, AEMO recommends the following web browsers: Browser Platform Version More information Microsoft Internet Explorer Windows IE11 Microsoft Edge Windows 10 Latest Google Chrome All platforms Latest AEMO 6

12 Chapter 3 Components Data Delivery 8 Data Interchange bundles 8 Data Subscription 8 Electricity and Gas Data Models 8 Data 9 MarketNet 9 Participant's DBMS 10 Participant Data Replication Batcher (pdrbatcher) 10 Participant Data Replication Loader (pdrloader) 10 Participant Data Replication Monitor (pdrmonitor) 10 Participant File Server 11 Performance Monitor 11 This chapter describes the applications and resources that make up a Data Interchange environment. Figure 1 Data Interchange components 2018 AEMO 7

13 Data Delivery Data Delivery The web application in the energy Markets Portal allowing Registered Participants to: View the Data Interchange Status Dashboard Make a file delivery query Re-request files View your organisation's data sharing arrangements. Data Interchange bundles All the components necessary for a new installation of Data Interchange are bundled together into compressed files; each bundle is for a specific DBMS and version of the Data Model. The Data Interchange bundles are available from: Electricity or gas IT Systems web pages on AEMO s website. These bundles are for new participants who do not have their access to the Markets Portal. Data Subscription The web application in the Markets Portal allowing registered participants to: Obtain the latest Data Model scripts for building and maintaining their DBMS Subscribe and unsubscribe to the report files available in the participant file server View the entire list of available files. Electricity and Gas Data Models Establishes the target tables in a participant's DBMS conforming to the Data Model, including database tables, indexes, and constraints. It is currently delivered for Oracle and SQL Server. AEMO recommends subscribing to all available files and selectively loading them into your DBMS as required. This way the file is created and available if required in the future (approximately 6 months of history is retained online). Unsubscribing from files makes them unavailable in the future AEMO 8

14 Data Data sets provided from AEMO s pre-production are different from AEMO s production because the environments are operated independently. The Data Model is usually updated with each release for example, new tables or new columns. AEMO recommends participants keep their Data Model current with new releases to avoid data loss. Data Current data AEMO's website has the same data as distributed through the NEM file interfaces, with the exception of participant s private data. The data is mostly in CSV format with some graphical summaries. Data retention depends on the file type, for example, Dispatch files are kept for 2 days. The section for each file type explains how long each file remains in the current directory. To obtain current data, see Electricity or Gas data on AEMO's website. Baseline data Intended for initial loading into an participant's empty DBMS, to provide key standing data immediately prior to commencing the routine data feed. Participants can fully populate the tables with baseline data by copying each file in the \MarketData\Baseline folder into the pdrloader input folder for the matching source environment, for either production or pre-production. Historical data Older files are moved to the Reports archive and kept for 13 months. To obtain historical data, see MarketNet All participants must connect to the Participant File Server using AEMO s private data network MarketNet, which uses the FTP protocol. The pdrbatcher defaults to use the FTP protocol, simplifying the set-up to access the participant file server AEMO 9

15 Participant's DBMS Participant's DBMS Contains the data received from AEMO into tables matching the Data Model. AEMO s production and pre-production systems are independently operated, so ensure the data from one is not confused with the other. When maintaining the database, ensure compatibility is maintained with the Data Model. Not keeping up-to-date with new releases will compromise data integrity. Participant Data Replication Batcher (pdrbatcher) Responsible for transferring files to and from AEMO's participant file server to the participant's local Data Interchange folders. It can be installed in any environment supporting Java. pdrbatcher is a batch application, therefore does not have a graphical user interface. Configuration of the pdrbatcher is done using the.properties file, which is only read when the pdrbatcher starts up, so any changes require a restart of the application. pdrbatcher recovers from network failure to allow for recovery of missing data. Participant Data Replication Loader (pdrloader) Responsible for loading files to and from participant's local Data Interchange folders to the participant's DBMS. It can be installed in any environment supporting Java. pdrloader polls the participant s local Data Interchange folders and processes data from files into the participant s DBMS conforming to the Data Model. pdrloader is a batch application, therefore does not have a graphical user interface. Configuration of the pdrloader is done using the.properties file, which is only read when the pdrloader starts up, so any changes require a restart of the application. pdrloader has mechanisms to identify and recover missing data. Participant Data Replication Monitor (pdrmonitor) A web-based interactive interface allowing participants to configure and interrogate their Data Interchange instances. Participants install the software locally to run on either Windows or unix-like operating systems AEMO 10

16 Participant File Server Participant File Server The publishing point from AEMO systems to participant systems, with each participant allocated an account and access to private and public areas. Each participant is responsible for interfacing with the participant file server. If uncollected, files are moved to the archive folder after a couple of days. If your environment is configured properly it automatically retrieves the missing files from the archive. Files are kept in the archive for approximately six months. AEMO s production and pre-production environments are independently operated, so each environment has its own IP address for its participant file server. Performance Monitor The web application in the Markets Portal allowing Registered Participants to view a graphical interface of the current Data Interchange Performance Monitor environment run and managed by AEMO. Participants not receiving data, or experiencing data delays can contrast the data delivery performance of their Data Interchange environment to AEMO's Data Interchange performance environment AEMO 11

17 Chapter 4 Baseline Assumptions This chapter provides information about the minimum supported configuration of software and systems participants need to run Data Interchange. Compatible software 12 Data Interchange versions 12 Database management systems 13 Compatible software Data Interchange software runs on Windows and Unix -like operating systems. Current version DI release Java Data Interchange versions Component Current version Still supported Proposed future release Electricity Data Model Gas Data Model January n/a March 2019 PDR Loader (supporting pdrmonitor) 7.3 (until 30 November 2018) n/a 2018 AEMO 12

18 Database management systems Component Current version Still supported Proposed future release PDR Batcher (supporting pdrmonitor) 7.3 (until 30 November 2018) n/a PDR Monitor n/a n/a Replication Manager No longer supported (replaced by pdrmonitor) no no Database management systems DMS Current Still supported Microsoft SQL Server Oracle 12c 11g 2018 AEMO 13

19 Chapter 5 Implementation This chapter provides important information about upgrading to a new Data Model version. Read this section in conjunction with Data Model release schedules and technical specifications. About Legacy Files 14 Important upgrade information 15 Upgrade implications 15 Transition 16 About Legacy Files The design of the data model supports the latest and previous version of a file. AEMO refers to the previous version as the Legacy version, for example NETWORK_ OUTAGE_LEGACY. The example below describes what happens to an existing Legacy file in v4.19 when AEMO releases a new Data Model version, AEMO 14

20 Important upgrade information Important upgrade information The following notes are important for each upgrade of a Data Model release: When the new release is deployed, participants on the Previous Versions subscribed to files that change, continue to receive the same data because AEMO automatically moves their subscriptions to the Legacy file. For help, see Important upgrade information above. Participants on Discontinued Versions subscribed to Legacy Files may receive different content if the Legacy file content changes. For help upgrading or setting up a new Data Interchange environment, see Guide to Upgrading Data Interchange and Guide to setting up a standard Data Interchange Environment. To start receiving the Latest Version content, participants must subscribe to the latest version of files in the Data Subscription web application. The technical specification details the changes to the data model. Once subscribed, and the new data is received, you can unsubscribe from the Legacy Files in the Data Subscription web application. For help, see Guide to Data Subscription. Participants must ensure that all internal dependencies on discontinued tables are removed prior to a new release otherwise participant processes may be impacted. Upgrade implications To maintain systems in-line with AEMO s market systems, participants need to: Review and assess the impact on their market systems with respect to the changes implemented as part of the release. Change their systems prior to the implementation of the release to ensure they are up-to-date. AEMO recommends participants wholesale market systems are compliant with the latest Data Interchange software versions; see Chapter 4 Baseline Assumptions on page 12. Schedule staff and resources to upgrade their Data Interchange environments. If you have a system dependency on the formats of the non-electricity or non-gas Data Model reports you need to manage these dependencies using the detail provided in the technical specification AEMO 15

21 Transition Transition To receive new data in the modified packages and tables, you must upgrade your Data Model to the new version. AEMO automatically moves participants subscriptions to the Legacy file. This ensures participants on earlier Data Model versions, subscribed to files that change, continue to receive the same data. Some Data Model report versions may be discontinued. You must ensure you remove all internal dependencies on discontinued tables prior to the deployment of the release; otherwise, your processes may be impacted. Participants on earlier Data Model versions remain on any subscribed Legacy files but may receive different content if the Legacy file content changes. AEMO encourages participants using Data Interchange to make use of the four-week preproduction period to assess and test any impact to their market systems and business processes AEMO 16

22 Chapter 6 Data Interchange Resources This chapter explains where to find the Data Interchange software and explains the resources available to assist with installation, upgrades, and troubleshooting. Data Interchange software 17 Data Interchange guides 17 Data Interchange software Existing participants You can find the latest Data Interchange software for upgrades and associated documentation in the following locations: 1. Releases directory on the participant file share: FTP to > Data Interchange, pdrbatcher, pdrloader, and pdrmonitor. 2. Data Subscription web application in the Markets Portal: o o Production: Pre-production: New Participants The Data Interchange bundles on AEMO s website > electricity or gas IT Systems web pages, have all the components necessary for a new Data Interchange installation for participants who do not have their access to the Markets Portal or Releases share. Data Interchange guides Changing your password on the Participant Server, explains the password rules and how to You can find resources on AEMO's website AEMO 17

23 Data Interchange guides change your password to access the Participant Server. Comprehensive guide to Data Interchange, that includes options to consider before set up, different configuration options, and data recovery options. Concise Guide to Data Interchange, assists participants to understand AEMO's Data Interchange software, describing how to set up a standard Data Interchange environment to replicate data between AEMO's wholesale energy market systems and participants' local DBMS conforming to the electricity or gas Data Models. How to Connect to AEMO's Electricity IT Systems, explains the interfaces available for electricity participants and how to access them. Data Model GUI Installer Guide, covers the installation of the Data Model using one of the GUI Installers. Data Model Installation Note, lists the minimum infrastructure and steps required for setting up the Data Model on a suitably prepared database. For notes relevant to each DBMS, see the README.txt in the software bundle. Data Model Loader Configuration Implementation Note, contains instructions for installing pdrloader configuration files. Data Model Package Summary, summarises the Data Model report. For each package, it lists the tables in the package, together with a brief summary and entity diagrams. There are separate summaries for electricity and gas. Data Model Report, provides full details of all packages and tables in the Data Model. There are separate reports for electricity and gas. Data Model Table to File to Report Relationships Workbook, details the relationships between the Data Model tables, files, and reports. Guide to Data Sharing, explains how to share data with another participant ID using Data Interchange software. Guide to Information Systems, comprehensive guide providing an overview of AEMO's market systems used by participants. It is relevant to IT decision making during participant onboarding and provides an understanding of the IT systems requiring set up. Guide to Participant Data Replication Batcher (pdrbatcher), covers the setup and use of the Participant Data Replication Batcher (pdrbatcher) to move files between AEMO and participants' local folders. Guide to Participant Data Replication Loader (pdrloader), covers the setup and use of the Participant Data Replication Loader to parses csv files into a local DBMS conforming to the Data Model AEMO 18

24 Data Interchange guides Guide to Participant Data Replication Monitor (pdrmonitor), covers the configuration and use of the pdrmonitor software. Guide to setting up a standard Data Interchange environment, explains how to set up a standard Data Interchange environment. Guide to Troubleshooting Data Interchange, provides assistance with troubleshooting Data Interchange issues. Guide to upgrading a standard Data Interchange environment, explains how to upgrade a standard Data Interchange environment. Guide to User Rights Management, assists Participant Administrators to manage their Participant User's access to AEMO s systems. It also explains how to set up single user IDs for use with the Set Participant function in AEMO s web portals. Markets Portal Help, online searchable help for the Markets Portal web applications. Only available over a MarketNet connection: Participant Data Replication Monitor GUI Installer Guide, assists participants to install and configure the Participant Data Replication Monitor (pdrmonitor) software. Release Schedules and Technical Specifications, information about projects and changes to AEMO's IT systems affecting participants. AEMO provides this information as a service targeting business analysts and IT staff in participant organisations to assist with their technical implementation AEMO 19

25 Needing Help AEMO's Support Hub 20 Feedback 21 AEMO's Support Hub Contacting AEMO's Support Hub IT assistance is requested through one of the following methods: Phone: 1300 AEMO 00 ( ) For non-urgent issues, normal coverage is 8:00 AM to 6:00 PM on weekdays, Australian Eastern Standard Time (AEST). Information to provide Please provide the following information when requesting assistance from AEMO: Your name AEMO recommends participants call AEMO's Support Hub for all urgent issues. Organisation name Participant ID System or application name Environment: production or pre-production Problem description Screenshots For AEMO software-related issues please also provide: Version of software. Properties or log files AEMO 20

26 Feedback Support dump and Data Interchange instance name (if Data Interchange problem). If you are using pdrmonitor, you can send a support dump directly to AEMO: Feedback Your feedback is important and helps us improve our services and products. To suggest improvements, please contact AEMO's Support Hub AEMO 21

27 Glossary AEMC Australian Energy Market Commission AEMO Australian Energy Market Operator AER The Australian Energy Regulator, established by section 44AE of the Trade Practices Act AEST Australian Eastern Standard Time API Application Programming Interface; a set of clearly defined methods of communication between various software components. Batcher pdrbatcher CSV Comma-separated values; a file format for exchanging data. Data Interchange A set of cooperating applications used to replicate data between AEMO's energy market systems and a participant's DBMS conforming to the MMS Data Model. Data Interchange component An application making up part of a Data Interchange instance, such as pdrbatcher, pdrmonitor etc. Data Model The definition of the interface to participants of data published by AEMO for gas or electricity. A database conforming to the Data Model can contain a local copy of all current participant-specific data recorded in the main database. The Data Model includes database tables, indexes, and primary keys AEMO 22

28 Glossary Data Sharing Each participant ID can share their data with other participant IDs, subject to an authorised request. Implementing data sharing means the full Data Interchange data set (including private data) for the requesting participant ID is made available to another participant's sets of folders in the EMMS file server, as nominated in the request. The sharing of data is for both pre-production and production. A data sharing requirement may emerge, for example, due to a merger with, takeover of, or sale of another registered participant. The requirement may be to use one company's IT system to manage access, and process the data for two or more participant IDs using just one participant IDs set of folders in the EMMS file server. The Data Interchange web application in the Markets Portal provides the status of any data sharing agreements you have implemented. Data Subscription A web application in the Markets Portal used to subscribe to Data Interchange files and to obtain the latest MMS Data Model. These files are typically csv files containing public data and participant s private data. AEMO recommends participant use of the Data Interchange suite of products, supplied by AEMO, to automate receiving and storage of the files to their systems. AEMO provides working sample software for FTP access (Participant Batcher) to MarketNet and working sample software for Data Interchange. DBA Database administrator DBMS Database Management System DI Data Interchange DI System A collection of Data Interchange applications working together. Discontinued Report When a Legacy version of a file replaces an existing Legacy file, some versions of the included reports may change. The replaced report versions are referred to as discontinued. Participants on Discontinued versions may be impacted if a report is discontinued in a Legacy file they are receiving AEMO 23

29 Glossary Discontinued Version Refers to the tables, files, and reports deployed two versions ago. AEMO only supports the current and previous Data Model versions. EMMS Wholesale Electricity Market Management System; software, hardware, network and related processes to implement the energy market. File Logically groups one or more reports delivered as a physical file, for example BILLING. Participants subscribe to files in the Markets Portal > Data Interchange > Data Subscription interface. FTP File transfer protocol; a standard network protocol used for the transfer of computer files between a client and server on a computer network. GUI Graphical User Interface Latest Version Refers to the tables, files, and reports deployed in the latest Data Model release. Legacy File Currently supported but no longer updated. If a report is changed in a release, the previous version of the report and the latest version of other reports in the file are delivered from the Legacy file. AEMO automatically moves participants subscriptions to the Legacy file. This ensures participants on previous Data Model versions subscribed to files that change, continue to receive the same data. Once upgraded, and the new data is received, you can unsubscribe from the Legacy files. Participants still on discontinued file versions may receive different content if the Legacy file content has changed. Loader pdrloader Manifest Contains all known data discrepancies. MarketNet AEMO s private network available to participants having a participant ID 2018 AEMO 24

30 Glossary Markets Portal Web portal for access to AEMO's wholesale web-based applications. MSATS Retail Market Settlement and Transfer Solution MSATS Web Portal MSATS web-based interactive interface MW Megawatt MyTerm NEM National Electricity Market NER National Electricity Rules NGR National Gas Rules PA Participant Administrator Package One package contains several tables, files, and reports but not all files are in a package. Participant's DBMS Contains the data received from AEMO into tables matching the Data Model. Participant Administrator The company's Participant Administrator (PA) who manages the company's user access and security. The PA is setup during registration AEMO 25

31 Glossary Participant File Server The publishing point from AEMO systems to participant systems. Each participant is allocated an account and access to private and public areas. participants are responsible for interfacing with the participant file server. If uncollected, files are moved to the archive folder after a couple of days. If your Data Interchange environment is configured properly it automatically retrieves the missing files from the archive. Files are kept in the archive for approximately six months. AEMO s production and pre-production environments are independently operated, so each environment has its own IP address for its participant file server. Participant ID Registered participant identifier Participant User An end user of AEMO's systems set up by the company's Participant Administrator. pdr Participant Data Replication pdrbatcher Software responsible for transferring files to and from AEMO's participant file server to the participant's local Data Interchange folders. It can be installed in any environment supporting Java. pdrloader Software responsible for loading files to and from participant's local Data Interchange folders to the participant's DBMS. It can be installed in any environment supporting Java. pdrmonitor A component of Data Interchange, providing participants with visualisation of the status of their Data Interchange systems. The pdrmonitor replaces the Replication Manager. Pre-production AEMO s test system available to participants Previous Version Refers to the tables, files, and reports deployed in the previous Data Model release AEMO 26

32 Glossary Production AEMO s live system Replication Manager The previous software used to manage the configuration settings and monitor the status of the replication of data from AEMO s systems to the participant s DBMS. The Data Interchange replaces the Replication Manager. Report A data report that loads into a data model table. Identified by its type, subtype, and version. For example: BILLING,BILLINGASPAYMENTS,2 Report ID The name of the report contained in the Package (Report Type) Report Type The Data Model package name Rights The access right setup by the administrator providing access to different areas of the pdrmonitor. Rules The National Electricity or Gas Rules. Standard Data Interchange environment A single data feed, from one Participant ID folders on the participant file server, to a single participant database. For example, you can have your pre-production environment connected to the participant file server pre-production environment and your production environment connected to the participant file server production environment. Under no circumstance point your pre-production environment to the AEMO production environment or vice versa. URM User Rights Management; see the Guide to URM on AEMO's website Zip The file compression format used for exchanging data with AEMO AEMO 27

33 Index A About Legacy Files 14 B Baseline Assumptions 12 Baseline data 9 C Changing your password on the Participant Server 17 Compatible software 12 Components 7 Comprehensive guide to Data Interchange 18 Concise Guide to Data Interchange 18 Current data 9 D Data Delivery 8 Data Interchange bundles 8 Data Interchange resources 17 Data Interchange software and resources 17 Data Interchange versions 12 Data Model GUI Installer Guide 18 Data Model Installation Note 18 Data Model Loader Configuration Implementation Note 18 Data Model Package Summary 18 Data Model Report 18 Data Model Table to File to Report Relationships Workbook 18 Data Subscription 8 E Electricity or Gas Data Models 8 F Feedback 21 G Glossary 22 Guide to upgrading a standard Data Interchange environment 19 Guide to Data Sharing 18 Guide to Participant Data Replication Batcher (pdrbatcher) 18 Guide to Participant Data Replication Loader (pdrloader) 18 Guide to setting up a standard Data Interchange environment 19 Guide to Troubleshooting Data Interchange 19 H Historical data 9 I Important upgrade information 15 M MarketNet 9 Markets Portal Help 19 P Participant's DBMS 10 Participant Data Replication Batcher (pdrbatcher) 10 Participant Data Replication Loader (pdrloader) 10 Participant file server 11 pdrbatcher 10 pdrloader 10 pdrmonitor 10 Performance Monitor 11 S Set participant AEMO Data Interchange Framework and Glossary 28

34 Support dump 21 Supported Web Browsers 6 System requirements 5 T Transition 16 U Upgrade implications 15 User rights access AEMO Data Interchange Framework and Glossary 29