ELECTRONIC FILE TRANSFER SPECIFICATION

Transcription

1 S ELECTRONIC FILE TRANSFER SPECIFICATION COUNT: Page 1 of 29

2 DOCUMENT APPROVAL ROLE NAME SIGNATURE DATE Author Ben Haworth 19/03/2014 Editor Nicole Williamson-Ashi 11/11/2015 Reviewer Mark Pearce 20/11/2015 DOCUMENT HISTORY ISSUE DATE ISSUED REASON FOR ISSUE VERSION /07/2009 General review including removal of X.400 option. VERSION /08/2010 Definition of enhanced validation acknowledgement file format and error codes. VERSION /10/2010 Correction of J numbers. VERSION /2012 Document refresh VERSION /05/2015 Document and branding refresh. Additional enhanced validation codes added. VERSION /02/2016 Reformatting NOTE: If this document is received in electronic format approved signatures will not appear. Users should refer to hard copy versions if necessary. COUNT: Page 2 of 29

3 CONTENTS Document Approval 2 Document History 2 Introduction 4 DTS Acknowledgement background 5 Message Logs, Alerts and Acknowledgements 5 Significant Points in the Transfer 5 Transfer Service Levels 6 DTS Acknowledgement protocol 7 Operation 7 Acknowledgement File Format 8 Standard Acknowledgement File Format 9 Enhanced Acknowledgement File Format 11 examples of Standard Acknowledgement file 15 Example of Enhanced Acknowledgement file 16 Other aspects of data transfers 18 Application level acknowledgement and error handling 18 Sequencing and cycle numbers 18 Message version control Filenaming 19 Appendix a Acknowledgement Data Flows 20 A.1 Standard Acknowledgement Flow Structure Description 20 A.2 Standard Acknowledgement Data Item Descriptions 21 A.3 Enhanced Acknowledgement Flow Structure Description 24 A.4 Enhanced Acknowledgement Data Item Descriptions 25 COUNT: Page 3 of 29

4 INTRODUCTION 1. This section of the DTH defines the File Transfer Protocols used by the Data Transfer Service (DTS), so that application developers can design and implement the necessary applications, to enable successful transfer of data across the DTS. 2. The File Transfer Protocols are used to ensure the reliable transfer of data files across the DTS, and encompass the acknowledgements and errors that may be returned during the transfer. 3. In order for data to be transferred across the DTS, the transfer must conform to the following levels of protocols, in addition to the File Transfer Protocols specified in this document: 4. Host Interface Protocol (HIP): This includes the network protocol to be used for access to the Gateways (for example, TCP/IP), the file transfer mechanism (for example, FTP) and the local acknowledge mechanism. These details are covered in the DTH section 3 part 2, "Gateway Interface Specification". 5. User File Design Specification: This describes the standard structure required for files to be transferred over the DTS, including the information required in the file header and footer, and the rules for describing the data contained in the file. Details are covered in the DTH section 3 part 4, User File Design Specification. 6. Business Level or Application Protocols: This includes business level (application to application) acknowledgements and the handling of application level errors. Business level and application protocols are not within the scope of this document, although some of the key issues are discussed in this document in the section entitled DTS Acknowledgement Protocol. These issues are covered in detail by the various Data Flows defined in the Data Transfer Catalogue. COUNT: Page 4 of 29

5 DTS ACKNOWLEDGEMENT BACKGROUND 7. This section provides background information associated with the DTS Acknowledgement protocol. This is the level of acknowledgement that deals with the reliable transfer of data files from one User to another, that is, from the Sending User's Host to the Receiving User s Gateway. It does not include application-toapplication level acknowledgements, which may be required for business-specific purposes. Message Logs, Alerts and Acknowledgements 8. If data is transferred across the DTS various levels of reporting are available to the User. In addition to the User and network acknowledgements specified in this document, there are two other acknowledgements available to the User organisation regarding transfers: a) Audit logs are held centrally within the DTS and are accessible via the Web Tools application. They provide information on all significant events associated with file transfers through Gateways which the particular User has authority to see. Events recorded in the audit log include the arrival of a file at the Gateway, the successful conversion, encryption and compression, and the receipt of the file at the Receiving Gateway and the production and transfer of the network acknowledgement file. Access to the audit logs is provided through the DTS Web Tools, as documented in the DTS Web Tools User Guide. b) Alerts constitute all "significant" events and errors associated with the overall DTS and are raised at the Service Provider's network management centre. These are not visible to the end User, but allow the Service Provider to manage the network efficiently. Significant Points in the Transfer 9. The DTS provides a reliable end-to-end data transfer service, that is, from a Sending Gateway to a Receiving Gateway, and Users need not be concerned with the details of many of the stages of transfer within this end-to-end service. There are various points within the transfer however where the User is likely to require information about the progress of a file. The significant points are: a) Start of transfer is the point at which the DTS starts to transfer the file from the sender to the recipient. For FTP transfers, where the Gateway is "pulling" the files, it is the point when the Gateway successfully pulls a file, and renames or removes it from the originating Host. For FTP transfers where the Host "pushes" the file, it is the point when the file has been pushed, and has been accepted by the Gateway. This is the point from which the DTS Service Levels apply and the point at which the User knows that they have started to transfer the required data, which may be required for their bilateral service level calculations. COUNT: Page 5 of 29

6 b) Delivery of data is the point at which the DTS has delivered the data to the recipient or made it available on the Gateway for delivery (see below). It corresponds to the discharging of the Service Provider s responsibility for message delivery, and is the point at which the sending User can relinquish responsibility for transferring the file to the recipient. c) Data processed is the point at which the data has actually been processed by the recipient application, either successfully or with errors. This corresponds to the business level protocol and is outside the scope of this document. However, because business level protocols are key to successful end-to-end operation, this issue is discussed further in the section on other aspects of data transfers later in this document. d) The DTS has the ability to provide status information on other steps in the process of transferring a file from one party to another, much of which is recorded in the audit logs. In addition to the audit log, there are customised reports provided which can be used for routine monitoring of the achievement of service levels. 10. Any error detected during the transfer of a file is reported to the sending Host in the form of a negative DTS Acknowledgement. Transfer Service Levels 11. The DTS offers a variety of connection types depending on User anticipated traffic volumes and required availability Service Levels. Each connection type has a specified loading profile for each two hour period of the day. Provided the loading profile is not breached, the DTS will deliver 99% of files within two hours and 100% of files within four hours. In practise almost all files are processed and delivered with 5 minutes. Performance against these SLA s is published on a monthly basis by the ElectraLink helpdesk team. COUNT: Page 6 of 29

7 DTS ACKNOWLEDGEMENT PROTOCOL 12. This section defines how the DTS Acknowledgements are supported by the DTS. There are two types of DTS Acknowledgement: Standard and Enhanced. 13. User can also receive further, optional enhanced validation Reports in a CSV or XML format delivered to their Gateway which includes the same information as the Enhanced Acknowledgement, but in a different format. 14. The general principle for Standard Acknowledgements is that the application only requires a single acknowledgement from the DTS to the sending application, which indicates either: a) there was a failure in the transmission, and the reason for that failure; b) The transmission was successful, and the application relinquishes its responsibility for sending the data. 15. The general principle for Enhanced Acknowledgements is that the application may determine from the Enhanced Acknowledgement additional information regarding the content of the data which may help improve data quality, even if the data file has been successfully processed and delivered. 16. The sender can elect to receive the Standard Acknowledgement only, the Enhanced Acknowledgement only or both the Standard and Enhanced Acknowledgements. 17. Any acknowledgement by the receiving application that it has successfully processed the data is outside the scope of the DTS, and hence not covered in this document. 18. The remainder of this document uses the generic term DTS Acknowledgement to refer to both Standard Acknowledgements and Enhanced Acknowledgements. Where the text relates to only one type of Acknowledgement, the text will make this clear. Operation 19. When the DTS delivers a User File, a DTS Acknowledgement message is generated. The protocol in use at the Receiving Gateway determines the point at which the DTS Acknowledgement message is initiated: a) Where the Receiving Host is using the FTP interface to the Gateway in Host Push-Pull or Pull-Pull mode, the Acknowledgement message is initiated at the point where the Gateway has completed its processing of the message, and the User File is available for collection by the Receiving Host; or b) Where the Receiving Host is using the FTP interface in Gateway Push-Pull or Push-Push mode, the Acknowledgement message is generated at the point where the Gateway has completed its processing of the message, and the User File has been forwarded to the Receiving Host. COUNT: Page 7 of 29

8 20. The DTS generates a DTS Acknowledgement message in User File Format as described below, and this is transferred to the Host from which the User File originated using the appropriate protocol. You may elect to have your DTS Acknowledgements delivered to your normal receiving directory within the host, a separate Acknowledgement directory within the host, or you may have separate directories for your Standard Acknowledgements, your Enhanced Acknowledgements. 21. Should there be a failure in delivery by the Central Hub or a Gateway, for instance due to addressing or security problems, then an appropriate negative DTS Acknowledgement message is generated indicating the reason for the failure. Likewise, this is routed to the originating Host. 22. For each User File submitted to the DTS, there will be at one and only DTS Acknowledgement message generated where the sending User has elected to receive only Standard OR Enhanced Acknowledgements. Where a sending User has elected to receive Standard and Enhanced Acknowledgements, the sender will receive two DTS Acknowledgements for each message sent. 23. An application that wishes to check that the data has been received by the Receiving Gateway should wait for the DTS Acknowledgement (file or files) to be returned. If the file is not returned within a reasonable time this will usually be due to a problem, and there will be no benefit gained from re-sending the file. Automatic re-sending may in fact compound an already existing problem as the file may be delivered twice. If the application requires confirmation that the file has been received and processed by the remote application, rather than just delivered to the remote system, then a higher level of acknowledgement is required. This would be at a Business Process level and is outside the scope of the DTS. See also Application level acknowledgement and error handling later in this document. 24. Where an error occurs during the transfer of a file, then a negative DTS Acknowledgement, or error return, is sent by the DTS, indicating to the sending application the nature of the error. This negative DTS Acknowledgement is returned in the same form as the positive DTS Acknowledgement, in a file as described in Acknowledgement file format. Acknowledgement File Format 25. Positive and negative Acknowledgements are indicated via a DTS Acknowledgement file. This file is similar to the normal User File Format structure, and contains a single Acknowledgement flow, as defined below. The DTS Acknowledgement file uses the same variant of the User File format as the original User File, which may be fixed or variable. In the case of custom formats, such as Pool Transfer File Format, the DTS Acknowledgement file is returned in variable format. To identify the DTS Acknowledgement flows, and to easily differentiate them from the normal Data Flows, the format is slightly different as described below. COUNT: Page 8 of 29

9 Standard Acknowledgement File Format 26. This section details the structure and content of the standard acknowledgement files received by Users that have not selected enhanced acknowledgement. a) Header: The DTS Standard Acknowledgement file for normal data files contains a header record. The header record identifies the Data Flow as type A0001. It is a copy of the header record in the original file that was sent, except that the timestamp is changed and the original To and From fields are reversed. This ensures that the file can be handled as a normal User File. However, the DTS Standard Acknowledgement file is not routed by the Gateway using the normal routing tables. It is returned to the Host that sent the original data file; refer to the Data Transfer Handbook Section 3, Technical Information "Gateway Interface Specification" for a complete explanation of this process. Alternatively, you may request for your DTS Acknowledgements to be delivered to separate directories, as described above. The Timestamp reflects the time that the DTS Acknowledgement file was created. b) DTS Standard Acknowledgement Data Flow is a single Data Flow, consisting of two groups, each which appear only once in the file, with the following fields: Data Item Status Format Comments ORIGINATING FLOW GROUP M Char (3) 9ZY FLOW ORIGINATOR M Char(1) 1 = DTS ORIGINAL FLOW REFERENCE ORIGINAL FILE IDENTIFIER M Char(8) Data Flow and Version Number from the original message. M Char(10) From the original message ACTION TAKEN M Char(1) 1 = Delivered OK 3 = Delivery Failed Table 1: Standard Acknowledgement Data Flow First Group COUNT: Page 9 of 29

10 Data Item Status Format Comments SET OF ERRORS GROUP M Char (3) 9ZZ ERRONEOUS FLOW INSTANCE NUMBER ERRONEOUS GROUP IDENTIFIER M INT(8) 0 = APPLIES TO WHOLE MESSAGE M Char(3) 0 = applies to whole message or where the Group Identifier cannot be determined ERROR CLASSIFICATION M INT(4) See Table 3 below ERROR DESCRIPTION M Char(40) Textual Description of Error See Table 3 below Table 2: Standard Acknowledgement Data Flow Second Group A definition for this flow, using DTC 3.1 nomenclature, is provided in Appendix A. Error Classification Codes Code Description 10 Failed to Translate User File 20 Failed to Encrypt User File 30 Failed to Address Network File 40 Failed to Queue Network File 50 Failed to Send Network File 60 Failed to Acknowledge Network File 0 User File Delivered Table 3: Error Classification Codes c) Trailer record: This is the standard format trailer, containing mandatory items as defined in the User File Design Specification, with the timestamp reflecting the DTS Acknowledgement creation time. The mandatory items are: File id, Group Count, Flow Count and Timestamp. COUNT: Page 10 of 29

11 Enhanced Acknowledgement File Format 27. This section details the structure and content of enhanced acknowledgement files that will be received by those Users that have selected enhanced acknowledgements. Header: The DTS Enhanced Acknowledgement file for normal data files contains a header record. The header record identifies the Data Flow as type A0002. It is a copy of the header record in the original file that was sent, except that the timestamp is changed and the original To and From fields are reversed. This ensures that the file can be handled as a normal User File. However, the DTS Enhanced Acknowledgement file is not routed by the Gateway using the normal routing tables. It is returned to the Host that sent the original data file; refer to the Data Transfer Handbook Section 3, Technical Information "Gateway Interface Specification" for a complete explanation of this process. Alternatively, you may request for your DTS Acknowledgements to be delivered to separate directories, as described above. The Timestamp reflects the time that the DTS Acknowledgement file was created. DTS Enhanced Acknowledgement Data Flow is a single Data Flow, consisting of two groups, the first is mandatory and will only appear once; however the second group is optional but may appear many times in the same Enhanced Acknowledgement flow, each instance relating to a specific validation error up to a maximum of 100 errors. The groups have the following fields: Data Item Status Format Comments ORIGINATING FLOW GROUP M Char (3) 99Y FLOW ORIGINATOR M Char(1) 1 = DTS ORIGINAL FLOW REFERENCE ORIGINAL FILE IDENTIFIER M Char(8) Data Flow and Version Number from the original message. M Char(10) From the original message ACTION TAKEN M Char(1) 1 = Delivered OK 2 = Delivered OK (data validation warning) 3 = Delivery Failed ERROR CLASSIFICATION M INT(4) See Table 3 above. ERROR DESCRIPTION M Char(40) Textual Description of Error See Table 3 above. Table 4: Enhanced Acknowledgement Data Flow First Group COUNT: Page 11 of 29

12 Data Item Status Format Comments SET OF ERRORS GROUP M Char (3) 99Z ERRONEOUS FLOW INSTANCE NUMBER ERRONEOUS GROUP IDENTIFIER LINE NUMBER IN THE FILE LINE NUMBER IN THE FLOW M INT(8) 0 = applies to whole message M Char(3) 0 = applies to whole message or where the Group Identifier cannot be determined M INT(10) 0 failed before validation 1-n error line number in file M INT(10) 0 failed before validation 1-n error line number in flow instance DATA ITEM REFERENCE O Char(5) Data item reference (J number) from DTC or EFD VALIDATION ERROR CODE VALIDATION ERROR DESCRIPTION M INT(4) See Table 6 below M Char(40) Textual Description of Error See Table 6 below. Table 5: Enhanced Acknowledgement Data Flow Second Group A definition for this flow, using DTC 3.1 nomenclature, is provided in Appendix A. d) Trailer record: This is the standard format trailer, containing mandatory items as defined in the User File Design Specification, with the timestamp reflecting the DTS Acknowledgement creation time. The mandatory items are: File id, Group Count, Flow Count and Timestamp. The DTS acknowledgment file is formatted in the same variant, which is fixed or variable, as the original data file. There is no provision for custom format of acknowledgment files. CODE DESCRIPTION 101 Data item too long 102 Data item incorrectly formatted COUNT: Page 12 of 29

13 103 Group not recognised for this flow 131 Missing parent group 132 Group out of order 133 Group count outside allowed range 134 Mandatory data item missing 135 Data item present when should be null 136 Data item not in valid set 137 Data item not in MDD valid set 138 Data item value is invalid 141 Group count incorrect in trailer 142 Flow count incorrect in trailer 151 Missing or unrecognisable header 152 Incorrectly formatted header 153 Unrecognised flow and flow version number 161 Missing or unrecognisable trailer 162 Incorrectly formatted trailer 163 File ID in trailer does not match the header 171 Data item contains invalid character 181 Group too short 182 Group too long Table 6: Validation Error Codes COUNT: Page 13 of 29

14 28. The procedure for handling these errors needs to be defined locally. For many of the errors, for example incorrect formatting of the data file, it is unlikely that the end user applications are able to handle this directly, and some level of manual intervention may be expected. However, such local procedures are outside the scope of the DTS, and hence this document. Web Tools includes a product called D-Flow Master which may be used to correct data files ready for resubmission back onto the DTS. 29. If the content of the header record within the User File is not recognised, or does not contain sufficient information to constitute a DTS Acknowledgement file, no Acknowledgement file will be generated. In this instance, an internal alert will be generated and you may be contacted by the Service Provider. COUNT: Page 14 of 29

15 EXAMPLES OF STANDARD ACKNOWLEDGEMENT FILE 30. Given below is an example of a DTS Standard Acknowledgement file, using fixed variant User File format, for both a successful and a failed transfer. Note that the _ character is shown to separate fields for clarity and this must not be present in the actual file. White space is used to indicate the padding, needed to support the User File encoding rules for the fixed format variant. These must be included in the actual file. a) File sent by Host: ZHF_FILE567890_D _M_EELC_C_YELG_ _SAPP1_RAPP1_B_OPER... data... ZPT_FILE567890_ _ _ _ b) In the case of successful delivery, the Acknowledgement returned by the DTS would be: ZHF_FILE567890_A _C_YELG_M_EELC_ _RAPP1_SAPP1_B_OPER 9ZY_1_D _FILE567890_1 9ZZ_ 0_0 _ 0_User File Delivered ZPT_FILE567890_ 2_ 1_ c) In the case of rejection due to a translation error, the Acknowledgement returned by the DTS would have the following form: ZHF_FILE567890_A _C_YELG_M_EELC_ _RAPP1_SAPP1_B_OPER 9ZY_1_D _FILE567890_3 9ZZ_ 0_0 _ 10_Failed to Translate User File ZPT_FILE567890_ 2_ 1_ Given below is an example of a DTS Standard Acknowledgement file, using variable variant User File format, for both a successful and a failed transfer. Note that the character is the data item separator and this must be present in the actual file when using variable format. As this file is in variable format, there is no white space as padding is not needed. a) File sent by Host: ZHV FILE D M EELC C YELG SAPP1 RAPP1 B OPER... data... ZPT FILE b) In the case of successful delivery, the Acknowledgement returned by the DTS would be: ZHV FILE A C YELG M EELC RAPP1 SAPP1 B OPER 9ZY 1 D FILE ZZ User File Delivered ZPT FILE c) In the case of rejection due to a translation error, the Acknowledgement returned by the DTS would have the following form: ZHV FILE A C YELG M EELC RAPP1 SAPP1 B OPER 9ZY 1 D FILE ZZ Failed to Translate User File ZPT FILE COUNT: Page 15 of 29

16 Example of Enhanced Acknowledgement file 32. Given below is an example of a DTS Enhanced Acknowledgement file, using fixed variant User File format, for a successful transfer with no validation warnings, a successful transfer with two validation warnings and a failed transfer. Note that the _ character is shown to separate fields for clarity and this must not be present in the actual file. White space is used to indicate the padding, needed to support the User File encoding rules for the fixed format variant. These must be included in the actual file. a) File sent by Host: ZHF_FILE567890_D _M_EELC_C_YELG_ _SAPP1_RAPP1_B_OPER... data... ZPT_FILE567890_ _ _ _ b) In the case of successful delivery with no validation warnings, the Acknowledgement returned by the DTS would be: ZHF_FILE567890_A _C_YELG_M_EELC_ _RAPP1_SAPP1_B_OPER 99Y_1_D _FILE567890_1 0_User File Delivered ZPT_FILE567890_ 1_ 1_ c) In the case of successful delivery with two validation warnings, the Acknowledgement returned by the DTS would be: ZHF_FILE567890_A _C_YELG_M_EELC_ _RAPP1_SAPP1_B_OPER 99Y_1_D _FILE567890_2 0_User File Delivered 99Z_ 1_1 _ 2_ 1_Jnnnn_ 136_Data item not in valid set 99Z_ 1_1 _ 2_ 1_Jyyyy_ 134_Mandatory data item missing ZPT_FILE567890_ 3_ 1_ d) In the case of rejection due to a translation error, the Acknowledgement returned by the DTS would have the following form: ZHF_FILE567890_A _C_YELG_M_EELC_ _RAPP1_SAPP1_B_OPER 99Y_1_D _FILE567890_3_ 10_Translation Failure 99Z_ 0_0 _ 0_ 0 141_Group count incorrect in trailer ZPT_FILE567890_ 2_ 1_ Given below is an example of a DTS Standard Acknowledgement file, using variable variant User File format, for a successful transfer with no warnings, a successful transfer with 2 warnings and a failed transfer. Note that the character is the data item separator and this must be present in the actual file when using variable format. As this file is in variable format, there is no white space as padding is not needed. a) File sent by Host: ZHV FILE D M EELC C YELG SAPP1 RAPP1 B OPER... data... ZPT FILE COUNT: Page 16 of 29

17 b) In the case of successful delivery with no validation warnings, the Acknowledgement returned by the DTS would be: ZHV FILE A C YELG M EELC RAPP1 SAPP1 B OPER 99Y 1 D FILE User File Delivered ZPT FILE c) In the case of successful delivery with two validation warnings, the Acknowledgement returned by the DTS would be: ZHV FILE A C YELG M EELC RAPP1 SAPP1 B OPER 99Y 1 D FILE User File Delivered 99Z Jnnnn 136 Data item not in valid set 99Z Jyyyy 134 Mandatory data item missing ZPT FILE d) In the case of rejection due to a translation error, the Acknowledgement returned by the DTS would have the following form: ZHV FILE A C YELG M EELC RAPP1 SAPP1 B OPER 99Y 1 D FILE Translation Failure 99Z Group count incorrect in trailer ZPT FILE COUNT: Page 17 of 29

18 OTHER ASPECTS OF DATA TRANSFERS 34. This section discusses some of the issues around the higher level aspects of the transfer of data between parties, in particular the areas of application level acknowledgements, and the sequencing and ordering of Data Flows. Application level acknowledgement and error handling 35. The preceding section has described the acknowledgement process required for the transfer of files across the DTS. Additionally, there may be a requirement for application level acknowledgements or error handling. Such acknowledgements and errors are at a level above the transfer of data between systems, and associated with the receiving application, and whether it could successfully interpret the data contained within the file transferred. 36. Within this area, the following types of issues need to be addressed: a) Whether each flow requires a business level acknowledgement, to confirm that the receiving application has successfully processed the data; b) Whether there is an electronic mechanism provided to return errors found when processing the data, or whether manual or ad-hoc means are used. 37. These details vary between individual types of flows. This type of information is already provided in some flows, for example MPAS related flows, where each instruction flow has a corresponding accept or reject flow. Where this information is required by other flows, these are defined in the record structure within the Data Transfer Catalogue. Sequencing and cycle numbers 38. Each file transferred across the DTS includes a header record, which contains a file identifier. As noted in the specification for the User Files, the sending application / Host must ensure that these identifiers are unique (within the sending market participant). However, there is no other implied meaning to this identifier, and it is in the file header purely to enable the unique identification of files passing across the network. The DTS does not validate that the file identifier is unique. There is no requirement on the sending party to ensure that subsequent identifiers are consecutive. 39. Some application processes may require sequencing or cycle numbers within data transfers, for example: a) to enable applications to determine when a file in the sequence is missing; b) to enable applications to determine the order in which multiple files should be processed; and COUNT: Page 18 of 29

19 c) to ensure that the file that is received is the file that is expected. 40. Where such functionality is required it is provided for within the definition of the Data Flow itself. This has already been done for several flows where this is necessary, for example MPAS flows, and Settlement flows. The DTS functionality and the file header do not provide support for this type of functionality, and the User must be aware of the above conditions. 41. The DTS does not guarantee to deliver User Files across the network in the same order that they are submitted to the local Gateway. Where such ordering is important it must be provided within the data content by the application. Message version control 42. The Gateway software provides considerable flexibility in the control of different versions of messages being transferred over the network. 43. Message versions and version control are managed by Gemserv on behalf of the MRA Executive Committee. Users will have to ensure that their Gateway configuration is correct to support the required versions of messages. 5.4 Filenaming 44. The Gateway software also provides considerable flexibility in the naming of files, including acknowledgement files, including the use of data items from the header record in the filename. This means, for example, that the Acknowledgement filename could include, for example, the File Identifier of the failed file. COUNT: Page 19 of 29

20 APPENDIX A ACKNOWLEDGEMENT DATA FLOWS This appendix shows the DTS Acknowledgements in their logical format. Due to the fact that they are not a business-level flow, there are no plans to include the flows themselves in the DTC. A.1 Standard Acknowledgement Flow Structure Description Flow Reference: A0001 Flow Name: File Transfer Standard Acknowledgement Flow Description: Standard Acknowledgement Data Flow returned by the DTS for each file transferred Flow Version Number: 001 Flow Type: Electronic Flow Sensitivity: Low Data Protection: No Flow Ownership: ElectraLink FROM TO VERSION DTS All 3.2 Data Items: REFERENCE J0937 J1064 J1220 J1221 J1223 J1224 J1225 J1226 ITEM NAME Data Flow Reference and Flow Version Number File Identifier Action Taken Erroneous Flow Instance Number Erroneous Group Identifier Error Classification Error Description Flow Originator COUNT: Page 20 of 29

21 Flow Structure: Group Group Description Range Condition L1 L2 L3 L4 L5 L6 L7 L8 Item Name 9ZY Originating Flow Identification 1 G 9ZZ Set of Errors 1 G 1 Flow Originator 1 Data Flow Reference and Flow Version Number 1 File Identifier 1 Action Taken 1 Erroneous Flow Instance Number 1 Erroneous Group Identifier 1 Error Classification 1 Error Description Notes: This flow is generated by the DTS in response to each flow sent. A.2 Standard Acknowledgement Data Item Descriptions Item name: Item reference: Item ownership: Item Description: Action Taken J1220 DTP Code indicating status of original flow Values are: 1 Delivered OK 3 Delivery Failed As Valid set Text COUNT: Page 21 of 29

22 Logical Format: CHAR(1) Physical Length: 1 Item name: Erroneous Flow Instance Number Item reference: J1221 Item ownership: DTP Item Description: Indication of the erroneous flow instance if possible to identify specific flow Values are: 0 error applies to whole message Positive number indicating error instance As Valid set Integer Logical Format: INT(8) Physical Length: 8 Item name: Erroneous Group Identifier Item reference: J1223 Item ownership: DTP Item Description: Indication of the erroneous group identifier if possible to identify specific group Values are: 0 error applies to whole message or when the Group Identifier cannot be identified, for example where the Group Identifier is missing or incomplete Group identifier As Valid set String Logical Format: Char(3) Physical Length: 3 Item name: Error Classification Item reference: J1224 Item ownership: DTP Item Description: Code indicating the error that occurred Values are: 10 Failed to Translate User File 20 Failed to Encrypt User File 30 Failed to Address Network File 40 Failed to Queue Network File 50 Failed to Send Network File 60 Failed to Acknowledge Network File 0 User File Delivered COUNT: Page 22 of 29

23 As Valid set Integer Logical Format: INT(4) Physical Length: 4 Item name: Error Description Item reference: J1225 Item ownership: DTP Item Description: Text description indicating the error that occurred Values are: Failed to Translate User File Failed to Encrypt User File Failed to Address Network File Failed to Queue Network File Failed to Send Network File Failed to Acknowledge Network File User File Delivered As Valid set Text Logical Format: CHAR(40) Physical Length: 40 Item name: Flow Originator Item reference: J1226 Item ownership: DTP Item Description: Code indicating originator of Acknowledgement Values are: 1 DTS As Valid set Text Logical Format: CHAR(1) Physical Length: 1 COUNT: Page 23 of 29

24 A.3 Enhanced Acknowledgement Flow Structure Description The following definition sets out the flow structure and data items for use in the construction of Enhanced Acknowledgement files. Group Group Description Range Condition L1 L2 L3 L4 L5 L6 L7 L8 Item Name 99Y Originating Flow Identification 1 G 99Z Set of Errors 0-* G 1 Flow Originator 1 Data Flow Reference and Flow Version Number 1 File Identifier 1 Action Taken 1 Error Classification 1 Error Description 1 Erroneous Flow Instance Number 1 Erroneous Group Identifier 1 Line Number in the File 1 Line Number in the Flow 0 Data Item Reference 1 Validation Error Code 1 Validation Error Desc Note that unlike the standard Acknowledgement flow, Enhanced Acknowledgements may contain multiple instances of the 99Z group, each corresponding to and identifying a specific Enhanced Validation error within the original User data file. COUNT: Page 24 of 29

25 A.4 Enhanced Acknowledgement Data Item Descriptions Item name: Item reference: Item ownership: Item Description: Flow Originator J1226 DTP Code indicating originator of Acknowledgement Values are: 1 DTN As Valid set Text Logical Format: CHAR(1) Physical Length: 1 Item name: Data Flow Reference and Flow Version Number Item reference: Item ownership: Item Description: Data flow type and version number of the original inbound file Taken from original file Text Logical Format: CHAR(8) Physical LengTh: 8 Item name: File Identifier Item reference: Item ownership: Item Description: User file id of the original inbound file Taken from original file Text Logical Format: CHAR(10) Physical Length: 10 Item name: Item reference: Item ownership: Item Description: Action Taken J1220 DT Code indicating status of original flow COUNT: Page 25 of 29

26 Values are: 1 Delivered OK 2 Delivered OK (data validation warning) 3 Delivery Failed As Valid set Text Logical Format: CHAR(1) Physical Length: 1 Item name: Error Classification Item reference: J1224 Item ownership: DTP Item Description: Code indicating the error that occurred Values are: 0 User File Delivered 010 Failed to Translate User File 020 Failed to Encrypt User File 030 Failed to Address Network File 040 Failed to Queue Network File 050 Failed to Send Network File 060 Failed to Deliver Network File As Valid set Integer Logical Format: INT(4) Physical Length: 4 Item name: Item reference: Item ownership: Item Description: Error Description J1225 DTP Text description indicating the error that occurred Values are: User File Delivered Failed to Translate User File Failed to Encrypt User File Failed to Address Network File Failed to Queue Network File Failed to Send Network File Failed to Acknowledge Network File As Valid set COUNT: Page 26 of 29

27 Text Logical Format: CHAR(40) Physical Length: 40 Item name: Erroneous Flow Instance Number Item reference: J1221 Item ownership: DTP Item Description: Indication of the erroneous flow instance if possible to identify specific flow Values are: 0 error applies to whole message Positive number between 1 and n where n is equal to the flow count of the User file identifying the sequential position of the erroneous flow within the User file. As Valid set Integer Logical Format: INT(8) Physical Length: 8 Item name: Erroneous Group Identifier Item reference: J1223 Item ownership: DTP Item Description: Indication of the erroneous group identifier if possible to identify specific group Values are: 0 error applies to whole message or when the Group Identifier cannot be identified, for example where the Group Identifier is missing or incomplete. Group identifier As Valid set String Logical Format: Char(3) Physical Length: 3 Item name: Line Number in the File Item reference: Item ownership: Item Description: 0 failed before validation. Line number was not identified Positive number between 1 and n where n is equal to the line number within the user file where a validation error has been detected. Integer Logical Format: INT(10) Physical Length: 10 COUNT: Page 27 of 29

28 Item name: Item reference: Item ownership: Item Description: Line Number in the Flow 0 failed before validation. Line number was not identified Positive number between 1 and n where n is equal to the line number within a flow instance where a validation error has been detected. Integer INT(10) Logical Format: Physical Length: 10 Item name: Data Item Reference Item reference: J9999 Item ownership: ElectraLink Item Description: Code indicating the Reference in the DTC or EFD to the Data Item Text Logical Format: CHAR(5) Physical Length: 5 Item name: Validation Error Code Item reference: Item ownership: Item Description: Code indicating the validation error that occurred Values are: COUNT: Page 28 of 29

29 As Valid set Integer Logical Format: INT(3) Physical Length: 3 Item name: Validation Error Description Item reference: Item ownership: Item Description: Text description indicating the validation error that occurred Values are: Data item too long Data item incorrectly formatted Group not recognised for this flow Missing parent group Group out of order Child group count outside allowed range Mandatory data item missing Data item present when should be null Data item not in valid set Group count incorrect in trailer Missing or unrecognisable header Incorrectly formatted header Unrecognised flow and version Missing or unrecognisable trailer Incorrectly formatted trailer File id in trailer does not match header Data item contains invalid character Group too short Group too long As Valid set Text Logical Format: CHAR(40) Physical Length: 40 COUNT: Page 29 of 29