Technical Note HDE TN GMSCS E TN GMSCS E HDE R3.02

Transcription

1 Technical Note HDE Technical Note TN GMSCS E TN GMSCS E HDE R3.02 Issue th June 2016

2 Blank Page

3 HDE Technical Note 1 Copyright and Trademark Notices 2016 Yokogawa Electric Corporation All Rights Reserved The copyright of the programs and online manuals contained in the software medium of the Software Product shall remain with YOKOGAWA. You are allowed to print the required pages of the online manuals for the purposes of using or operating the Product; however, reprinting or reproducing the entire document is strictly prohibited by the Copyright Law. Except as stated above, no part of the online manuals may be reproduced, transferred, sold, or distributed to a third party in any manner (either in electronic or written form including, without limitation, in the forms of paper documents, electronic media, and transmission via the network). Nor it may be registered or recorded in the media such as films without permission Trademark Acknowledgements CENTUM, ProSafe, Exaquantum, Vnet/IP, PRM, Exaopc, Exaplog, Exapilot, Exasmoc and Exarqe are registered trademarks of YOKOGAWA. Microsoft, Windows, Windows Server, SQL Server, Excel, Internet Explorer, SharePoint, ActiveX, Visual Basic, Visual C++, and Visual Studio are either registered trademarks or trademarks of Microsoft Corporation in the United States and other countries. Adobe and Acrobat are registered trademarks of Adobe Systems Incorporated, and registered within particular jurisdictions. Ethernet is a registered trademark of XEROX Corporation. All other company and product names mentioned in this manual are trademarks or registered trademarks of their respective companies. We do not use TM or mark to indicate those trademarks or registered trademarks in this manual. We do not use logos in this manual.

4 HDE Technical Note 2 Highlights The Highlights section gives details of the changes made since the previous issue of this document. Summary of Changes This is R3.02 Issue 1.0 of the document related to Product Library version 1.0. Detail of Changes The changes are as follows: Chapter/Section/Page Change

5 HDE Technical Note 3 Table of Contents Copyright and Trademark Notices... 1 Highlights... 2 Table of Contents... 3 Chapter 1 Introduction Glossary About this Document Purpose of Document Intended Audience Deployment Security... 6 Chapter 2 Overview...7 Chapter 3 Configuration Settings Configuration File File Format Settings Date Time Formats Actual Date Times Date Time Keywords Tag Configuration File Command Line Parameters Chapter 4 Exporting Data Filtering File Format Export File Naming Convention File Compression Saving of Exported Files Chapter 5 Running the Application Running Automatically Configuring Windows Task Scheduler Running Manually Running Multiple Instances Chapter 6 Logging and Reporting Log Creation Log File Naming Convention... 20

6 HDE Technical Note Log Content and Format Activity Log Progress Summary Activity Log - Errors Execution Failures... 21

7 HDE Technical Note 5 Chapter 1 Introduction This document explains the usage of Exaquantum HDE also known as Exaquantum Historical Data Export Tool. The tool is included for use with Exaquantum R3.02 onwards. This section describes the following: 1.1 Glossary Glossary (Section 1.1) About this document (Section 1.2) Deployment (Section 1.3) Security (Section 1.4) The following terms have a particular meaning in the Exaquantum HDE context. Term CSV HDE XML HDA OPC Data Historian Exaquantum Meaning Comma-separated values Historical Data Export Extensible Markup Language OPC Historical Data Access OLE for Process Control. An open standard based on Microsoft OLE that is quickly becoming the interface standard between process software packages. Exaquantum PIMS, as a Data Historian, receives and historizes plant data which is available to plant users and third party systems such as SAP, Maximo, etc. The Yokogawa Data Historian/PIMS/IIS/MES product 1.2 About this Document Purpose of Document This document explains the usage of the Exaquantum Historical Data Export Tool (HDE) Intended Audience This Technical Note is provided for Exaquantum Administrators and Engineers.

8 HDE Technical Note Deployment HDE is installed on the following Installation types: PIMS Server Combined Server The application is located: <Exaquantum Installation Folder>\System\HDE\HDE.exe 1.4 Security Users need to be a member of the following groups to run this tool. IT security Legacy model : QUserGroup IT security Standard model : QTM_DATA_READ A user running the tool - and not in these groups - will generate an execution error (see Section 6.4).

9 HDE Technical Note 7 Chapter 2 Overview The Historical Data Export tool is used to extract tag data from the Exaquantum Historian database. Extracted data is saved to CSV files in a configurable folder which can then be used by third party applications. Data is filtered using a CSV Tag Configuration file, this file specifies which tags will be extracted from the Exaquantum database. The HDE application can be executed manually via a command window or automatically as a Windows scheduled task. All application settings are stored in configuration files, these file paths are passed to the application via the command line; this means that multiple instances of the application can be executed at the same time performing different data extractions based on different configurations. Figure 2-1 Historical Data Export Flow

10 HDE Technical Note 8 Chapter 3 Configuration The HDE tool has two configuration files: Settings Configuration File (Section 3.1) Tag Configuration File (Section 3.1.3) 3.1 Settings Configuration File The Settings Configuration file contains settings which the application needs to function. The file uses an XML format which can be edited with a standard text editor. NOTE: A template settings file is installed along with the application called SETTINGS.xml, located in the application installation folder File Format Table 3-1 contains a sample of the settings configuration file format (contents of the actual file for your installation may differ). Table 3-1 Settings Configuration File format <?xml version="1.0"?> <Configuration> <Task_Name>Task One</Task_Name> <Start_DateTime>01/01/ :00:00</Start_DateTime> <End_DateTime>01/06/ :00:00</End_DateTime> <EQ_Server_Address>Exaquantum01</EQ_Server_Address> <Export_File_Directory>C:\Temp\HDE\Output</Export_File_Directory> <Export_Log_Directory>C:\Temp\HDE\Output</Export_Log_Directory> <Export_Log_File_Limit_Count>10</Export_Log_File_Limit_Count> <Compress_Files>True</Compress_Files> </Configuration>

11 HDE Technical Note Settings Table 3-2 contains all configuration settings and their descriptions. Table 3-2 Configuration Settings and Descriptions Setting Task_Name Start_DateTime End_DateTime EQ_Server_Address Export_File_Directory Export_Log_Directory Description The Task Name should be a unique name which is used to identify the task, this is particularly important when logs files are written as this will help identify multiple tasks. Specifies the starting date time for data extraction, please see Date Time Formats in Section for more information. Specifies the ending date time for data extraction, please see Date Time Formats in Section for more information. Server name or IP where the Exaquantum Historian exists. Directory path where exported files will be written to. This can be a network location if required. Directory path where log files will be written to. This can be a network location if required. Export_Log_File_Limit_Count Log file limit count, the tool will delete all old files to maintain the directory file count, regardless of file type. Therefore it is recommended that export log directory should be used only for export logs. If set to zero no file removal will take place. Compress_Files Specifies whether files should be zipped or not TRUE, FALSE Please note: this setting only applies to data files and not log files Date Time Formats There are two methods to specify start/end time periods: Actual Date times (i.e. 01/04/ :30:00) Date time Keywords (i.e. NOW, MIDNIGHT) It is expected that actual date times will be used for manual data extractions; where a fixed time range is required. Date time keywords are suited for automated extractions where fixed date times periods are inappropriate and a dynamic time is required.

12 HDE Technical Note Actual Date Times Actual Date times must be in local time, formatted as follows: dd/mm/yyyy HH:mm:ss. Table 3-3 describes the tokens in the date time format. Table 3-3 Actual Date Time Format Tokens Token dd MM yyyy HH mm ss Description The day of the month. Single-digit days must have a leading zero. The numeric month. Single-digit months must have a leading zero. The year in four digits, including the century. The hours in a 24-hour clock. Single-digit hours must have a leading zero. The minutes. Single-digit minutes must have a leading zero. The seconds. Single-digit seconds must have a leading zero Date Time Keywords Date Time keywords must be in UTC time formatted as follows D H M S KEYWORD. Table 3-4 describes the tokens in the date time format. Table 3-4 Date Time Keyword Format Tokens Token D H M S Description The number of days before the value of KEYWORD The number of hours before the value of KEYWORD The number of minutes before the value of KEYWORD The number of seconds before the value of KEYWORD KEYWORD NOW represents the current date and time when the application is run MIDNIGHT represents midnight of the current day The keyword part of the date time keyword format specifies a point in time relative to when the application is run (see KEYWORD token in Table 3-4) The time duration parts of the date time keyword format, i.e. D H M S, specifies the amount of time before the keyword value. The time duration parts are added together and then subtracted from the keyword value to produce a date time in the past. Examples of how to use date/time keywords: 1. Extracting the previous day s data daily Task is scheduled to run after midnight. Start date time = MIDNIGHT End date time = MIDNIGHT 2. Extract the previous 2 1/2 hours worth of data manually Start date time = NOW End date time = NOW

13 HDE Technical Note Tag Configuration File The Tag Configuration file specifies which tags data will be extracted from the Historian database. The configuration file is a comma separated format (.csv file). The name of the configuration file is of the user s choice, for example TagConfig.csv and can be created using notepad or any text editor. Each line within the configuration file will contain a tag item; tag items must be specified with.value appended to their full paths for example if Tag2 s full path value Root.Tag2 was being exported, then the tag item would be Root.Tag2.Value (without the double quotes). The configuration file should contain no duplicates and only valid intrinsic tags which currently exist in the Exaquantum Historian database. Note that RBNS paths are not supported. Table 3-5 shows the format of the configuration file. Root.Tag1.Value Root.Tag2.Value Root.Tag3.Value Table 3-5 Tag Configuration File Format 3.3 Command Line Parameters The HDE tool uses command line parameters in order to define the location of its configuration files. This allows multiple instances of the application to run simultaneously while performing different functions; this is because each instance of the application could use different configuration files. Table 3-6 contains all the applications command line parameters and their descriptions. Table 3-6 Command Line Parameters Parameter Setting Configuration File Path Tag Configuration File Path Description Path to a configuration settings file. Path to a tag configuration file. NOTE: Standard command line rules apply; parameters which have spaces should be contained within quotes.

14 HDE Technical Note 12 Chapter 4 Exporting Data 4.1 Filtering The application only exports tag data from the tags which are listed in the Tag Configuration file(see section 3.2). If no tags are listed in the file the application does not raise an error but instead displays the values for Percentage Complete & No of Items Processed as 0% & 0 respectively, in the corresponding Activity log file (see Chapter 6). Invalid tags will be logged to the Activity log (see Chapter 6), they will not halt the extraction process; instead the application will attempt to continue and process the remaining valid tags. 4.2 File Format A CSV file will be formatted with the following comma separated columns please note the absence of tag names, this is because the file only contains values for a particular tag and the tag name is contained within the file name. The CSV column order is: Timestamp Value Quality (usually displayed as a numeric value - e.g. 0, 64, 192 etc.) Table 4-1 Tag Export File Format 26/12/ :30:30, , Good (or as a numeric value) 26/12/ :30:35, , Good (or as a numeric value) 26/12/ :30:43, , Good (or as a numeric value) 4.3 Export File Naming Convention A comma separated file is created for each tag configured in the Tag Configuration for the defined extraction period. File names comprise of the following parts: Date and Time (ddmmyyyy_hhmmss) Tag Name (any periods in the name will be replaced by underscores) Number of entries in the file Example: _ Root_Tag1_Value 835.CSV Each part of the file name is separated by a double underscore. NOTE: The dates and times in the filename and the file contents are in UTC. This makes the data easier to move from one server to another which is not in the same time zone. 4.4 File Compression Exported data files can be compressed, when providing the correct configuration setting (see section 3.1). Standard zipping compression algorithms are used; this means third party zipping utilities can unzip these files. This document will not discuss the compression ratio as this may vary based on the content being zipped, that said one should expect file sizes to be halved at the very least.

15 HDE Technical Note Saving of Exported Files Exported files are saved to a designated location which is specified in the Settings file (see section 3.1). If an export directory has not been provided in the Settings file the tool will raise an error to the event log, it will also not continue to process files until the setting has been set. If the export directory cannot be found when files are being exported the application will raise an error to the event log. The application will not continue to process remaining files and thus will not fill the event log with hundreds of failed file write errors.

16 HDE Technical Note 14 Chapter 5 Running the Application 5.1 Running Automatically The HDE tool has no functionality to automate its exportation of data; this functionality is reserved for a third party application such as Windows Task Scheduler Configuring Windows Task Scheduler Creating a scheduled task requires 5 main steps to be performed; this document will explain these steps in a summarized format, for additional information please see Microsoft s documentation on task scheduling Step 1 General Settings The General settings allow for the naming and describing of the scheduled task; also included are various security settings. Figure 5-1 Windows Task Scheduler General Tab

17 HDE Technical Note Step 2 Trigger Settings The triggers section defines when a task will run, there are numerous trigger options which can be utilized; our design choice will be based on date and time. Figure 5-2 Windows Task Scheduler Trigger Settings

18 HDE Technical Note Step 3 Action Settings The Actions settings allow scheduled task to be linked to software, it is here where the Exaquantum HDE tool location and command line parameters will be specified. Figure 5-3 Windows Task Scheduler Action Settings

19 HDE Technical Note Step 4 Condition Settings These settings are used to control whether a scheduled task can run, there are numerous conditions which can be configured to fine tune the execution of the tool. Figure 5-4 Windows Task Scheduler Condition Settings Tab

20 HDE Technical Note Step 5 - Settings The settings tab contains additional settings which can be used to further effect the behavior of the scheduled task. Figure 5-5 Windows Task Scheduler Settings Tab

21 HDE Technical Note Running Manually The Exaquantum HDE tool can be run manually; this activity is expected to be performed for one off data extractions. Windows offers a multitude of ways by which applications can be executed, all of these methods are appropriate when running the Exaquantum HDE tool, so long as the command line parameters can be set as described in section 3.3. Manual execution of the application will require the creation of a new settings configuration file which is populated with the custom data extraction periods and a new Tag Configuration file which will specify which tag data to export. These configuration file paths can then be passed as parameters to the Exaquantum HDE tool; the tool will read these files and perform the extraction as a one off. Figure 5-6 demonstrates the usage of the Exaquantum HDE tool through the Windows command line. 5.3 Running Multiple Instances Figure 5-6 Executing Application using Command Line Multiple instances of the HDE tool can be run, in such cases each application instance should be using different configuration files. CAUTION: Care must be taken not to run too many instances which might degrade the performance of production servers.

22 HDE Technical Note 20 Chapter 6 Logging and Reporting The activity log summarizes the applications activity on completion, provides a progress report while the application is running and reports any errors that may have been raised during execution. 6.1 Log Creation An Activity Log will be generated every time the Exaquantum HDE tool is executed successfully; it will be saved to the configured location as defined in the Settings Configuration file (Section 3.1). The log file will be continuously updated while the application is running; this means users will be able to view the progress of the application while it is extracting data. 6.2 Log File Naming Convention The log file name comprises of the following parts: Task Name (Defined in Setting Configuration File) Date and Time (ddmmyyyy_hhmmss) Example: Daily Tag Extraction _ log Each part of the file name is separated by a double underscore. NOTE: The dates and times in the filename are in local time. 6.3 Log Content and Format The log file is formatted as follows: Progress Summary Percentage Complete: 23% - Number of Items Processed: Time Taken: 06:00: Errors Tag Root.Test.CPV1 - Unknown Item path Activity Log Progress Summary The log file periodically updates the Number of Items Processed with the total number of items processed; the application will also keep a running time on how long it has taken to process these items which will get written to the Time Taken value Activity Log - Errors The Errors section contains configuration specific errors such as invalid tag names or paths.

23 HDE Technical Note Execution Failures Execution failures are reported in the Windows Event log, with the source HDETool. Table 6-1 contains all the applications error messages and their descriptions. Table 6-1 Application Error Messages Error Message Setting Task Name is invalid Setting Extraction start and or end date time is invalid Setting Exaquantum Server Address is invalid Setting Export Directory does not exist Setting Export Log Directory does not exist Setting Export Log File Limit is invalid Setting Compress files type in not a valid type. Tag [Tag Path Name] is a duplicate entry in the Tag Configuration file. Description "Task_Name" in the "Settings configuration file" is incorrect. "Start_DateTime" or "End_DateTime" in the "Settings configuration file" is incorrect. "EQ_Server_Address" in the "Settings configuration file" is incorrect. "Export_File_Directory" in the "Settings configuration file" is incorrect. "Export_Log_Directory" in the "Settings configuration file" is incorrect. "Export_Log_File_Limit_Count" in the "Settings configuration file" is incorrect. "Compress_Files" in the "Settings configuration file" is incorrect. The tag path names are duplicated in the "Tag Configuration File".