ARIES. ARIES C&DH Software Data Content & Format

Transcription

1 585 Charest Blvd. East, Suite 300 Québec, Québec G1K 9H4, Canada Phone (418) Fax (418) or (418) Airborne Research Interferometer Evaluation System C&DH Software Data Content & Format Document Number: BOM Issue Date: 24 August 2008 Issue: 1 Revision: Function Name Signature Date Checked by Software Developer Hugo Savard Updated & System/Project Scientist Robert Poulin Approved by PROPRIETARY The information contained herein is proprietary to and should be considered a trade secret of ABB Bomem Inc. It shall not be reproduced in whole or in part without the written authorization of ABB Bomem Inc. FO , rev 0603 Document Ingénierie

2 C&DH Software Data Content & Format Document No: BOM Date: 24 August 2008 i TABLE OF CONTENTS 1. INTRODUCTION Purpose of Document Reference Documents Acronyms Definitions BINARY FORMAT AND CONTENT OF C&DH BFS FILES Type definitions Data Format Outline Data Format Description Reading primary header and offsets table Reading secondary header Reading data records FORMAT AND CONTENT OF HOUSEKEEPING DATA FILES Detailed Housekeeping Data Format and Content... 8

3 C&DH Software Data Content & Format Document No: BOM Date: 24 August 2008 ii LIST OF TABLES Table 2 1 Definition of data types used in data files... 2 Table 2 2 Primary header and offsets table data fields Table 2 3 Secondary header data fields Table 2 4 Data records (measurements) data fields Table 2 5 Interferogram data size as function of resolution settings Table 3 6 Housekeeping data file sample Table 3 7 Housekeeping data fields description.... 8

4 C&DH Software Data Content & Format Document No: BOM Date: 24 August 2008 iii DOCUMENT CHANGE RECORD Issue/Rev Date (yy-mm-dd) Change Description Version updated to comply with final format and information transmitted to MetOffice. Internal information annotation added to Table 2-2. Unused data fields "Field of view channel A", "Field of view channel B", "Target area" and "Range to target" removed from Table 2-3. Unused "Acquisition Time" removed from Table 2-4 and replaced by a specification of the number of data bytes to skip at beginning of the data record data block. Cold blackbody PRT#8 and #9 data fields added for data files having file version "1.01". Those data fields are not present for files having version "1.00". Table 2-5 modified to remove unused material. A section has been added to document the format and content of housekeeping data files. Updated by Robert Poulin. Approved by Robert Poulin. (Draft)A First draft of document transmitted to MetOffice. Document structure improved to ease understanding of the file format and content and how to access the data. Updated by Robert Poulin. Approved by Robert Poulin. (Draft) First draft of document submitted for internal revision. Some data fields are not relevant for the instrument and will be removed in a future revision of the document. Some other data fields (to be defined) will be added. Prepared by Jean-Francois Leroux. Approved by Robert Poulin.

5 C&DH Software Data Content & Format Document No: BOM Date: 24 August INTRODUCTION 1.1 PURPOSE OF DOCUMENT The purpose of this document is to describe the content and format of the data files generated by the C&DH software when used in data acquisition mode. The data can then be converted as described in Ref REFERENCE DOCUMENTS Ref. 1 AA C&DH Software User's Guide, August Ref. 2 AR-BOM-061/94 Software Description, March ACRONYMS BFS C&DH Bomem File System Command and Data Handling 1.4 DEFINITIONS Data Record Part of a data file that contains annotated measurement data corresponding to a specific acquisition time, e.g. one interferometer sweep.

6 C&DH Software Data Content & Format Document No: BOM Date: 24 August BINARY FORMAT AND CONTENT OF C&DH BFS FILES The C&DH software saves data files using the Bomem File System (BFS) format. This section describes how to access the data fields contained in both the headers and data records parts of the data files. 2.1 TYPE DEFINITIONS Table 2 1 lists the data types that are contained in a data file. Array of data types are also used to specify some data fields, e.g. <long(12)> means an array of 12 longs. It is important to note that byte-ordering is little-endian (i.e. Intel/PC like). Table 2 1 Definition of data types used in data files. Data Type byte Description 8-bits signed integer. char Unicode character made of two <bytes>, e.g. 'A' = 0x0041. short integer long double string 16-bits signed integer. 32-bits signed integer. 64-bits signed integer. 32-bits IEEE ing point number. 64-bits IEEE ing point number. A string made of an <integer> containing N followed by N <char> where N is the size of the string, i.e. <integer> followed by <char(n)> for a total of 4 + N*2 bytes. 2.2 DATA FORMAT OUTLINE Each data file generated in the BFS format is made of the following sections: Primary header this small section has a fixed size binary format and contains information such as the BFS format ID and version, the number of data records contained in the file and various byte offsets to provide direct access to other data file sections. Data records this section is made of successive data records which contain the actual measurements data and their corresponding auxiliary data fields. Offset and time tables this section contains byte offset and time tables that allow direct/random access to any data record. Secondary header this section contains auxiliary data fields that do not depends on measurements, i.e. that are the same for all data records (e.g. resolution setting).

7 C&DH Software Data Content & Format Document No: BOM Date: 24 August The BFS file format is optimized for data acquisition. As a result, information such as the number of data records and table of byte offsets to data records are generated and written to the file after the data records. CAUTION: Specific data content depends on the data file version. A special data field used to identify the data file version is found in the secondary header (see Table 2 3). That data field must be taken into account to know how to read properly a specific data file version. 2.3 DATA FORMAT DESCRIPTION Reading primary header and offsets table Table 2 2 below lists the data fields that can be accessed sequentially or randomly according the byte offset. The first column defines variables used to compute byte offsets necessary to browse the data randomly. Table 2 2 Primary header and offsets table data fields. Byte Offset Type Description 0 char(17) BFS file ID and format version (last <char> is 0x001a). 34 char(33) Data Origin (last <char> is 0x001a). 100 char(33) Application (last <char> is 0x001a) Internal information. 421 char(128) Description (last <char> is 0x001a). 422 byte = 0 Byte ordering (0 = little-endian, 1 = big-endian) Internal information. N 428 integer Number of data records (measurements). 432 long Data file creation date (in milliseconds since January 1 st 1970 GMT) Internal information. O1 456 long Byte offset of the secondary header data fields. O2 464 long Byte offset of the first data record data. O3 472 long Byte offset of the table containing the offsets of each data record Internal information and data records. O4(N) O3 long(n) Byte offset table containing the N offsets of each data record Reading secondary header The secondary header is located at the byte offset <O1> with respect to the beginning of the file. The data fields of the secondary header can be read sequentially as defined by Table 2 3 below.

8 C&DH Software Data Content & Format Document No: BOM Date: 24 August Table 2 3 Secondary header data fields. Type Description string Data file identification (always ' C&DH Data File') string Data file version ('1.00' or '1.01') string Data file description (e.g. 'Raw data for NEdT') string Software version (e.g. '1.0.9') byte Scan mode (0:half-scan, 1:coadded scans) integer Number of measurements (count) integer Number of coadded scans (count) byte Trigger mode (not used, always 0) byte Resolution setting (0, 1, 2, 3 or 4) byte Autogain flag (always 0) byte Target type (0:science, 1:calibration) byte Data type (0:interferogram) string Detector type channel A (always 'MCT') string Detector type channel B (always 'InSb') byte Phase corrected flag (not used, always 0) byte Apodization type (not used, always 0) double Sampling frequency channel A (samples/cm) double Sampling frequency channel B (samples/cm) Spectral correction channel A (ppm, not used, always 0) Spectral correction channel B (ppm, not used, always 0) Blackbody temperature (ºC) Blackbody emissivity (no unit, >0, <=1) byte Aperture setting channel A (not used, always 0) byte Aperture setting channel B (not used, always 0) byte Error code (0:ok, 1: an error occured) Reading data records The first data record is located at the byte offset <O2> with respect to the beginning of the file. The data fields of the first data record can be read sequentially as defined by Table 2 4 below. The remaining data records (up to <N>) can be read successively in the same manner. Alternatively, one can directly read the ith data record which is located at the byte offset <O4(i)> with respect to the beginning of the file.

9 C&DH Software Data Content & Format Document No: BOM Date: 24 August Table 2 4 Data records (measurements) data fields. Type Description byte(12) Internal data (to skip) double UTC measurement time (milliseconds since Jan 1st, 1970) integer Measurement count integer Interferometer sweep count byte Interferometer sweep direction (0:forward, 1:reverse) byte Gain setting channel A (0, 1, 2, 3 or 4) byte Gain setting channel B (0, 1, 2, 3 or 4) Gain channel A (1, 2, 4, 8 or 16) Gain channel B (1, 2, 4, 8 or 16) ZPD value channel A (mv) ZPD value channel B (mv) byte Detector saturated channel A (0:no, 1:yes) byte Detector saturated channel B (0:no, 1:yes) byte ADC overflown channel A (0:no, 1:yes) byte ADC overflown channel B (0:no, 1:yes) Maximum proportion of ADC filled for channel A (%) Maximum proportion of ADC filled for channel B (%) Scan speed/frequency (scans/min, one scan is 1 forward + 1 reverse interferometer sweep) Scan mirror angle (degree) byte Pressure status (0:bad, 1:good) Sterling cooler voltage (mv) Beam splitter temperature (ºC) Reference plate temperature (ºC, not used) Auxiliary input temperature (ºC, not used) Temperature near the detectors (ºC) Interferometer input window temperature (ºC) Hot blackbody center temperature (ºC) Hot blackbody edge#1 temperature (ºC) Hot blackbody edge#2 temperature (ºC) Hot blackbody edge#3 temperature (ºC) Hot blackbody edge#4 temperature (ºC) Hot blackbody shroud temperature (ºC)

10 C&DH Software Data Content & Format Document No: BOM Date: 24 August Type Description Cold blackbody center temperature (ºC) Cold blackbody edge#1 temperature (ºC) Cold blackbody edge#2 temperature (ºC) Cold blackbody edge#3 temperature (ºC) Cold blackbody edge#4 temperature (ºC) Cold blackbody shroud temperature (ºC) Scan mirror temperature (ºC) Second input port blackbody PRT temperature reading (ºC) Second input port blackbody thermistor temperature reading (ºC) integer Scan mirror angle (10-bit count) integer Hot blackbody PRT1 (12-bit count) integer Hot blackbody PRT2 (12-bit count) integer Hot blackbody PRT3 (12-bit count) integer Hot blackbody PRT4 (12-bit count) integer Hot blackbody PRT5 (12-bit count) integer Hot blackbody PRT6 (12-bit count) integer Hot blackbody Ref1 (12-bit count) integer Hot blackbody Ref2 (12-bit count) integer Hot blackbody Ref3 (12-bit count) integer Cold blackbody PRT1 (12-bit count) integer Cold blackbody PRT2 (12-bit count) integer Cold blackbody PRT3 (12-bit count) integer Cold blackbody PRT4 (12-bit count) integer Cold blackbody PRT5 (12-bit count) integer Cold blackbody PRT6 (12-bit count) integer Cold blackbody PRT7 (12-bit count) integer Cold blackbody PRT8 (12-bit count) <Present only if data file version is "1.01"> integer Cold blackbody PRT9 (12-bit count) <Present only if data file version is "1.01"> integer Cold blackbody Ref1 (12-bit count) integer Cold blackbody Ref2 (12-bit count) integer Cold blackbody Ref3 (12-bit count) (DataSize) Interferogram data channel A (V). Data size depends on resolution setting (see Table 2 5). (DataSize) Interferogram data channel B (V). Data size depends on resolution setting (see Table 2 5).

11 C&DH Software Data Content & Format Document No: BOM Date: 24 August Table 2 5 below lists the size of the interferogram data which depends on the resolution setting, as read from the secondary header (cf. Table 2 3). Table 2 5 Interferogram data size as function of resolution settings. Resolution Setting Resolution (cm-1) Data Size

12 C&DH Software Data Content & Format Document No: BOM Date: 24 August FORMAT AND CONTENT OF HOUSEKEEPING DATA FILES The C&DH software records housekeeping data using a simple comma separated value ASCII text file format. This section describes how to access the data fields contained in such files. See Ref. 2 to know how to convert the data. 3.1 DETAILED HOUSEKEEPING DATA FORMAT AND CONTENT A housekeeping data set is collected and then written into the housekeeping data file as one line terminated by a carriage return. This process is repeated for each data set collected. CAUTION: The newline characters sequence must be converted when data are moved from Unix/Linuxlike to Windows-like platforms. FTP transfer tools usually perform such conversion tasks when moving text files from different kinds of platforms. The data fields are sequentially recorded to the text file, separated by a comma "," and the last data field is followed by a newline, such that successive data sets takes the form of successive lines into the text file. The content of a typical housekeeping data file is shown on Table 3 6. Some data fields are not displayed for the sake of the compactness of the illustration. Table 3 6 Housekeeping data file sample. 03/01/08,00:06:13:61,2476,0,3390,3404,...,3237, ,14.60,14.58,1,0,2,2,0,1,120.97,1,15.67,1 03/01/08,00:06:14:87,2476,0,3390,3404,...,3241, ,14.60,14.59,1,0,2,2,0,1,120.97,1,15.69, /01/08,00:08:11:13,2476,0,3390,3401,...,3243, ,15.23,15.20,1,0,2,2,0,1,120.97,1,16.65,1 03/01/08,00:08:12:37,2476,0,3390,3401,...,3243, ,15.23,15.20,1,0,2,2,0,1,120.97,1,16.66,1 Table 3 7 below lists housekeeping data fields that are written to a housekeeping data file. For example, from the sample file presented in Table 3 6, the data field P1 (scan mirror angle 10-bit counts) has recorded values of 2476 for each data set whereas the data field P35 (interferometer's input window temperature) has recorded values of 15.67, 15.69,..., and at GMT time stamps of 00:06:13:61, 00:06:14:87, 00:08:11:13, 00:08:12:37 respectively. All those data have been recorded on GMT March 1st, Table 3 7 Housekeeping data fields description. Field Format Description Date MM/DD/YY GMT date at the beginning of the housekeeping data collection where MM:month (1-12), DD:day (1-31), YY:year (00-99). Time HH:MM:SS.CC GMT time at the beginning of the housekeeping data collection where HH:hour (0-23), MM:minutes (0-59), SS:seconds (0-59), CC:1/100 seconds (0-99). P1 Decimal Scan mirror angle (10-bit count). P2 Decimal Unused. Always 0.

13 C&DH Software Data Content & Format Document No: BOM Date: 24 August Field Format Description P3 Decimal Hot BB PRT1 (center) (12-bit count). P4 Decimal Hot BB PRT2 (edge #1) (12-bit count). P5 Decimal Hot BB PRT3 (edge #2) (12-bit count). P6 Decimal Hot BB PRT4 (edge #3) (12-bit count). P7 Decimal Hot BB PRT5 (edge #4) (12-bit count). P8 Decimal Hot BB PRT6 (shroud) (12-bit count). P9 Decimal Hot BB Ref1 (reference resistor) (12-bit count). P10 Decimal Hot BB Ref2 (reference resistor) (12-bit count). P11 Decimal Hot BB Ref3 (reference resistor) (12-bit count). P12 Decimal Cold BB PRT1 (center) (12-bit count). P13 Decimal Cold BB PRT2 (edge #1) (12-bit count). P14 Decimal Cold BB PRT3 (edge #2) (12-bit count). P15 Decimal Cold BB PRT4 (edge #3) (12-bit count). P16 Decimal Cold BB PRT5 (edge #4) (12-bit count). P17 Decimal Cold BB PRT6 (shroud) (12-bit count). P18 Decimal Cold BB PRT7 (12-bit count). P19 Decimal Cold BB PRT8 (12-bit count). P20 Decimal Cold BB PRT9 (12-bit count). P21 Decimal Cold BB Ref1 (reference resistor) (12-bit count). P22 Decimal Cold BB Ref2 (reference resistor) (12-bit count). P23 Decimal Cold BB Ref3 (reference resistor) (12-bit count). P24 Float(x.xx) Sterling cooler voltage (millivolts). Can be converted to give the temperature near the detectors ships (InSb/MCT) in Kelvin with the formula: T = K K/V * P24 * 1e-3 V/mV. P25 Float(x.xx) 2nd input port BB PRT (ºC). P26 Float(x.xx) 2nd input port BB Thermistor (ºC). P27 Decimal Unused. Always 1. P28 Decimal Unused. Always 0. P29 Decimal MCT channel gain (1, 2, 4, 8 or 16). P30 Decimal InSb channel gain (1, 2, 4, 8 or 16). P31 Decimal Unused. Always 0. P32 Decimal Unused. Always 1. P33 Decimal Scan frequency (# of scans/minute, where one scan is two (2) interferometer's sweeps, forward/reverse).

14 C&DH Software Data Content & Format Document No: BOM Date: 24 August Field Format Description P34 Decimal Resolution (1, 2, 4, 8 or 16 cm-1). P35 Float(x.xx) Interferometer's input window temperature (ºC). P36 Decimal Pressure status (0:bad, 1:good). End of document