Important Notes on the Release of FTS SWIR Level 2 Data Products For General Users (Version 02.xx) June, 1, 2012 NIES GOSAT project 1. Differences of processing algorithm between SWIR L2 V01.xx and V02.xx The followings are the summary of the differences between SWIR L2 V01.xx and V02.xx. The last two digits in the SWIR L2 version name corresponds to the version of input TANSO-FTS L1B, and users don't care about them except for the case of last two digits indicating detailed specific versions. <INPUT DATA> TANSO-FTS L1B V141.141 is used to process SWIR L2 V02.00 product. It is a preliminary sample of TANSO-FTS L1B V150.150 and covers the observation period from June 1, 2009 to July 31, 2010. No additional observation period is available by TANSO-FTS L1B V141.141, however, it is planned to reprocess whole observation data (from April 2009 to present) by TANSO-FTS L1B V150.150. The version name of SWIR L2 from TANSO-FTS L1B V150.150 is V02.10. Detail about the differences in TANSO-FTS L1B V141.141 and V150.150 from the former versions are found in the document "Release Notes for GOSAT FTS L1 Products (Ver.141.141) 1) " and "Release Notes for GOSAT FTS L1 Products (Ver.150.150) 2) ", respectively. TANSO-CAI L1B V01.00 and TANSO-CAI L2 V01.00 are used for cloud detection scheme of the SWIR L2 V02.xx processing. Reprocessing of TANSO-CAI L1B V01.00 and TANSO-CAI L2 V01.00 are undergoing for whole observation data (from April 2009 to present). Detail about the differences in these product from the former versions are found in the document "On the Release of New Version 01.00 of TANSO-CAI L1B Data 3) " and "Notes on the use of CAI L2 Cloud Flag Product Ver.01.00 4) ", respectively. 1)https://data.gosat.nies.go.jp/GosatWebDds/productorder/distribution/user/ V141141_FTS_L1_ReleaseNote_en.pdf 2)https://data.gosat.nies.go.jp/GosatWebDds/productorder/distribution/user/ V150150_FTS_L1_ReleaseNote_en.pdf 3)https://data.gosat.nies.go.jp/GosatWebDds/productorder/distribution/user/V 0100_CAI_L1B_en.pdf - 1 -
4)https://data.gosat.nies.go.jp/GosatWebDds/productorder/distribution/user/ V0100_CAI L2_Cloud_Flag_ReleaseNote_en.pdf <GEOMETRIC CORRECTION> As reported in the document "Geolocation Error in TANSO-FTS L1B and its countermeasure 5) ", non-negligible geometric error was found through the calibration activities. The geometric error is modeled as a function of observation period and viewing angle and corrected in the pre-processing of the SWIR L2 V02.xx processing. See above document for more detail. As reported in the document "Important Notes on the Use of FTS SWIR Level 2 Data Products (Version 01.xx) for General Users 6) ", there existed a limitation and bugs in the setting of the TANSO-FTS IFOV area, especially for the specific-point observation data with the image-motion-compensation (IMC) function regarded as "not in operation". Because all observations have been made the "IMC in operation" after August, 2010, we made a complete scan list with "IMC not in operation" to avoid the miss-setting of the TANSO-FTS IFOV area. As a result, the TANSO-FTS IFOV area is properly set in the SWIR L2 V02.xx. 5) https://data.gosat.nies.go.jp/gosatwebdds/productorder/distribution/user/ Geolocation_error_in_FTS_L1B.pdf 6) https://data.gosat.nies.go.jp/gosatwebdds/productorder/distribution/user/ V01XX_FTS_L2_gu_en.pdf <RETRIEVAL SETTING & REFERENCE DATA> Differences in the state vector, pre-screening items, and post-screening items are summarized in the following tables. For more details, see "ALGORITHM THEORETICXAL BASIS DOCUMENT (ATBD) FOR CO2 and CH4 COLUMN AMOUNTS RETRIEVAL FROM GOSAT TANSO-FTS SWIR". Replace the solar irradiance database. There remained a terrestrial absorption structure in the previous solar irradiance database, which made air-mass dependent biases in XCO2 (0 ~ 6 ppm) and XCH4 (0 ~ +20 ppb). The aerosol composition at the TANSO-FTS measurement point to calculate the a priori aerosol single-scattering properties used in the SWIR L2 processing is assumed to be the same as the simulated results by the three-dimensional aerosol transport model SPRINTARS. In the SWIR L2 V01.xx processing, the - 2 -
retrieved aerosol parameter was an aerosol optical depth (AOD) at a reference wavelength (defined as 1.6 m); i.e., the wavelength structure of AOD normalized with the AOD at the reference wavelength was fixed. The contribution of coarse particles seemed dominant for SPRINTARS V3.54, which used in the SWIR L2 V01.xx processing. As a result, the retrieved AOD was overestimated, and it brought negative biases in the retrieved XCO2 and XCH4. The SWIR L2 V02.xx uses SPRINTARS V3.84, which gives more adequate aerosol optical properties. Further, the retrieved aerosol parameters are increased in the SWIR L2 V02.xx processing. In the SWIR L2 V02.xx processing, the SPRINTARS simulated aerosols are categorized into two-types of aerosols (i.e., fine-mode particles and coarse-mode particles), and their profiles are retrieved simultaneously with XCO2 and XCH4. The actual state vector elements (12 elements) are a logarithm of number concentration of aerosol for each layer (6 layers) and each type (2 types). Update the radiometric conversion factor for TANSO-FTS Band 1 responded to the non-linearity correction in the TANSO-FTS L1B V141.141 and V150.150. To reduce the non-linearity effect in the O2 A band spectrum, a zero-level offset term is added in the state vector in the SWIR L2 V02.xx processing. To reduce the surface pressure bias, a scaling factor of 1.01 is applied to the O2 A band absorption cross section. Except this scaling factor, the spectroscopic parameters for gaseous absorption are the same as those of SWIR L2 V01.xx. Add 2.0 m CO2 absorption band spectrum (TANSO-FTS Band 3; 4800-4900 cm -1 ) in the measurement vector. As a square-root of a diagonal element of the covariance matrix for the measured spectrum, SWIR L2 V01.xx processing used "true" noise evaluated from the out-of-band spectrum. Because the chi-squared increased with SNR, SWIR L2 V02.xx processing uses "empirical" noise to get a SNR-independent chi-squared. The empirical noise is expressed as a sum of the true noise and a scaling factor, and this scaling factor is a function of SNR. Added ILS interpolation. SWIR L2 V01.xx processing used single ILS for each sub-band; e.g., ILS@6280 cm -1 was used to calculate the forward spectrum from 6180 to 6380 cm -1. SWIR L2 V02.xx processing interpolates ILS linearly to each target wavenumber. Stop channel selection. The spectrum channels overlapped with the solar Fraunhofer lines were masked in the SWIR L2 V01.xx retrieval analysis. The - 3 -
SWIR L2 V02.xx processing uses all spectrum channels within the defined sub-bands. Modified pre- / post-screening items and criteria. - 4 -
Table 1. State vector for TANSO-FTS SWIR L2 retrieval. State Vector V01.xx V02.xx CO2 mixing ratio (15-layer) mixing ratio (15-layer) CH4 mixing ratio (15-layer) mixing ratio (15-layer) H2O mixing ratio (15-layer) mixing ratio (15-layer) aerosol AOD (1-layer, 1-type) logarithm of number conc. (6-layer, 2-type) surface pressure retrieved retrieved temperature profile bias retrieved retrieved wavenumber dispersion retrieved (3-sub band) retrieved (4 sub-band) surface albedo (land) retrieved (3-sub band) retrieved (4 sub-band) surface wind speed (ocean) retrieved retrieved radiance adjustment factor (ocean) retrieved not retrieved band 1 zero-level offset not retrieved retrieved L1B quality Cloud Table 2. Pre-screening items for TANSO-FTS SWIR L2 retrieval. Items Rejection Criteria V01.xx V02.xx L1B flags See Table 3 See Table 3 out-of-band characteristics Not Considered outlier scans > 0 % for area 1 CAI cloud flag > 10 % for area 3 (see Fig. 1) > 8 % for area 5 > 0 % for area 1 CAI coherent (ocean) satisfy Eq. (1) satisfy Eq. (1) FTS 2 m satisfy Eq. (2) or (3) satisfy Eq. (2) or (3) solar zenith angle > 70 deg. > 70 deg. surface undulation satisfy Eq. (4) or (5) satisfy Eq. (4) or (5) land fraction 0 < LF < 100 % 0 < LF < 60 % SNR < 100 for one of used sub-bands < 70 for O2 sub-band - 5 -
STD[CAI_band1] > 0.0125 * AVG[CAI_band1] - 0.4 (1) CAI_band1: TANSO-CAI band 1 radiance within the TANSO-FTS IFOV AVG[x], STD[x]: average and standard deviation of parameter x AVG[FTS_band3P_WVsat] / NOISE[FTS_band3P] > 1 (2) AVG[FTS_band3S_WVsat] / NOISE[FTS_band3S] > 1 (3) FTS_band3P_WVsat, FTS_badn3S_WVsat: TANSO-FTS band 3 radiance at the water vapor saturation channels NOISE[x]: measured noise of TANSO-FTS band x AVG[elevation_IFOV]-AVG[elevation_shiftedIFOV] > 25 m (4) STD[AVG[elevatino_IFOV]-AVG[elevation_shiftedIFOV]] > 25 m (5) elevation_ifov: DEM surface elevation within the TANSO-FTS IFOV elevation_shiftedifov: DEM surface elevation within the shifted TANSO-FTS IFOV Figure 1. Setting of the area used in the CAI cloud flag judgment. Left) Example for IMC in operation. Right) Example for IMC not in operation. - 6 -
Table 3. List of TANSO-FTS L1B data quality flags referred in the TANSO-FTS SWIR L2 processing. Corresponding paths are shown below this table. The contents for the SWIR L2 V02.xx processing are the completely same those for the SWIR L2 V01.xx processing. Data quality flags Path Rejection criteria missingflag 1 Partial loss (1), Full loss (2) pointingmirrorstableflag 1 Invalid exposure data (0), undeterminable exposure data (2) ELU_StableFlag 2 Unstable (0) pointingstableflag 2 Unstable (0) CT_obsPoints 2 Other than electronic calibration (0) Undefined (-1) AT_Ang_Error_Flag 2 exceeds +/- 0.1 degree (1) CT_Ang_Error_Flag 2 exceeds +/- 0.1 degree (1) ZPD_SaturationFlag_SWIR 2 Saturation in any of six SWIR bands (non-zero values) spikenoiseflag_swir 2 Sharp, abrupt noises in any of six SWIR bands (non-zero values) satelliteattitudestabilitystatusflag 3 Controlling attitude (1), possibility in controlling (2), judgment impossible (3) yawsteeringflag 3 Not execute (0) CenterLat 4 invalid value (-8192) CenterLon 4 invalid value (-8192) 1) /exposureattribute/pointattribute/ 2) /exposureattribute/pointattribute/radiometriccorrectioninfo/ 3) /exposureattribute/pointattribute/satellite/ 4) /exposureattribute/pointattribute/geometricinfo/ - 7 -
Table 4. Post-screening items for TANSO-FTS SWIR L2 retrieval. Items Rejection Criteria V01.xx V02.xx # of iteration > 20 > 20 chi-squared > 5 Not Considered residual mean-squared > 1.2, 1.2, 1.3, and 1.4 for > 3 for one of used O2, CO2, CH4, and SCO2 sub-bands sub-band, respectively DFS < 1 < 1 AOD > 0.5 > 0.1 radiance adjustment factor Edge Value 1) Not Considered surface wind speed Not Considered Edge Value 1) blended albedo Not Considered > 1 surface pressure Not Considered rtrv. - prior > 20 hpa 1) We set the maximum and minimum values for some state vector elements. When such elements reaches to the edge value (i.e., maximum or minimum values) during the retrieval, the scan was rejected. - 8 -
2. Items stored in the TANSO-FTS SWIR L2 data product Major changes in the product format are as follows. Table 5 shows a part of the Product Format Descriptions. For more detail, please check the full document of the Product Format Descriptions. Add "/ancillary/ftsl2datainformation/" to store the algorithm version dependent dataset. Add some dataset according to the geometric correction. Add state vector information. Add total pre-/post-screening results. Although these two datasets exists but are filled by invalid value for the SWIR L2 V02.00 product, they are corrected in the SWIR L2 V02.10 product. Table 5. Short descriptions of the TANSO-FTS SWIR L2 V02.xx data product. Bolded red datasets indicate something changed from those of the TANSO-FTS SWIR L2 V01.xx product. Group Path / Dataset Descriptions remarks scanattribute (observation information) numscan Number of FTS scans stored in the product scanid Scan ID number scandirection Direction of FTS scan (0: Reverse, 1: Forward) scanduration Duration of one FTS scan (in seconds) crosstrackobservationpoint Number of cross-track observation points (1, 3, 5, 7, 9, 10) time ZPD point passage time [UTC] sensor observationmode Observation mode (OB1D, OB2D, SPOD) sunglintmodeflag SPOD (0: specific-point observation mode, 1: sun-glint observation mode) Inaccurate IMCFlag IMC operation (0: not in operation, 1: in operation) gain Gain (H, M, L) alongtrackangle Along-track angle after geometric Change correction [deg.] crosstrackangle Cross-track angle after geometric Change correction [deg.] alongtrackangleoriginal Along-track angle before geometric correction [deg.] - 9 -
crosstrackangleoriginal Cross-track angle before geometric correction [deg.] referencedata (information referred for SWIR L2 processing) surfacepressure Surface pressure [hpa] (GPV data) temperatureprofile Temperature profile [K] (GPV data) watervaporprofile Water Vapor profile [ppmv] (GPV data) surfacealbedo Ground surface albedo (MODIS climatological values) surfacewindspeed Surface wind speed [m/s] (GPV data) aerosolopticalthickness Aerosol optical thickness (SPRINTARS) aerosolopticalthicknessprofile Profile of aerosol optical thickness (SPRINTARS) aerosolsinglescatteringalbedo Aerosol single-scattering albedo (SPRINTARS) aerosolasymmetryfactor Aerosol asymmetry factor (SPRINTARS) cirrusopticalthickness Optical thickness of cirrus cloud (CALIOP climatological value) cirrustoppressure Cloud-top pressure for cirrus [hpa] (CALIOP climatological value) XCO2 CO2/CH4 column averaged dry air XCH4 mole fraction (NIES TM) Normal with invalid value for water surface Change 4) 4) dryairpartialcolumn Dry air profile CO2Profile CH4Profile CO2/CH4 dry air mixing ratio profile variancecovariancematrix variance-covariance matrix for CO2/CH4 cloudinformation cloudpixelinnerfov Number of cloudy pixels within FTS IFOV AREA-1 of Fig. 1 cloudpixelouterfov Number of cloudy pixels within the extended FTS IFOV AREA-2 of Fig. 1 cloudpixelinnersunwardside Number of cloudy pixels in FTS IFOV projected toward the sun AREA-3 of Fig. 1 cloudpixeloutersunwardside Number of cloudy pixels in extended FTS IFOV projected toward the sun AREA-4 of Fig. 1 cloudpixelinnersatellitewardside Number of cloudy pixels in FTS IFOV projected toward the satellite AREA-5 of Fig. 1 cloudpixeloutersatellitewardside Number of cloudy pixels in extended AREA-6 of Fig. 1-10 -
FTS IFOV projected toward the satellite sunglintflag Indicating sun glint flags for CAI pixels within FTS IFOV qualityinformation (quality information) Signal-to-noise ratio of the retrieval SNR sub-bands (defined with maximum brightness and the noise level) L1BQA Quality of FTS Level 1B data (0: OK, 1: NG) Screening result for ground surface roughterrainsurfacescreening roughness within FTS IFOV (0: OK, 1: NG) highsolarzenithscreening Screening result for solar zenith angle (0: OK, 1: NG) highaltitudeaerosolscreening Screening result for high altitude aerosol (0: OK, 1: NG) TIRCloudScreening Screening result for clouds observed in FTS Band 4 (0: OK, 1: NG) Screening result for high altitude watersaturationbandscreening scattering substance observed in FTS band 3 (0: OK, 1: NG) CAIRadianceScreening Screening result for clouds (based on CAI radiance) (0: OK, 1: NG) totalprescreeningresult Overall pre-screening result (Table 2) (0: OK, 1: NG) CAIRadiance CAI radiance within the TANSO-FTS IFOV Type of a priori gas information (0: gasprofileaprioritype Predictions by NIES atmospheric transport model, 1: climatological value) Weight factors for aerosol a priori aerosolaprioriweight information obtained from CAI and SPRINTARS Data totalcolumn CO2TotalColumn CO2/CH4 Column amount CH4TotalColumn [molecules/cm 2 ] CO2TotalColumnSmoothingError Smoothing Error in CO2/CH4 column CH4TotalColumnSmoothingError amount [molecules/cm 2 ] CO2TotalColumnRetrievalNoise Retrieval Noise in CO2/CH4 column CH4TotalColumnRetrievalNoise amount [molecules/cm 2 ] Corrected Inaccurate Inaccurate Name is changed Inaccurate 1) - 11 -
CO2TotalColumnInterferenceError CH42TotalColumnInterferenceErro Interference Error in CO2/CH4 r column amount [molecules/cm 2 ] CO2TotalColumnExternalError External Error in CO2/CH4 column Normal with CH4TotalColumnExternalError amount [molecules/cm 2 ] invalid value mixingratio XCO2 XCH4 CO2/CH4 dry air mixing ratio [ppmv] XCO2SmoothingError XCH4SmoothingError Smoothing Error in CO2/CH4 dry air mixing ratio [ppmv] XCO2RetrievalNoise XCH4RetrievalNoise Retrieval Noise in CO2/CH4 dry air mixing ratio [ppmv] XCO2InterferenceError XCH4InterferenceError Interference Error in CO2/CH4 dry air mixing ratio [ppmv] XCO2ExternalError External Error in CO2/CH4 dry air Normal with XCH4ExternalError mixing ratio [ppmv] invalid value CO2Profile CO2/CH4 dry air mixing ratio profile CH4Profile [ppmv] retrievalquality Chi2 Chi squared for the retrieval result Root mean squares of the residual residualmeansquare spectra normalized with the noise level CO2DFS CH4DFS Degree of freedom for signal iterations Number of retrieval iterations errorcovariancematrix Error covariance matrix averagingkernelmatrix Averaging kernel matrix columnaveragingkernel Column averaging kernel totalpostscreeningresult Overall post-screening result (Table 4) (0: OK, 1: NG) Inaccurate 1) auxiliaryparameter surfacepressure Simultaneously retrieved surface pressure [hpa] prsretlevel Pressure level for retrieved state aerosolopticalthickness Simultaneously retrieved aerosol Change optical thickness Definition aerosolopticalthicknessprofile Profile of aerosol optical thickness for retrieved state aerosolsinglescatteringalbedo Aerosol single-scattering albedo for retrieved state - 12 -
aerosolasymmetryfactor Aerosol asymmetry factor for retrieved state temperatureshift Retrieved temperature shift surfacealbedo Retrieved surface albedo (only for land) surfacewindspeed Retrieved surface wind speed (only for ocean) zeroleveloffste Retrieved band 1 zero-level offset dryairtotalcolumn Dry air total column used for convert the total column amount to the mixing ratio dryairpartialcolumn Dry air profile geolocation Geodetic latitude at center of FTS latitude IFOV after geometric and ortho correction [deg.] Longitude at center of FTS IFOV longitude after geometric and ortho correction [deg.] Geodetic latitude at center of FTS latitudeoriginal IFOV before geometric and ortho correction [deg.] Longitude at center of FTS IFOV longitudeoriginal before geometric and ortho correction [deg.] Geodetic latitude of 36 observation footprintlatitude points around FTS IFOV after geometric and ortho correction [deg.] Longitude of 36 observation points footprintlongitude around FTS IFOV after geometric and ortho correction [deg.] Average altitude of ground surface height within FTS IFOV [m] (GDEM/GTOPO30) solarzenith Solar zenith angle at IFOV center (at ZPD point passage time) [deg.] solarazimuth Solar azimuth angle at IFOV center (at ZPD point passage time) [deg.] satellitezenith Satellite zenith angle at IFOV center (at ZPD point passage time) [deg.] satelliteazimuth Satellite azimuth angle at IFOV center (at ZPD point passage time) Change Change Change Change Change - 13 -
[deg.] satelliteattitude Satellite attitude (ECR coordinate system) at ZPD point passage time Satellite position (ECR coordinate satelliteposition system) [m] at ZPD point passage time landseamask Land/sea indicator (within FTS IFOV) (0: land, 1: water surface, 2: mixed) REMOVED landfraction land fraction within the TANSO-FTS IFOV [%] sunglintflag sunglint Flag for TANSO-FTS measurement (same as L1B flag) ancillary FTSL1BDataInformation FTSL1BGranuleID Granule ID of the FTS Level 1B data scansequencenumber The scan number given serially in the FTS Level 1B data FTSL2DataInformation numsubband number of sub-band wavenumberrangesubband wavenumber range of sub-band numreflevel number of level for reference data (GPV, NIES TM) prsreflevel pressure level for reference data numretlayer number of layer for retrieval numretalb number of wavenumber of retrieved albedo grid indxretalb index for retrieved albedo grid to sub-band wnretalb wavenumber of retrieved albedo grid numretaer number of wavelength to output the aerosol optical properties wavelengthretaer wavelength to output the aerosol optical properties 1) This dataset is inaccurate for the SWIR L2 V02.00 product. - 14 -