OPUS Science Data Processing

Science Data Processing www.stsci.edu/software/opus/ www.dpt.stsci.edu/ 1 of 13

OPUS Science Data Processing PMDB ASSIST PDB pod file Data Partitioning Support Schedule Data Validation EDT binary dataset EDT binary dataset + dgx EDT binary dataset, dgx + dvx check for bad segments insert fill data if telemetry dropouts extract proposal and scheduling information from PMDB dredge mnemonic values from telemetry and apply conversions if necessary compare actual vs. planned observation parameters World Coordinate System convert pointing informationinto standard FITS WCS keywords Keyword Database Keyword Exceptions Table Keyword Rules Generic Conversion Best Ref/ Switch EDT binary dataset, dgx, dvx + dcx uncalibrated FITS dataset uncalibrated FITS dataset convert data into FITS format properly orient images or spectra populate keyword values populate calibration reference file and switch keywords (also used by StarView/web) CDBS Data Collector hold individual exposures for association processing uncalibrated FITS dataset 2 of 13

OPUS Processes ΠData Partitioning segments the telemetry stream into standard EDT dataset appends multiple dumps into a single exposure file > uses an index file to avoid sorting packets > flags packet errors caught by PACOR ΠSupport Schedule gathers proposal information from PMDB test proposals required for development > test version of PMDB must be populated by TRANS 3 of 13

OPUS Processes (cont.) ΠData Validation decodes the exposure and engineering parameters in the telemetry and compares them to the planned values internal header specification (from Ball) > PDB (EUDL.DAT, TDFD.DAT) must be fully populated and defined in DM-06 [DONE] Planned value comparisons ( flags and indicators ) need to be determined by the Instrument Scientists [09/2002] ΠWorld Coordinate System implements a translation from telescope coordinates through the instrument light-path to an astronomically valid pointing aperture positions must be defined [09/2002] 4 of 13

OPUS Processes (cont.) ΠGeneric Conversion Generic Conversion outputs uncalibrated data > raw, asn, spt, trl data will be output in standard FITS format with image or table extensions primary header will contain keywords inherited by all extensions and a null data array Extensions > Images grouped by imsets consisting of science array, data quality array, and null error array. > data quality array will be null if no telemetry dropouts 5 of 13

OPUS Processes (cont.) ΠGeneric Conversion (cont.) Required for development > DM-06 to develop algorithms for data formatting [DONE] > keyword definitions (ICD-19) are currently being worked by the Instrument Scientists world coordinate definitions [09/02] calibration switches and selection criteria [09/02] calibration file name keywords [09/02] 6 of 13

OPUS Processes (cont.) ΠData Collector OPUS will ensure all component exposures are present before processing further association table contains information about the product dataset > dataset self-documenting only associations required for data processing will be constructed in the OPUS pipeline > CR splits > Repeat obs 7 of 13

Calibration ΠOPUS will use STSDAS calibration software Πrun on ALPHA / Tru64 UNIX platform in operations Πexpands size of dataset converts integer raw data to real possible expansion of data quality array. expansion of the error array. ΠNeed calibration reference files for testing (at least dummies) 8 of 13

Other Science Data Modes Πdata content & keywords for other science data modes must be defined by Instrument Scientists microprocessor memory dump [09/02] engineering diagnostic data [09/02] 9 of 13

CODE REUSE ΠCore OPUS system (OPUS 13.2) ~90,000 lines of code 100% reuse ΠWFC3 specific processes Based on FUSE study (Rose et al. 1998, OPUS: The FUSE Data Pipeline, www.stsci.edu/software/opus/kona2.html) > 5076 lines of code > 71% reuse of existing OPUS modules (ACS, NICMOS) ΠMajor effort in functional and throughput testing 10 of 13

Major Science Data Processing Requirements Summary ΠInternal header specification DM-06 to document content and format of science internal header [DONE] PDB (EUDL.DAT, TDFD.DAT) defined. [DONE] ΠKeyword definitions ΠPlanned values flags and indicators for Data Validation ΠAperture definitions [DONE] 11 of 13

Test Data Requirements ΠTest data from detectors on ops bench expected in March 2002 and from integrated instrument in June 2002 including all science and engineering modes ΠSMGT & IPT test data must include: PMDB population and PDB definition images that enable OPUS to determine proper orientation list of possible error conditions to simulate data that simulate those error conditions enough data for throughput test 12 of 13

Thermal Vac Support Œ.SDI files will trigger full dataset name: ipppssoot_csiiyydddhhmmss allows reuse of SMS Œ Optical Stimulus Snapshots named the same as the image file: csiiyydddhhmmss.oss allows corresponding OSC data to populate keywords Œ Support Schedule defaults since exposures will not have PMDB support, a default support schedule will be used. Auxilliary keyword values can be supplied in a separate exposure-specific ASCII file. 13 of 13