CRaTER SOC Operations

Transcription

1 CRaTER SOC Operations LRO MOR Section 10 Mr. Michael Golightly CRaTER Deputy Project Scientist Boston University LRO Mission Operations Review (MOR) Slide - 1

2 CRaTER SOC Overview MOC R/T TCP Realtime archive SSH Housekeeping monitors MOC DMP SCP Raw archive PDIST Secure web server MOC CMD PDS PPI node SCP SCP Uplink archive Archive for PDS Pipeline Processing Pipeline archive SCP Archive mirror firewall LRO Mission Operations Review (MOR) Slide - 2

3 CRaTER Software Component Status Component C++ Object library CRaTER Pipeline Higher level products Function Object-oriented interface to Level 0, 1, and 2 data products Time ordering and gap detection in raw (EDR) files; Generation of Level 0, 1, and 2 products Model-dependent generation of LET spectra Status 100% written Released 100% written Released 0% written Data distribution Engineering display Science display Private web interface Public web interface TCP/IP packet distribution server and clients TCL/Tk graphical interface to display and analyze secondary instrument science and housekeeping IDL graphical interface to display and analyze primary and secondary science data products HTTPS/SSL server to provide secure access to CRaTER data products and processing information HTTP server to provide access to CRaTER status 100% written Under review 100% written Under review 90% written 10% written 50% written LRO Mission Operations Review (MOR) Slide - 3

4 CRaTER SOC Functions Uplink command generation (SCP to MOC, as needed) Continuous real-time data collection (TCP from MOC) and display Daily receipt of recorded data (SCP from MOC) Daily pipeline processing Daily quick-look analysis and trending; examine data gaps for existence of anomalous data loss Daily report to SOC lead and MOC Daily distribution to non-secure hosts (CRaTER internal server) Daily review of system logs for evidence of attempted intrusions Daily/weekly data back-ups to offline storage Weekly SOC team meetings Weekly processing & trending report and security inspection Monthly archive collection and rearrangement 3-monthly archive generation and transfer to PDS Yearly security analysis and report Anomaly response, e.g., replication on engineering unit or flight unit #2 LRO Mission Operations Review (MOR) Slide - 4

5 CRaTER SOC Organization Principal Investigator (Spence) Project Manager 1 (Foster) Project Scientist (Kasper) Deputy Project Scientist (Golightly) SOC Lead 2 (Ford) Project Engineer 3 (Goeke) Calibration algorithms Conversion algorithms Science display On-orbit commanding Pipeline software Document preparation LDWG/PDS Interfaces Archive generation IT security and hardware maintenance GSE control S/W Engineering display Document reviews Code reviews 1 MIT project management ends Dec Golightly takes over as SOC Lead Jan Support reduced to 25% beginning Jan 2008 LRO Mission Operations Review (MOR) Slide - 5

6 CRaTER SOC Staffing FY08 FY09 Principal Investigator Proj. Scientist, Science Lead Deputy PS, SOC Lead Temporary Soc Lead Instrument Scientist Operations Lead Science Software Spence Kasper Golightly Ford Case Bradford Wilson O N D J F M A M J J A S O N D J F M A M J J A Student Student SOC Operations S/W Development/Maintenance Integration/Test Science Algorithms Science Algorithm Improvements/Evolution LRO Mission Operations Review (MOR) Slide - 6

7 CRaTER Key Documentation Status Document Description/Purpose Need Date Current Status SOC Requirements Document Plans for SOC component verification 12/01/2006 Rev B: 11/02/2006 Delivered / signed off CRaTER SOC to PDS PPI Node ICD Defines the process of data transfer from SOC to PDS 12/01/2006 Rev B 11/21/2006 Delivered / signed off Data Management and Archive Plan Describes the generation, format, and archival of CRaTER pipeline products 12/01/2006 Rev A: 10/25/2006 Delivered / signed off SOC Risk Assessment Identifies security risks and their proposed mitigations 04/30/2007 Rev B: 06/01/2007 Delivered / signed off IT Security Plan and Contingency Plan Describes the controls to apply to minimize IT risks 06/30/2007 Rev A: 07/01/2007 Delivered / signed off SOC Test Plan Describes IT testing methodology and test suites traceable from IT requirements 10/30/2007 Rev 01: 08/31/2007 Draft released Standard Software Interface Specification Describes the raw (EDR) and pipeline (CDR) products and PDS archive organization 05/31/2007 Rev A: 05/31/2007 Rev B: under review Calibration Plan Describes how detector counts are converted to energy and lineal energy transfer 10/01/2007 Rev 01: 08/12/2007 Draft released Special Software Interface Specification Describes the Level 3 and 4 spectral data products 02/29/2008 Not started LRO Mission Operations Review (MOR) Slide - 7

8 CRaTER SOC Hardware Facilities To backup server BU NETWORK console switch Logger Crater-Devel SOC-A mux RAID 15x500 GB UPS SOC-B Crater-A 2x500 GB Locked cabinet in key-coded room Crater-B 2x500 GB CRaTER LAN (BU) Recorder Printer A CRaTER Team Member s LAN Printer Secure LAN Semi-open LANs External networks Power supply Disk access Multiplexed consoles LRO Mission Operations Review (MOR) Slide - 8

9 CRaTER Engineering Unit The SOC will maintain a CRaTER engineering unit and associated 1553 interface (S/C simulator) to assist in the diagnosis of post-delivery problems with the flight unit. SOC staff will interact with the simulator via a UDP interface to the following set of software programs Graphical command interface Output capture to disk Analog housekeeping display User status display Primary science stats display Output conversion to ASCII CRaTER Engineering Unit 2 x 1553, 1 Hz tick 1553B-UDP Emulator Internet (UDP) Example: send commands to the engineering unit and monitor the resulting housekeeping channels [1] Start the graphical command interface: host# CCmd & 28VDC [2] Start the graphical housekeeping interface host# CHouse & the CRaTER flight spare can also be used in place of the engineering unit, e.g., to assist in flight-unit recalibration LRO Mission Operations Review (MOR) Slide - 9

10 CRaTER Software Maintenance Operating system maintenance SOC-A, SOC-B, Crater-A, Crater-B, and Crater-Devel Redhat Linux updates provided under 3-year maintenance contract with Redhat SOC Operations Lead responsible for applying updates/patches Pipeline code development/modifications Through start of instrument I&T: MIT/Ford, BU/Bradford After start of instrument I&T: BU/Bradford, BU/Wilson Configuration Control Code version control maintained under Subversion version control system ( Copies of all implemented version will be maintained under version control in case it is necessary to roll-back to an earlier version System stability/integrity carefully controlled during all upgrades Code development/testing done outside SOC on Crater-Devel User manuals, installation guides, code reference manuals, etc must be updated and released with new versions/changes to existing software Changes to operating system, pipeline, or science software implemented in sequence of distinct steps Install changes/updates on Crater-Devel--test for compatibility, stability, & functionality; Install on the appropriate backup machine (e.g., machine-b if machine-a prime, etc)--test for compatibility, stability, & functionality; Switch backup machine to prime, and prime to backup (e.g., machine-b! prime, machine-a! backup) Install on remaining machine--test for compatibility, stability, & functionality. NO FLIGHT INSTRUMENT SOFTWARE LRO Mission Operations Review (MOR) Slide - 10

11 CRaTER Software Testing The Test Plan is directly derived from the following documents: CRaTER SOC Requirements Document CRaTER IT Security and Contingency Plan subject to the constraints identified in the following: CRaTER Data Management and Archive Plan CRaTER Standard Product Software Interface Specification CRaTER Science Team and the PDS PPI Node ICD 431 ICD External Systems ICD for the LRO Ground System The Traceability Matrix identifies 40 separate requirements: 13 functional the basic operations expected from the SOC 10 design requirements imposed by external constraints 10 performance ability of the s/w and human interfaces to do the job 7 security factors that protect the system from unintended behavior which are verified by a set of 10 tests, using various methodologies: 5 by explicit test did the system pass the prescribed procedural test? 3 by demonstration did the system produce the expected results? 1 by analysis are the algorithms adequate to the task? 1 by inspection are the planned resources available? LRO Mission Operations Review (MOR) Slide - 11

12 CRaTER SOC Development Status Current Status Hardware (5 processors, RAID storage, etc.) is being assembled Documentation deliveries are up-to-date Pipeline Software (Levels 0, 1, 2) is written and is under review Standard product SIS is under review by the PDS PPI node Engineering Unit interface is built and tested Draft SOC test plan is out; individual test scripts are being written Draft calibration plan has been released SOC is fully staffed (except for students) Future Tasks Place SOC software under configuration control Erect network firewall around secure SOC hardware Complete SOC testing, including SOC-MOC and SOC-PDS tests Conduct SIS peer review by PDS PPI node LRO Mission Operations Review (MOR) Slide - 12

13 Backup Charts LRO Mission Operations Review (MOR) Slide - 13

14 CRaTER SOC Hardware Facilities SOC computers SOC-A Host data pipeline process Receive data pushed from MOC Process data into specified L0, L1, L2 (, L3, L4) products Push processed data to RAID storage, PDS, and Crater computers CONNECTIVITY IS ONE WAY will not be able to access SOC machines except via computer s console in controlled access facility A & B machines mirror each other for redundancy Crater computers Crater-B Receive CRaTER data products Serves data for public and internal web sites, nominal displays, team scientists applications, etc (link between CRaTER data stores and anything outside of the SOC firewall) No connectivity from Crater computers back to the SOCs A & B machines mirror each other for redundancy LRO Mission Operations Review (MOR) Slide - 14

15 CRaTER SOC Hardware Facilities To backup server BU NETWORK SOC-A RAID 15x500 GB SOC-B switch Crater-A Crater-B 2x500 GB 2x500 GB Crater-Devel CRaTER LAN (BU) Recorder Printer A CRaTER Team Member s LAN Printer Secure MOC SOC Ingest Connection SOC Crater, PDS Push Connection External networks Disk Access CRaTER Data Via Restricted Access Network Connections LRO Mission Operations Review (MOR) Slide - 15

16 CRaTER SOC Hardware Facilities SOC RAID Storage RAID storage selected for short-term reliability derived both from the SOC requirements and risk assessment plans prevent a disk I/O error from crashing SOC-A/B while receiving real-time data from MOC Sized for 4 years of data storage Uncompressed Level 0, 1, and 2 products! 789 GB/y Serves as backup to CRaTER PDS data submission LRO Mission Operations Review (MOR) Slide - 16