PDR.01.01: ASSUMPTIONS AND NON-CONFORMANCE

Transcription

1 PDR.0.0: ASSUMPTIONS AND NON-CONFORMANCE Document number Context MGT Revision Authors Ian Cooper, Ronald Nijboer Release Date Document Classification Status Draft Page of 6

2 Name Designation Affiliation Signature Owned by: Ian Cooper Acting Project Manager ICRAR Signature: Date: Ian Cooper Ian Cooper (Feb 9, 205) Approved by: Paul Alexander SDP Project Lead University of Cambridge Signature: Date: Paul Alexander Paul Alexander (Feb 9, 205) Version Date of Issue Prepared by Comments 0. 0/08/204 07:00:00 Ian Cooper, Ronald Nijboer ORGANISATION DETAILS Name Science Data Processor Consortium Page 2 of 6

3 Table of Contents Table of Contents LIST OF FIGURES LIST OF TABLES LIST OF ABBREVIATIONS... 7 Introduction Purpose of the document Scope of the document References Applicable documents Reference documents Assumptions Key assumptions for architectural decisions A.ARCH. Continued development of pipelines A.ARCH.2 There will exist regional centres Overall system sizing: assumptions made in order to produce cost estimates A.COST. Imaging at maximal rate A.COST.2 No bottleneck A.COST.3-8 Compute Load and Buffer Sizes High Priority Science Objectives A.SE. Rotation Measure Pipeline.... A.SE.2 Autocorrelation Data..... A.COST.9 Archive Size....2 Additional Observing Modes A.SE.3 Mosaicing A.SE.4 Drift Scan Other Assumptions A.COST.0 Distributed Software Development A.COST. Computational Efficiency A.COST.2 Moore s Law A.DELIV. Authentication and Authorisation Page 3 of 6

4 .3.5 A.DELIV.2 Regional Centres A.SKA2. SKA2 Parameters Calibration Strategy A.CAL. Ionospheric Effects A.CAL.2 Clock drifts - all telescopes A.CAL.3 Faraday rotation A.CAL.4 Electronic gains A.CAL.5 Complex Gains A.CAL.6 Tropospheric Effects A.CAL.7 Feed orientation A.CAL.8 Residual delays in the system A.CAL.9 X-Y Phase A.CAL.0 Time-dependent polarisation leakages A.CAL. Beam-specific gain/bandpass A.CAL.2 Antenna positions on the ground A.CAL.3 Pointing corrections A.CAL.4 Beam Shape A.CAL.4 Polarisation angle A.CAL.5 Calibration Observing modes Non-Conformance Issues Costing NC.COST. Power Cap NC.COST.2 Availability NC.COST.3 Off-site Archive (SKA-SYS_REQ-2350) High Priority Science NC.HPSO NC.HPSO NC.HPSO NC.ARCH. User accessible programming API to archive Page 4 of 6

5 Page 5 of 6

6 LIST OF FIGURES No table of figures entries found. LIST OF TABLES No table of figures entries found Page 6 of 6

7 LIST OF ABBREVIATIONS See PDR.6 Glossary of Terms Page 7 of 6

8 Introduction. Purpose of the document The purpose of this document is to capture the overall thinking behind the SDP PDR submission, which leverages the broad experience of the SDP Consortium members. The contents have been compiled from the following categories: Clarifications of interpretation of the requirements. Assumptions made in place of available requirements. Assumptions made in extrapolating requirements. Assumptions made where multiple requirements appear to be in conflict. Statements of prioritised requirements included/excluded from the design. Note: The term requirements in the above list, is to be considered in the broader sense, as to meaning provided data e.g. formal requirements, stated capabilities, guidance notes, documents released as early drafts, etc. The assumptions made here will help inform those areas of development which are on-going at system level and may lead to adoption as formal requirements (in appropriately modified form) if this is approach from the overall SKA system engineering perspective..2 Scope of the document This scope of this document is to state with as much clarity as possible, the assumptions, priorities, and non-conformances made in the SDP PDR submission, together with (if applicable) a brief justification of the associated benefits/impacts both within the SDP and the Telescope. Labelling: Assumptions and non-conformance have been labelled to identify them as either an assumption or a non-conformance (where non-conformance will take priority if the entity is both) and to point to the logical area explaining why they have been adopted. Type category A (Assumption) NC (Non-conformance) Reason Category ARCH (Adopted as a key element for COST (Adopted to enable costing for PDR) SE (Adopted to aid system engineering and the development of requirements and functions) CAL (Adopted to help work in the calibration work package proceed) DELIV (Adopted to enable work in the Data Delivery work package to proceed) Page 8 of 6

9 2 References 2. Applicable documents The following documents are applicable to the extent stated herein. In the event of conflict between the contents of the applicable documents and this document, the applicable documents shall take precedence. [AD0] PDR.0 Element Architecture Design Document [AD02] PDR.02.0 COMP - Element Subsystem Design Reports [AD03] PDR DATA - Element Subsystem Design Reports [AD04] PDR DELIV - Element Subsystem Design Reports [AD05] PDR LMC - Element Subsystem Design Reports [AD06] PDR PIP - Element Subsystem Design Reports [AD07] PDR.03 Requirements Analysis & Allocations [AD08] PDR.05 Parametric Models & Analysis Report [AD09] PDR.06 Preliminary Software Engineering Development Plan [AD0] PDR.07 Preliminary Cost Analysis [AD] PDR.08 Preliminary Plan for Construction [AD2] PDR.09 Configurations Items List [AD3] PDR.0 Compliance Matrix [AD4] PDR. Preliminary Integrated Logistic Support Plan [AD5] PDR.2 Preliminary Verification & Qualification Plan [AD6] PDR.3 High Level Risk Register [AD7] PDR.4 Development Plan [AD8] PDR.5 Analysis of System Scaling [AD9] PDR.0.03 SDP archive size estimates [AD20] PDR System sizing derived from outputs of the Parametric Model 2.2 Reference documents The following documents are referenced in this document. In the event of conflict between the contents of the referenced documents and this document, this document shall take precedence. [RD0] SKA-TEL-SKO-DD-00_BaselineDesign [RD02] L Requirements [RD03] Year on Telescope [RD04] SKA-TEL-SKO AG-OPS-UC-Rev_0-Technical_Use_Cases [RD05] SKA-TEL-SKO Telescope_CalibrationB (draft document) [RD06] A&A 527, A08 (20) Page 9 of 6

10 3 Assumptions 3. Key assumptions for architectural decisions 3.. A.ARCH. Continued development of pipelines Experience at existing facilities clearly indicates that there will be an ongoing need to develop pipeline software components throughout the operational lifetime of the instrument. This will be especially significant during the early operations when the characteristics of the instrument are fully established and both changes and additions to the pipelines are required in order to accommodate the new knowledge obtained during operation of the instrument. Further we can expect improvements, due to continuing and probably accelerating research, into imaging, time series and other automated processing approaches. We therefore take as a key architectural assumption in PDR.0 that it will be necessary to facilitate this with staff who are not leading HPC experts and therefore the system architecture should have a partitioning of the system to allow inclusion of developments without breaking the overall scalability A.ARCH.2 There will exist regional centres We take as a key assumption that there will exist regional centres and that this will be where users are provided with the means of direct access to data and the ability to develop and implement their algorithmic analysis of data products produced by the SKA. See A.DELIV.2 for further assumptions relating to regional centres. 3.2 Overall system sizing: assumptions made in order to produce cost estimates 3.2. A.COST. Imaging at maximal rate We present costs in this documentation set which are based on an analysis of the system performance required to run at a maximal rate (i.e. imaging at maximal spatial and spectral resolution) for each telescope A.COST.2 No bottleneck We assume that the system must process the data in the same length of time as was required to collect the data A.COST.3-8 Compute Load and Buffer Sizes In AD20: PDR03.04: System sizing derived from outputs of the Parametric Model we describe the process by which we have derived the compute load capability of the SDP systems for each instrument. There are very many assumptions adopted to do this: they are laid out in AD20 (please refer to tables -5 of this document). The major assumptions flowing out of this work and into the cost model are the overall (delivered) FLOP rate of each system and the size of the buffers. These are given in table 5 of AD20, but we reproduce them here for the interested reader Page 0 of 6

11 SKA-Low SKA-Mid (B) SKA-Survey (B) Total Performance requirements, maximal imaging case Total Compute load (PetaFLOPs) 25 A.COST.3 52 A.COST.4 72 A.COST.5 Visibility Buffer Size (PetaByte) 256 A.COST A.COST A.COST High Priority Science Objectives The High Priority Science Objectives (RD03) have NOT been used to size or cost the system but they have been taken as a reference set against which to check our requirements and functional breakdown. The results of these comparisons are presented in (REF PDR03.02 and PDR and PDR ). To accommodate the HPSOs we need to further assume two additional requirements, which are supported by Engineering Change Proposals at SKAO: 3.3. A.SE. Rotation Measure Pipeline There shall be a rotation measure pipeline. This is in our functional breakdown and requirements (as a placeholder, as we are still without a formal requirement from SKAO). See SKAO ECP A.SE.2 Autocorrelation Data The SDP shall be able to process autocorrelation data (SKAO ECP4000). We have not yet implemented this in the functional breakdown or requirements but we flag it up here as it will require changes to the system engineering. We have determined that our overall architecture will support this observational requirement A.COST.9 Archive Size We assume that the SDP Archives at both sites are identical for the time being, with an initial archive of 500PBytes, and annual growth of 30PBytes per site; this give a total archive size of 50PBytes at each site for 5years operation. Details of the analysis and justification for these numbers are laid out in AD9 (PDR0.03); they are partially based on the archive requirements for the High Priority Science Objectives, with a significant amount of additional reasoning Page of 6

12 3.5 Additional Observing Modes There are some observing modes that we tacitly assume to be necessary in a telescope but which are not covered by formal L requirements: 3.5. A.SE.3 Mosaicing The SDP will support mosaicing, as this is a common observing mode A.SE.4 Drift Scan The SDP will support using the telescope in drift-scan mode. 3.6 Other Assumptions 3.6. A.COST.0 Distributed Software Development The SDP software development will be done by a distributed team. This is an assumption required to have a firm basis of cost estimation: costs could reduce on another assumed model A.COST. Computational Efficiency The SDP computer will achieve a 25% computational efficiency. We apply this efficiency value uniformly to the system, so, we need to build four times more apparent FLOPS capability than the maximal imaging load requires A.COST.2 Moore s Law Moore s law scaling holds A.DELIV. Authentication and Authorisation It is not clear at this time where authentication and authorisation information will come from, but for now it is assumed that the delivery platform will receive what it needs from LMC. This will include lists of trusted Certificate Authorities (CAs) and Identity Providers (IdPs) A.DELIV.2 Regional Centres We assume as above that Regional Centres exist, and have the following properties: Will ultimately provide most of the astronomer services, to reduce the need to use SDP compute and network resources. Run the delivery platform software stack to enable high performance data transfers. Have network links tuned for high performance data transfers from the main SDP sites. Can serve as offsite backup if required A.SKA2. SKA2 Parameters We have been asked to produce an analysis of the system scaling to SKA2, but SKA2 is itself not defined anywhere. We have collated information from various sources to make our own definition of SKA2 which we hope will still enable useful scaling conclusions to be drawn. The assumptions we adopt are presented in AD8 (PDR5) Page 2 of 6

13 3.7 Calibration Strategy The Calibration Framework document, produced by the SKAO has only become available as of 26 January 205 (0 working days before the PDR submission deadline) as a document for review and comment. It has therefore been necessary to make assumptions about the calibration strategy which we have not been able to resolve (if needed) against the system-level Calibration Framework.. The effects we currently assume that the SDP will need to calibrate for are listed below. We have used this in the development of our calibration description, as presented in AD06 (PDR02.05, subelement design report for Pipelines) Section 9.4. These are not currently traced through into requirements, as they would appear at quite a low level, but this is work to be done A.CAL. Ionospheric Effects This affects all three telescopes, at frequencies of less than ~.5GHz. Rapid phase changes can have significant impact. We may have to accept some downtime for LOW due to severe ionospheric scintillation (simply not calibratable due to lack of calibration sources). LOFAR use phase screen fitting / rubber sheet model with an update rate of ~0 seconds using the brightest sources in the FoV to mitigate this. Calibration for the effects of the ionosphere requires a global solution (especially if dealing with Kolmogorov turbulence) A.CAL.2 Clock drifts - all telescopes It shall be necessary to calibrate for clock drifts A.CAL.3 Faraday rotation This affects all three telescopes, at frequencies of less than ~.5GHz. Effects can be seen in a bright highly polarized source at.4 GHz. In LOFAR, we also see differential Faraday rotation on unpolarised sources on baselines of order 0 km and longer. This can actually be used to deduce differential TEC (Total Electron Content) values to build the ionospheric model. In SKA this should be done in real time. At present this needs to be done post-observation by application of GPS-based corrections; however, GPS alone is insufficiently accurate, according to our LOFAR and MWA experience. GPS can be used for supporting data, but this also requires use of a Earth magnetic field model. A global solver is needed to do these calculations A.CAL.4 Electronic gains This affects all three telescopes. Electronic gain variations can lead to beam distortions (first and second order effects are mis-pointing and beam elongation in a certain direction) in Phased Array Feed (PAF) systems and Aperture Array (AA) stations; for Single Pixel Feeds this can be seen as a telescope based gain effect. We may require station calibration for AA stations and PAFs to mitigate impact on station / dish beam. However, there is no need to communicate across compute nodes, as it is direction independent A.CAL.5 Complex Gains This affects all three telescopes, but it seems strongly related to electronic gains, and may be solvable in similar ways Page 3 of 6

14 3.7.6 A.CAL.6 Tropospheric Effects This affects MID and SURVEY instruments at more than ~.4GHz. Effects can be severe, during bad weather or intermittent clouds - at least snow is unlikely to be a problem at SKA sites. We can use self-calibration plus correction for direction dependent effects A.CAL.7 Feed orientation This affects all three telescopes. We may be able to do this once, and offline (i.e. during a specifically scheduled calibration experiment) A.CAL.8 Residual delays in the system This affects all three telescopes. In order to calibration, we would observe a strong source or field, measure delays per baseline (initially via lag spectrum, then refine via fitting the phase slope), and then decompose this into an antenna-based delay. These delays are assumed to be stable, but they might change with power cycles and fiddling with the timing reference. In theory, bandpass calibration takes care of it. In practice, these residual delays may be large enough to preclude reliable bandpass solution by traditional algorithms and have to be taken care of separately. This is essentially an operation-specific calibration which is done offline (i.e. not during the scientific observations) A.CAL.9 X-Y Phase This affects all three telescopes. It is assumed to be constant in time, but it would be good to verify this A.CAL.0 Time-dependent polarisation leakages This affects all three telescopes. It affects the Primary beam, but it could include other, less dominant, effects as well A.CAL. Beam-specific gain/bandpass This affects the LOW and SURVEY instruments. This is the same as normal gain/bandpass calibration, but per beam. These are more stable in time than antenna-based gains/bandpasses (i.e. all beams change in a similar way with stable differences between beams) A.CAL.2 Antenna positions on the ground This affects all telescopes. For ASKAP this has been done via analysis of the ordinary time-dependent gain calibration solution in combination with specially crafted observations. So no specific real time code was required. Not done in observations. We will investigate whether this can be done once and offline (i.e. during a specific observation designed with this in mind, and not a part of the routine calibration required on a regular basis during or in between science observations) A.CAL.3 Pointing corrections This affects all telescopes. This needs to be done online (i.e. with solutions generated at intervals shorter than the observation length and as part of the standard calibration pipeline) - as was shown by the work on the Westerbork Wobble (see work by Smirnov, RD06). It can be dealt with using calibration of direction dependent effects, and is part of the beam calibration Page 4 of 6

15 3.7.4 A.CAL.4 Beam Shape This affects all telescopes. For SDP, this concerns the shape of the station beam for LOW, the shape of the compound beams for SURVEY and the shape of the primary beam for MID. Beam shape includes pointing errors as well as beam shape distortions. It requires direction dependent calibration, with typical timescales (based on the Westerbork Synthesis Radio Telescope and LOFAR) of ~0-30 minutes. Part of the beam calibration A.CAL.4 Polarisation angle Stokes V offsets. Part of the beam calibration. 3.8 A.CAL.5 Calibration Observing modes. The SDP will support modes used to calibrate the telescope. 4 Non-Conformance Issues 4. Costing 4.. NC.COST. Power Cap We size and cost the system based on required performance not to match the power cap. For SDP these two requirements are potentially conflicting and beyond the scope of design for the SDP element to resolve NC.COST.2 Availability We do not design for the specified availability budget. It is unrealistic and beyond the achievable requirements of an HPC or Data Centre facility. In order to be fully redundant we would need a far larger budget. Our preliminary analysis indicates that a significantly reduced formal availability will not impact science delivery unduly and we introduce into our architecture the concept of precious and non-precious data (see PDR.0 and PDR.0.02). Further development we will explore trade-offs in redundancy and resilience strategies in order to maximise availability and work with the SKAO system engineering team to define appropriate availability requirements NC.COST.3 Off-site Archive (SKA-SYS_REQ-2350) We do not cost an off-site secure location for a tape archive. We believe the requirements for a backup of the archive are met by (a) the existence of archives at both SKA sites which will mirror each other and (b) that regional centres will receive copies of the data. 4.2 High Priority Science The HPSOs contradict two requirements, namely SKA-SYS_REQ-2339 and SKA-SYS_REQ-234, which state that the maximum integration time for the Continuum and Spectral line pipelines respectively is 000hrs. We have continued to work to the 000hrs figure, essentially ignoring the HPSO request. We therefore identify two NC entities: 4.2. NC.HPSO. HPSO 5 requires 2500hrs on a spectral line observation Page 5 of 6

16 4.2.2 NC.HPSO.2 HPSO 37 requires 2000 hours for a continuum observation NC.HPSO.3 The HPSO experiments and 2 proposing to conduct experiments on high-redshift Hi from the Epoch of Reionisation all request that uv data be stored in the archive in addition to images. This request has been partially ignored: archive sizing has proceeded in a more general manner (as described in PDR03.02). Whilst some of the uv data for the EoR experiment could be stored within the proposed archive, not all of it can be. 6.3 Architecture NC.ARCH. User accessible programming API to archive We do not propose supporting a user-accessible API to the archive as the architecture does not support running user-supplied code for efficiency reasons. A full virtual observatory interface is supported and data are pushed to regional centres Page 6 of 6

17 PDR0-0AssumptionsandNonConf () EchoSign Document History February 09, 205 Created: February 09, 205 By: Status: Transaction ID: Verity Allan SIGNED XJE6UL76D7K3P2K PDR0-0AssumptionsandNonConformances () History Document created by Verity Allan February 09, 205-0:45 AM GMT - IP address: Document ed to Ian Cooper (ian.cooper@uwa.edu.au) for signature February 09, 205-0:45 AM GMT Document viewed by Ian Cooper (ian.cooper@uwa.edu.au) February 09, :7 AM GMT - IP address: Document e-signed by Ian Cooper (ian.cooper@uwa.edu.au) Signature Date: February 09, :8 AM GMT - Time Source: server - IP address: Document ed to Paul Alexander (pa@mrao.cam.ac.uk) for signature February 09, :8 AM GMT Document viewed by Paul Alexander (pa@mrao.cam.ac.uk) February 09, :3 AM GMT - IP address: Document e-signed by Paul Alexander (pa@mrao.cam.ac.uk) Signature Date: February 09, :3 AM GMT - Time Source: server - IP address: Signed document ed to Ian Cooper (ian.cooper@uwa.edu.au), Verity Allan (vla22@mrao.cam.ac.uk) and Paul Alexander (pa@mrao.cam.ac.uk) February 09, :3 AM GMT