Heliophysics Python Plan

Transcription

1 Heliophysics Python Plan Russell A. Stoneback Center for Space Sciences Photon Audio Lab The University of Texas at Dallas 27 July 2018 Snakes on a Spaceship: The Return of the Python

2 Motivation Developing for a community is going to be different than a single package Before we decide upon packages to standardize upon we should form a plan on what kind of structure, support, and features we need The overall community package structure needs to account for the realities of community development 2

3 Overall Plan Hub and spoke topology Individual packages implement bulk of functionality Installable via python setup.py install and pip install package Imported by higher level packages to mediate between packages and enable emergent functions Repeat as needed No dependency issues Supports broad package and feature base Individual packages are free to do as they do Integration packages require a more cohesive approach 3

4 Data Layer 4

5 Data Layer File formats should be transparent Transparent install Transparent use Individual package for each format netcdf (netcdf4) hdf4/5 (pyhdf, h5py) CDF (pysatcdf, CDF in spacepy, pure python CDF) ASCII (S. Morley proposed format) Higher level package Integrates sub packages to ensure data from all types returned in a consistent manner Data and metadata Dict keyed by variable name, meta dict keyed by variable name Not everyone understands or cares about self describing data formats High level package should take care of the details so the user only needs to present a filename and in return they get data and metadata ready to go (automagic) 5

6 SpaceTime Layer 6

7 SpaceTime Layer Part of making data consistent is good coordinate transformation and time support Time Support GPS and UTC time (TickTock - spacepy, spicepy,?) UNIX Time Julian Time (?) Easily updatable Pandas Index compatible (pandas and xarray) Coordinate transformations Geographic, geodetic, earth centered earth fixed, earth centered inertial, etc. (spacepy has some support) Magnetic coordinates (apexpy, aacgmv2) 7

8 Vector Systems Need to be able to express vectors in multiple basis systems consistently (pysatmagvect) East, North, Up Vectors Geographic and geodetic systems ECEF Vectors Geomagnetic Vectors Quaternion and Euler Matrix rotation support 8

9 Models 9

10 Indices 10

11 Models Pyglow and others currently provide access to several space science models Could implement same hub and spoke topology here By sub-discipline (?) Each model has an individual package Easier to express all model features Higher level package would then pull all the packages together Model use is application specific, may not be a larger organizing force here Alternatively, could provide a simple interface to run everything everywhere Community has asked for individual IGRF support so that they, and everyone else, can easily hook into IGRF Packages don t have to be fancy Provides a single source for the community to get IGRF values Indices could be a single package 11

12 Instrument Data Integration Common ground for all space science instrumentation is required pysat provides a common ground for data sets Supports integration of a wide variety of packages with no required changes on package side Already incorporates a range of packages Supports satellite and ground based instrument data Generalized data and metadata support, plus Current release uses pandas for data layer, pandas and initial xarray support in a github branch Implements all stages of space science data analysis 12

13 Instrument Data Integration 13

14 Distribution Layer Make writing data files as transparent as reading pysat supports this process using netcdf4 Consistent with NASA/ISTP metadata standards and NASA CDAWeb Generalizable to other formats using the data layer functionality 14

15 Applications Layer Use lower level packages for good One Example via pysat - Satellite Attitude and Ephemeris Support NASA CubeSat missions via ground station software In progress as part of NASA SPORT Predict satellite locations (sgp4 and pyephem) Couples with atmosphere models (IRI, MSIS, HWM, IGRF) Couples with magnetic coordinates (apexpy, aacgmv2) Supports projecting geophysical vectors onto spacecraft or instrument frames (pysatmagvect) Future Contact time planning Automatic TLE updates Ion drift model integration Space weather model development (plasma bubble model) 15

16 Application Examples Started as part of NASA/INPE (Brazilian NASA) SPORT CubeSat 16

17 Application Examples Started as part of NASA/INPE (Brazilian NASA) SPORT CubeSat 16

18 Application Examples Started as part of NASA/INPE (Brazilian NASA) SPORT CubeSat 16

19 Application Examples 17

20 Application Examples 18

21 Applications Layer Ground Station Support Operational Support Exploit ground based instrumentation 19

22 Analysis Layer pysat seasonal bin averaging Supports bin averaging any type of data Couple outputs into machine learning models Couple outputs into Data Assimilation models Superposed Epoch (spacepy) Occurrence Probability (pysat) orbit-by-orbit analysis (pysat) Produce Movies General plotting Basic maps in each coordinate system Orbit Visualisation Ground station maps with FOV 20

23 GUI Layer Need a package to make it easy for the public/developers to browse science data This can be coupled in general to pysat providing the generalized data access and modification tools needed for GUI foundation pysat will also provide triggers suitable for GUI Enables users to use all the same tools at the command line or in the GUI Applets - Focused GUI Apps Looking at seasonal averages has different UI requirements than iterating by orbit 21

24 License and Publications Adopt a license that preserves the openness of science Encourage publication in open source journals Must also be reasonable as open source publishing costs extra money Need to provide community with options Blockchain publishing Current journal system is not intrinsic to science Public can t generally read papers under current model Bitcoin for science Homegrown solution (?) 22

25 Publication Updates Cover article JGR June 2018 Community python article submitted (A. Burrell et al) Keep the momentum going with a solid development plan and robust software manuscript submitted to JGR-Space Physics Volume 123 Issue 6 June 2018 Pages Snakes on a Spaceship An Overview of Python in Heliophysics 3 4 A.G. Burrell 1,J.Klenzing 2, A. Halford 3, R.A. Stoneback 1, A.C. Kellerman 4, A.M. Annex 5,J.Ma 6, D. Stansby 7, K.M. Laundal 8, S. K. Morley William B. Hanson Center for Space Sciences, The University of Texas at Dallas, Richardson, TX, USA 2 ITM Physics Laboratory / Code 675, Goddard Space Flight Center Greenbelt, MD, Space Sciences Department, The Aerospace Corporation, Chantilly, VA 4 Department of Earth Planetary and Space Sciences, University of California, Los Angeles, CA 5 Department of Earth and Planetary Sciences, Johns Hopkins University, Baltimore, MD, Facebook AI Research, Menlo Park, CA Imperial College London, London, SW7 2AZ, UK Birkeland Centre for Space Science, University in Bergen, Norway 9 Space Science and Applications, Los Alamos National Laboratory, Los Alamos, NM 87545, USA 23