desidatamodel Documentation

Transcription

1 desidatamodel Documentation Release 18.6.dev502 DESI Jul 20, 2018

2

3 Contents 1 The DESI Data Tree 1 2 Indices and Tables 159 Python Module Index 161 i

4 ii

5 CHAPTER 1 The DESI Data Tree DESI data are grouped broadly by category, using environment variables to define the base directory under which files of that category are kept. These variables have a standard location relative to $DESI_ROOT, but code uses these variables so that one can swap out different input/output locations for testing. 1.1 DESI_SPECTRO_DATA DESI_SPECTRO_DATA contains raw data as produced by the telescope. The canonical location is $DESI_ROOT/ spectro/data, but one can set the environment variable DESI_SPECTRO_DATA to point anywhere. The exposures are grouped by night as a YEARMMDD string. The night roles over at noon local time, so all data taken between sunset and sunrise belong to the same night (i.e. the date of the sunset). Under each night, data are grouped in subdirectories by exposure ID (zero-padded 8-digit). Subdirectories: NIGHT NIGHT is the night of observation in YYYYMMDD format. The night roles over at noon local time, so all data taken between sunset and sunrise belong to the same night (i.e. the date of the sunset). Under each night, data are grouped in subdirectories by exposure ID (zero-padded 8-digit). EXPID EXPID is the 8-digit zero-padded exposure ID. Each exposure id (expid) generates multiple files: 1

6 desi-expid Summary Raw data from the DESI spectrographs, with one fpack-compressed HDU per spectrograph camera Naming Convention desi-expid.fits.fz, where EXPID is the zero-padded 8-digit exposure ID. Regex desi-[0-9]{8}\.fits.fz File Type FITS, 500 MB Contents There is one HDU per spectrograph camera with EXTNAMEs like B0, B1,... R0, R1,... Z8, Z9. The structure of each of these is the same; only one is explicitly documented below. Number EXTNAME Type Contents HDU0 PRIMARY IMAGE Blank except for header keywords HDU1 B0 IMAGE Raw data from the b0 spectrograph FITS Header Units HDU0 EXTNAME = PRIMARY Blank except for header keywords TELRA float Telescope pointing RA [degrees] TELDEC float Telescope pointing dec [degrees] FLAVOR dark str Flavor [arc, flat, science,... ] EXPTIME 1000 int Exposure time [sec] NIGHT str Night of observation YEARMMDD EXPID 2 int DESI exposure ID DATE-OBS T22:00:00 str Start of exposure in UTC AIRMASS 1.0 float Airmass at middle of exposure TILEID 4 int DESI tile ID DOSVER SIM str FEEVER SIM str DETECTOR SIM str DEPNAM00 python str DEPVER str DEPNAM01 numpy str DEPVER str DEPNAM02 scipy str DEPVER str Continued on next page 2 Chapter 1. The DESI Data Tree

7 Table 1 continued from previous page DEPNAM03 astropy str DEPVER str DEPNAM04 yaml str DEPVER str DEPNAM05 desiutil str DEPVER dev402 str DEPNAM06 desispec str DEPVER dev1288 str DEPNAM07 desitarget str DEPVER dev754 str DEPNAM08 desimodel str DEPVER dev178 str DEPNAM09 desisim str DEPVER dev825 str DEPNAM10 specter str DEPVER dev1 str DEPNAM11 speclite str DEPVER str Empty HDU. TODO: Synchronize with ICS for keywords (e.g. FLAVOR -> PROGRAM) HDU1 EXTNAME = B0 Unprocessed spectrograph raw data, including overscans, from camera B0. NAXIS int width of table in bytes NAXIS int number of rows in table EXTNAME B0 str name of this binary table extension TELDEC float Telescope pointing dec [degrees] FLAVOR dark str Flavor [arc, flat, science,... ] EXPTIME 1000 int Exposure time [sec] NIGHT str Night of observation YEARMMDD EXPID 2 int DESI exposure ID TELRA float Telescope pointing RA [degrees] DATE-OBS T22:00:00 str Start of exposure AIRMASS 1.0 float Airmass at middle of exposure TILEID 4 int DESI tile ID DOSVER SIM str CAMERA b0 str Continued on next page 1.1. DESI_SPECTRO_DATA 3

8 Table 2 continued from previous page FEEVER SIM str DETECTOR SIM str GAIN1 1.0 float Gains from ICS GAIN2 1.0 float GAIN3 1.0 float GAIN4 1.0 float RDNOISE1 3.0 float Expected readnoise from ICS, not measured from these data RDNOISE2 3.0 float RDNOISE3 3.0 float RDNOISE4 3.0 float PRESEC1 [1:4,1:2048] str DATASEC1 [5:2052,1:2048] str BIASSEC1 [2053:2102,1:2048] str CCDSEC1 [1:2048,1:2048] str PRESEC2 [4201:4204,1:2048] str DATASEC2 [2153:4200,1:2048] str BIASSEC2 [2103:2152,1:2048] str CCDSEC2 [2049:4096,1:2048] str PRESEC3 [1:4,2049:4096] str DATASEC3 [5:2052,2049:4096] str BIASSEC3 [2053:2102,2049:4096] str CCDSEC3 [1:2048,2049:4096] str PRESEC4 [4201:4204,2049:4096] str DATASEC4 [2153:4200,2049:4096] str BIASSEC4 [2103:2152,2049:4096] str CCDSEC4 [2049:4096,2049:4096] str INHERIT T bool Data: int32 FITS image [ny, nx] Notes and Examples Add notes and examples here. You can also create links to example files. Expected Changes Coordinate with ICS for header keywords Add telemetry HDU with contents TBD fibermap-expid.fits Summary The fibermap contains the fiber positioner configuration information for each exposure: what fiber is placed where, what target that is, etc. Naming Convention fibermap-{expid}.fits, where {EXPID} is the 8-digit exposure ID. Regex fibermap-[0-9]{8}\.fits File Type FITS, 1 MB 4 Chapter 1. The DESI Data Tree

9 This table is also propagated forward to the frame, cframe, and spectra files. Contents Number EXTNAME Type Contents HDU0 PRIMARY IMAGE Blank HDU1 FIBERMAP BINTABLE Fiber map table FITS Header Units HDU0 EXTNAME = PRIMARY Empty HDU. This HDU has no non-standard required keywords. HDU1 EXTNAME = FIBERMAP The fiber map table of which targets where placed on which fibers at which locations. NAXIS1 378 int length of dimension 1 NAXIS int length of dimension 2 NIGHT str YEARMMDD of sunset for this night EXPID 2 int unique DESI exposure ID TILEID 4 int DESI tile ID PROGRAM DARK str program [dark, bright,... ] FLAVOR dark str Flavor [arc, flat, science, zero,... ] TELRA float Telescope pointing RA in J2000 degrees TELDEC float Telescope pointing dec in J2000 degrees AIRMASS float Airmass at middle of exposure EXPTIME float Exposure time [sec] SEEING float Seeing FWHM [arcsec] MOONFRAC float Moon illumination fraction 0-1; 1=full MOONALT float Moon altitude [degrees] MOONSEP float Moon:tile separation angle [degrees] DATE-OBS T03:35: str Start of exposure TODO: standardize keywords with ICS, e.g. FLAVOR -> PROGRAM DESI_SPECTRO_DATA 5

10 Required Data Table Columns Name Type Units Description OBJTYPE char[10] Target type [ELG, LRG, QSO, STD, STAR, SKY] TARGETCAT char[20] Name/version of the target catalog BRICKNAME char[8] Brickname from target imaging TARGETID int64 Unique target ID DESI_TARGET int64 DESI dark+calib targeting bit mask BGS_TARGET int64 DESI Bright Galaxy Survey targeting bit mask MWS_TARGET int64 DESI Milky Way Survey targeting bit mask MAG float32[5] magnitudes in each of the filters FILTER char[200] SDSS_R, DECAM_Z, WISE1, etc. SPECTROID int64 Spectrograph ID [0-9] POSITIONER int32 Positioner Location ID [0-9542] LOCATION int32 DEVICE_LOC int32 PETAL_LOC int32 FIBER int32 Fiber ID [0-4999] LAMBDAREF float32 Reference wavelength at which to align fiber RA_TARGET float64 Target right ascension [degrees] DEC_TARGET float64 Target declination [degrees] RA_OBS float64 RA of obs from (X,Y)_FVCOBS and optics [deg] DEC_OBS float64 dec of obs from (X,Y)_FVCOBS and optics [deg] X_TARGET float32 X on focal plane derived from (RA,DEC)_TARGET Y_TARGET float32 Y on focal plane derived from (RA,DEC)_TARGET X_FVCOBS float32 X location observed by Fiber View Cam [mm] Y_FVCOBS float32 Y location observed by Fiber View Cam [mm] Y_FVCERR float32 Y location uncertainty from Fiber View Cam [mm] X_FVCERR float32 X location uncertainty from Fiber View Cam [mm] Expected Changes This format is out of sync with the Imaging Legacy Surveys datamodel, and will be updated to have consistent names and formats for values that are propagated from the tractor / sweep files. e.g. MAG[5] -> FLUX_G, FLUX_R, FLUX_Z, FLUX_W1, FLUX_W2. This format is out of sync with the tile datamodel, and will be updated to be consistent for values that are propagated forward from fiber assignment. e.g. POSITIONER -> LOCATION. focus-expid Summary Placeholder datamodel for the focus GFA raw data Naming Convention focus-expid.fits.fz, where EXPID is the zero-padded 8-digit exposure ID. Regex focus-[0-9]{8}\.fits.fz File Type FITS, 600 MB 6 Chapter 1. The DESI Data Tree

11 Contents Details are TBD, but the nominal format is: Number EXTNAME Type Contents HDU0 FOCUS IMAGE Header keywords only HDU1 FOCUS1 Compressed IMAGE Focus GFA frames from petal 1 HDU2 FOCUS1T BINTABLE Metadata about FOCUS1 frames HDU3 FOCUS3 Compressed IMAGE Focus GFA frames from petal 3 HDU4 FOCUS3T BINTABLE Metadata about FOCUS3 frames HDU5 FOCUS6 Compressed IMAGE Focus GFA frames from petal 6 HDU6 FOCUS6T BINTABLE Metadata about FOCUS6 frames HDU7 FOCUS8 Compressed IMAGE Focus GFA frames from petal 8 HDU8 FOCUS8T BINTABLE Metadata about FOCUS8 frames The FOCUSn data will be 3D[nframes, ny, nx] such that data[i] is the 2D GFA frame number i. Row i of the FOCUSnT table will contain the metadata about that frame, e.g. the DATE-OBS and EXPTIME. Note that other than the blank data primary HDU, the order of the other HDUs is arbitrary and some FOCUSn(T) HDUs may even be missing. The nominal set (1,3,6,8) is the plan for full DESI, but particularly during commissioning other combinations will appear in the data. Other than the name and number of the HDUs, the structure of this format is identical to the guide GFA raw data. guide-expid Summary Placeholder datamodel for the guider GFA raw data Naming Convention guide-expid.fits.fz, where EXPID is the zero-padded 8-digit exposure ID. Regex guide-[0-9]{8}\.fits.fz File Type FITS, 1 GB Contents Details are TBD, but the nominal format is: Number EXTNAME Type Contents HDU0 GUIDE IMAGE Header keywords only HDU1 GUIDE0 Compressed IMAGE Focus GFA frames from petal 0 HDU2 GUIDE0T BINTABLE Metadata about FOCUS0 frames HDU3 GUIDE2 Compressed IMAGE Focus GFA frames from petal 2 HDU4 GUIDE2T BINTABLE Metadata about FOCUS2 frames HDU5 GUIDE4 Compressed IMAGE Focus GFA frames from petal 4 HDU6 GUIDE4T BINTABLE Metadata about FOCUS4 frames HDU7 GUIDE5 Compressed IMAGE Focus GFA frames from petal 5 HDU8 GUIDE5T BINTABLE Metadata about FOCUS5 frames HDU9 GUIDE7 Compressed IMAGE Focus GFA frames from petal 7 HDU10 GUIDE7T BINTABLE Metadata about FOCUS7 frames HDU11 GUIDE9 Compressed IMAGE Focus GFA frames from petal 9 HDU12 GUIDE9T BINTABLE Metadata about FOCUS9 frames 1.1. DESI_SPECTRO_DATA 7

12 The GUIDEn data will be 3D[nframes, ny, nx] such that data[i] is the 2D GFA frame number i. Row i of the GUIDEnT table will contain the metadata about that frame, e.g. the DATE-OBS and EXPTIME. Note that other than the blank data primary HDU, the order of the other HDUs is arbitrary and some GUIDEn(T) HDUs may even be missing. The nominal set (0,2,4,5,7,9) is the plan for full DESI, but particularly during commissioning other combinations will appear in the data. Other than the name and number of the HDUs, the structure of this format is identical to the focus GFA raw data. sky-expid Summary Placeholder datamodel for the sky monitor raw data Naming Convention sky-expid.fits.fz, where EXPID is the zero-padded 8-digit exposure ID. Regex sky-[0-9]{8}\.fits.fz File Type FITS, 350 MB Contents Details are TBD, but the nominal format is: Number EXTNAME Type Contents HDU0 SKY IMAGE Header keywords only HDU1 SKYCAMERA Compressed IMAGE Sky camera frames HDU2 SKYCAMERAT BINTABLE Metadata about each frame The SKYCAMERA data will be 3D[nframes, ny, nx] such that data[i] is the 2D sky camera frame number i. Row i of the SKYCAMERAT table will contain the metadata about that frame, e.g. the DATE-OBS and EXPTIME. 1.2 DESI_SPECTRO_REDUX This directory contains production runs of the spectroscopic data reduction pipeline. Each different run is contained in different SPECPROD subdirectory. The desispec code refers to this location with the environment variable $DESI_SPECTRO_REDUX. The canonical location is $DESI_ROOT/spectro/redux, but changing the environment variable allows the code to write to other test directories SPECPROD Input files for spectroscopic reduction: $DESI_SPECTRO_DATA/{night}/ desi-{expid}.fits.fz fibermap-{expid}.fits $DESI_SPECTRO_SIM/{night}/ desi-{expid}.fits.fz fibermap-{expid}.fits pix-{camera}-{expid}.fits (continues on next page) 8 Chapter 1. The DESI Data Tree

13 simpix-{camera}-{expid}.fits simspec-{expid}.fits (continued from previous page) Output files: $DESI_SPECTRO_REDUX/$SPECPROD/ pix-{camera}-{expid}.fits -- optional calibnight/{night}/ fiberflatnight-{camera}-{night}.fits calibnight/psf/{night}/ psf-{camera}.fits exposures/{night}/{expid}/ calib-{camera}-{expid}.fits cframe-{camera}-{expid}.fits frame-{camera}-{expid}.fits psf-{camera}-{expid}.fits psfboot-{camera}-{expid}.fits sky-{camera}-{expid}.fits stdstars-{spectrograph}-{expid}.fits spectra-{nside}/{group}/{pixnum}/ coadd-{nside}-{pixnum}.fits -- does not yet exist spectra-{nside}-{pixnum}.fits zbest-{nside}-{pixnum}.fits zcatalog-{specprod}.fits calib1d Placeholder. calibnight NIGHT fiberflatnight-camera-night Summary Relative fiber-to-fiber variations ( fiberflat ) as measured by continuum lamp calibration spectra, combined across multiple exposures. Corrected flux = original flux / fiberflat. Naming Convention fiberflatnight-camera-night.fits, where CAMERA is e.g., b0, r5, etc. and NIGHT is the observation night in YYYYMMDD format. Regex fiberflatnight-[brz][0-9]-[0-9]{8}.fits File Type FITS, 10 MB 1.2. DESI_SPECTRO_REDUX 9

14 Contents Number EXTNAME Type Contents HDU0 FIBERFLAT IMAGE Relative fiber-to-fiber variation HDU1 IVAR IMAGE Inverse variance of fiberflat HDU2 MASK BINTABLE Mask of fiberflat (0=good) HDU3 MEANSPEC IMAGE Average spectrum HDU4 WAVELENGTH IMAGE Wavelength FITS Header Units HDU0 EXTNAME = FIBERFLAT Relative fiber-to-fiber variation. Corrected flux = original flux / fiberflat. NAXIS int NAXIS2 500 int EXPID 25 int NIGHT str FLAVOR flat str DOSVER SIM str DATE-OBS T22:00: str EXPTIME 10.0 float FEEVER SIM str AIRORVAC vac str Vacuum wavelengths CAMERA r6 str FIBERMIN 3000 int CHECKSUM 3GDaAE9W6ECaAE9U str HDU checksum updated T21:38:41 DATASUM str data unit checksum updated T21:38:41 CHI2PDF float Data: FITS image [float32, 2645x500] HDU1 EXTNAME = IVAR Inverse variance of fiberflat 10 Chapter 1. The DESI Data Tree

15 NAXIS int NAXIS2 500 int CHECKSUM QedlRcbkQcbkQcbk str HDU checksum updated T21:38:41 DATASUM str data unit checksum updated T21:38:41 Data: FITS image [float32, 2645x500] HDU2 EXTNAME = MASK Mask of fiberflat (0=good) NAXIS1 8 int width of table in bytes NAXIS2 500 int number of rows in table BSCALE 1 int BZERO int CHECKSUM LmE7NjE7LjE7LjE7 str HDU checksum updated T21:38:41 DATASUM str data unit checksum updated T21:38:41 Data: FITS image [int32 (compressed), 8x500] HDU3 EXTNAME = MEANSPEC Average continuum lamp spectrum NAXIS int CHECKSUM 9bCFEZAF9bAFEZAF str HDU checksum updated T21:38:41 DATASUM str data unit checksum updated T21:38:41 Data: FITS image [float32, 2645] 1.2. DESI_SPECTRO_REDUX 11

16 HDU4 EXTNAME = WAVELENGTH Wavelengths at which the fiberflat is measured NAXIS int BUNIT Angstrom str CHECKSUM 7A46A9240A str HDU checksum updated T21:38:41 DATASUM str data unit checksum updated T21:38:41 Data: FITS image [float32, 2645] Notes and Examples Corrected flux = original flux / fiberflat. fiberflat = desispec.fiberflat.compute_fiberflat(flatframe) desispec.fiberflat.apply_fiberflat(scienceframe, fiberflat) psf NIGHT psf Summary Gauss-Hermite PSF fits file generated by specex. Primary HDU is empty. It only contains the PSF type (keyword PSFTYPE) and version (keyword PSFVERSION) of the format. Naming Convention psf-as-expnum.fits, where A is the spectrograph arm ( b, r or z ) and S the spectrograph identifier [0-9]. Regex psf-[brz][0-9]\.fits File Type FITS, 2 MB Contents Number EXTNAME Type Contents HDU0 PRIMARY IMAGE Blank data. Header keywords PSFTYPE and PSFVER. HDU1 PSF IMAGE Coefficients for Gauss-Hermite PSF 12 Chapter 1. The DESI Data Tree

17 FITS Header Units HDU0 EXTNAME = PRIMARY Empty HDU. Blank except for keywords PSFTYPE and PSFVER. KEY Value Type Comment PSFTYPE GAUSS-HERMITE str Type of PSF (several PSF models in specex) PSFVER 2 str Version of the fits format for this PSFTYPE HDU1 EXTNAME = PSF This HDU contains all the data that describe the spectrograph PSF for a given spectrograph arm/ccd. Each row of the table contains the data vector of one PSF parameter. The size of the vector is ((FIBERMAX-FIBERMIN+1)*(LEGDEG+1)) Description of the NPARAMS parameters : X : CCD column coordinate (as a function of fiber and wavelength) Y : CCD row coordinate (as a function of fiber and wavelength) (X,Y)=(0,0) means that PSF is centered on center of first pixel GHSIGX : Sigma of first Gaussian along CCD columns for PSF core GHSIGY : Sigma of first Gaussian along CCD rows for PSF core GH-i-j : Hermite pol. coefficents, i along columns, j along rows, i is integer from 0 to GHDEGX, j is integer from 0 to GHDEGY, there are (GHDEGX+1)*(GHDEGY+1) such coefficents. TAILAMP : Amplitude of PSF tail TAILCORE : Size in pixels of PSF tail saturation in PSF core TAILXSCA : Scaling apply to CCD coordinate along columns for PSF tail TAILYSCA : Scaling apply to CCD coordinate along rows for PSF tail TAILINDE : Asymptotic power law index of PSF tail CONT : Continuum flux in arc image (not part of PSF) PSF_core(X,Y) = [ SUM_ij (GH-i-j)*HERM(i,X/GHSIGX)*HERM(j,Y/GHSIGX) ]*GAUS(X,GHSIGX)*GAUS(Y,GHSIGY) PSF_tail(X,Y) = TAILAMP*R^2/(TAILCORE^2+R^2)^(1+TAILINDE/2) with R^2=(X/TAILXSCA)^2+(Y/TAILYSCA)^2 PSF_core is integrated in pixel, PSF_tail is not, it is evaluated at center of pixel DESI_SPECTRO_REDUX 13

18 KEY Value Type Comment PSFTYPE GAUSS-HERMITE str Type of PSF (several PSF models in specex) PSFVER 2 str Version of the fits format for this PSFTYPE MJD 0 int MJD of arc lamp exposure PLATEID 0 int plate ID of arc lamp exposure CAMERA str camera ID ARCEXP 0 int ID of arc lamp exposure used to fit PSF NPIX_X 4096 int number of columns in input CCD image NPIX_Y 4096 int number of rows in input CCD image HSIZEX 14 int Half size of PSF in fit, NX=2*HSIZEX+1 HSIZEY 8 int Half size of PSF in fit, NY=2*HSIZEY+1 BUNDLMIN 0 int first bundle of fibers (starting at 0) BUNDLMAX 19 int last bundle of fibers (included) FIBERMIN 0 int first fiber (starting at 0) FIBERMAX 499 int last fiber (included) NPARAMS 91 int number of PSF parameters LEGDEG 7 int degree of Legendre pol.(wave) for parameters GHDEGX 8 int degree of Hermite polynomial along CCD columns GHDEGY 8 int degree of Hermite polynomial along CCD rows PSFERROR 0.0 float assumed PSF fractional error in chi2 READNOIS 3.0 float assumed read out noise in chi2 GAIN 1.0 float assumed gain in chi2 BnnRCHI float best fit chi2/ndf for fiber bundle 0 BnnNDATA int number of pixels in fit for fiber bundle nn BnnNPAR 300 int number of parameters in fit for fiber bundle nn Required Data Table Columns Name Type Units Description PARAM char[8] Name of parameter WAVEMIN float64 Min wavelength for mapping to [-1,1] domain WAVEMAX float64 Max wavelength for mapping to [-1,1] domain COEFF float64[91,500,8] 3D array [nparam, nspec, ncoeff] Notes and Examples This documentation was written for the output of specex version Chapter 1. The DESI Data Tree

19 exposures NIGHT EXPID calib-camera-expid.fits Summary This holds the flux calibration model for a given camera and exposure. Naming Convention calib-{camera}-{expid}.fits, where {CAMERA} is one of the spectrograph cameras (e.g. z1) and {EXPID} is the 8-digit exposure ID. Regex calib-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 10 MB Contents Number EXTNAME Type Contents HDU0 FLUXCALIB IMAGE Flux calibration model HDU1 IVAR IMAGE Inverse variance of flux HDU2 MASK IMAGE Mask (0 = good) HDU3 WAVELENGTH IMAGE wavelength in Angstrom FITS Header Units HDU0 EXTNAME = FLUXCALIB Flux calibration model such that calibrated flux = uncalibrated photons / model. NAXIS int NAXIS2 500 int NIGHT str Night of observation YEARMMDD EXPID 20 int DESI exposure ID TILEID int DESI tile ID PROGRAM DARK str program [dark, bright,... ] FLAVOR science str Flavor [arc, flat, science, zero,... ] TELRA float Telescope pointing RA [degrees] TELDEC float Telescope pointing dec [degrees] AIRMASS float Airmass at middle of exposure EXPTIME float Exposure time [sec] SEEING float Seeing FWHM [arcsec] Continued on next page 1.2. DESI_SPECTRO_REDUX 15

20 Table 3 continued from previous page MOONFRAC float Moon illumination fraction 0-1; 1=full MOONALT float Moon altitude [degrees] MOONSEP float Moon:tile separation angle [degrees] DATE-OBS T02:42: str Start of exposure PASS 1 int RA float DEC float EBMV float MJD float TRANSPAR float DOSVER SIM str FEEVER SIM str BUNIT 1e+17 cm2 electron s / erg str i.e. (electron/angstrom) / (1e-17 erg/(s cm2 Angstrom)) AIRORVAC vac str Vacuum wavelengths CAMERA r3 str FIBERMIN 1557 int CHECKSUM AXAlBW3iAWAiAW3i str HDU checksum updated T15:08:08 DATASUM str data unit checksum updated T15:08:08 Data: FITS image [float32, 2645x500] HDU1 EXTNAME = IVAR Inverse variance of flux calibration model. NAXIS int NAXIS2 500 int CHECKSUM 6MLF7LID6LID6LID str HDU checksum updated T15:08:08 DATASUM str data unit checksum updated T15:08:08 Data: FITS image [float32, 2645x500] HDU2 EXTNAME = MASK Mask of flux calibration model; 0=good. 16 Chapter 1. The DESI Data Tree

21 NAXIS int length of original image axis NAXIS2 500 int length of original image axis BSCALE 1 int BZERO int CHECKSUM YeVDcZTCZbTCaZTC str HDU checksum updated T15:08:08 DATASUM str data unit checksum updated T15:08:08 Data: FITS image [int32 (compressed), 2645x500] HDU3 EXTNAME = WAVELENGTH Wavelengths at which the flux calibration model is evaluated. NAXIS int BUNIT Angstrom str CHECKSUM 5A83A5535A53A553 str HDU checksum updated T15:08:08 DATASUM str data unit checksum updated T15:08:08 Data: FITS image [float32, 2645] Notes and Examples We may add an additional HDU with EXTNAME=METADATA containing a binary table with one row per standard star giving the details of which model was used, etc. This is not yet implemented and details TBD. cframe-camera-expid.fits Summary This holds the calibrated spectra for a given camera and exposure. See the datamodel for frame-camera-expid files for details of the format. Naming Convention cframe-{camera}-{expid}.fits, where {CAMERA} is one of the spectrograph cameras (e.g. z1) and {EXPID} is the 8-digit exposure ID. Regex cframe-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 86 MB 1.2. DESI_SPECTRO_REDUX 17

22 Contents Number EXTNAME Type Contents HDU0 FLUX IMAGE Flux, erg/s/cm2/a HDU1 IVAR IMAGE Inverse variance, (erg/s/cm2/a)^-2 HDU2 MASK IMAGE Mask (0 = good) HDU3 WAVELENGTH IMAGE wavelength in Angstrom HDU4 RESOLUTION IMAGE Resolution Matrix HDU5 FIBERMAP BINTABLE Fibermap details propagated from fibermap-expid.fits HDU6 SCORES BINTABLE Quality Assurance scores FITS Header Units HDU0 EXTNAME = FLUX Calibrated spectral flux in 1e-17 erg / (s cm2 Angstrom). NAXIS int NAXIS2 500 int NIGHT str Night of observation YEARMMDD EXPID 20 int DESI exposure ID TILEID int DESI tile ID PROGRAM DARK str program [dark, bright,... ] FLAVOR science str Flavor [arc, flat, science, zero,... ] TELRA float Telescope pointing RA [degrees] TELDEC float Telescope pointing dec [degrees] AIRMASS float Airmass at middle of exposure EXPTIME float Exposure time [sec] SEEING float Seeing FWHM [arcsec] MOONFRAC float Moon illumination fraction 0-1; 1=full MOONALT float Moon altitude [degrees] MOONSEP float Moon:tile separation angle [degrees] DATE-OBS T02:42: str Start of exposure PASS 1 int RA float DEC float EBMV float MJD float TRANSPAR float DOSVER SIM str FEEVER SIM str BUNIT 1e-17 erg/(s cm2 A) str AIRORVAC vac str Vacuum wavelengths Continued on next page 18 Chapter 1. The DESI Data Tree

23 Table 4 continued from previous page CAMERA z7 str FIBERMIN 3500 int CHECKSUM 9GKaD9HU9EHZC9HZ str HDU checksum updated T15:08:14 DATASUM str data unit checksum updated T15:08:14 Data: FITS image [float32, 2999x500] HDU1 EXTNAME = IVAR Inverse variance of flux (i.e. 1/error^2). NAXIS int NAXIS2 500 int CHECKSUM IY2iIX2iIX2iIX2i str HDU checksum updated T15:08:14 DATASUM str data unit checksum updated T15:08:14 Data: FITS image [float32, 2999x500] HDU2 EXTNAME = MASK Mask of spectra; 0=good. TODO: add documentation link to what bits mean what. NAXIS int length of original image axis NAXIS2 500 int length of original image axis BSCALE 1 int BZERO int CHECKSUM odsnqzplodplozpl str HDU checksum updated T15:08:14 DATASUM str data unit checksum updated T15:08:14 Data: FITS image [int32 (compressed), 2999x500] 1.2. DESI_SPECTRO_REDUX 19

24 HDU3 EXTNAME = WAVELENGTH Wavelengths at which flux is measured. NAXIS int BUNIT Angstrom str CHECKSUM iucmiuajiuajiuaj str HDU checksum updated T15:08:14 DATASUM str data unit checksum updated T15:08:14 Data: FITS image [float64, 2999] HDU4 EXTNAME = RESOLUTION Diagonal elements of convolution matrix describing spectral resolution. TODO: add code example for using this. NAXIS int NAXIS2 13 int NAXIS3 500 int CHECKSUM balae7jxbajab5ju str HDU checksum updated T15:08:15 DATASUM str data unit checksum updated T15:08:15 Data: FITS image [float32, 2999x13x500] HDU5 EXTNAME = FIBERMAP Fibermap of what targets were assigned to what fibers. NOTE: This format will be updated soon, e.g. to track FLUX instead of MAG. 20 Chapter 1. The DESI Data Tree

25 NAXIS1 378 int length of dimension 1 NAXIS2 500 int length of dimension 2 ENCODING ascii str CHECKSUM UUVAVUS8UUSAUUS5 str HDU checksum updated T15:08:15 DATASUM str data unit checksum updated T15:08:15 Required Data Table Columns Name Type Units Description OBJTYPE char[10] TARGETCAT char[20] BRICKNAME char[8] TARGETID int64 DESI_TARGET int64 BGS_TARGET int64 MWS_TARGET int64 MAG float32[5] FILTER char[200] SPECTROID int64 POSITIONER int32 LOCATION int32 DEVICE_LOC int32 PETAL_LOC int32 FIBER int32 LAMBDAREF float32 RA_TARGET float64 DEC_TARGET float64 RA_OBS float64 DEC_OBS float64 X_TARGET float32 Y_TARGET float32 X_FVCOBS float32 Y_FVCOBS float32 Y_FVCERR float32 X_FVCERR float32 HDU6 EXTNAME = SCORES Scores / metrics measured from the spectra for use in QA and systematics studies DESI_SPECTRO_REDUX 21

26 NAXIS1 96 int length of dimension 1 NAXIS2 500 int length of dimension 2 ENCODING ascii str CHECKSUM eqiceoz9eofceoz9 str HDU checksum updated T15:08:15 DATASUM str data unit checksum updated T15:08:15 Required Data Table Columns Name Type Units Description SUM_RAW_COUNT_Z float64 sum counts in wave. range 7600,9800A MEDIAN_RAW_COUNT_Z float64 median counts/a in wave. range 7600,9800A MEDIAN_RAW_SNR_Z float64 median SNR/sqrt(A) in wave. range 7600,9800A SUM_FFLAT_COUNT_Z float64 sum counts in wave. range 7600,9800A MEDIAN_FFLAT_COUNT_Z float64 median counts/a in wave. range 7600,9800A MEDIAN_FFLAT_SNR_Z float64 median SNR/sqrt(A) in wave. range 7600,9800A SUM_SKYSUB_COUNT_Z float64 sum counts in wave. range 7600,9800A MEDIAN_SKYSUB_COUNT_Z float64 median counts/a in wave. range 7600,9800A MEDIAN_SKYSUB_SNR_Z float64 median SNR/sqrt(A) in wave. range 7600,9800A SUM_CALIB_COUNT_Z float64 sum counts in wave. range 7600,9800A MEDIAN_CALIB_COUNT_Z float64 median counts/a in wave. range 7600,9800A MEDIAN_CALIB_SNR_Z float64 median SNR/sqrt(A) in wave. range 7600,9800A Notes and Examples Add notes and examples here. You can also create links to example files. fiberflat-camera-expid.fits Summary This file contains the fiberflat such that newflux = rawflux/fiberflat. Naming Convention fiberflat-{camera}-{expid}.fits, where {camera} is the camera name (e.g. b0, r1, z9) and {EXPID} is the zero padded 8-digit exposure ID. Regex fiberflat-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 45 MB Contents Number EXTNAME Type Contents HDU0 FIBERFLAT IMAGE fiberflat[nspec, nwave] HDU1 IVAR IMAGE inverse variance of fiberflat HDU2 MASK IMAGE bitmask of fiberflat (0=good) HDU3 MEANSPEC IMAGE average spectrum[nwave] HDU4 WAVELENGTH IMAGE wavelength grid[nwave] in Angstroms 22 Chapter 1. The DESI Data Tree

27 TODO: Current pipeline (as of ) writes HDU0 with EXTNAME=FLUX, having inherited that from the input frame. That should be changed to FIBERFLAT (as described above) or no HDU 0 EXTNAME (formal FITS standard). astropy currently writes this as FITS file which it can read but cfitsio cannot. It is unknown which library is at fault. FITS Header Units HDU0 EXTNAME = FIBERFLAT Mean fiberflat. Header keywords are inherited from the input Frame file. NAXIS int NAXIS2 500 int EXPID 0 int DESI exposure ID NIGHT str Night of observation YEARMMDD FLAVOR flat str Flavor [arc, flat, science,... ] DOSVER SIM str DATE-OBS T22:00:00 str Start of exposure EXPTIME 10 float Exposure time [sec] FEEVER SIM str AIRORVAC vac str CAMERA z0 str FIBERMIN 0 int CHECKSUM a6zgc4zga4zga4zg str HDU checksum updated T22:02:54 DATASUM str data unit checksum updated T22:02:54 CHI2PDF float Data: FITS image [float32, 2975x500] HDU1 EXTNAME = IVAR Inverse variance of the fiberflat. NAXIS int NAXIS2 500 int CHECKSUM 9PBADOB69OBAAOB3 str HDU checksum updated T22:02:54 DATASUM str data unit checksum updated T22:02: DESI_SPECTRO_REDUX 23

28 Data: FITS image [float32, 2975x500] HDU2 EXTNAME = MASK Mask of the fiberflat; 0=good. NAXIS1 8 int length of original image axis NAXIS2 500 int length of original image axis BSCALE 1 int BZERO int CHECKSUM FcV3FZT2FbT2FZT2 str HDU checksum updated T22:02:54 DATASUM str data unit checksum updated T22:02:54 Data: FITS image [int32 (compressed), 2975x500] HDU3 EXTNAME = MEANSPEC Average flat lamp spectrum of fibers in this frame. Fiberflat is relative to this mean spectrum. NAXIS int CHECKSUM cbajf94gcaagc93g str HDU checksum updated T22:02:54 DATASUM str data unit checksum updated T22:02:54 Data: FITS image [float32, 2975] HDU4 EXTNAME = WAVELENGTH Wavelength grid in Angstroms used by this fiberflat. 24 Chapter 1. The DESI Data Tree

29 NAXIS int BUNIT Angstrom CHECKSUM 9mXPJkUO9kUOGkUO str HDU checksum updated T22:02:54 DATASUM str data unit checksum updated T22:02:54 Data: FITS image [float32, 2975] frame-camera-expid.fits Summary Frame files contain the raw extracted electrons from DESI data, without any further calibration. Naming Convention frame-{camera}-{expid}.fits, where {CAMERA} is one of the spectrograph cameras (e.g. z1) and {EXPID} is the 8-digit exposure ID. Regex frame-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 70 MB Contents Number EXTNAME Type Contents HDU0 FLUX IMAGE Extracted electrons (photons) HDU1 IVAR IMAGE Inverse variance of extracted electrons HDU2 MASK IMAGE Bad value mask; 0=good HDU3 WAVELENGTH IMAGE Wavelength grid of the extraction HDU3 RESOLUTION IMAGE Resolution matrix HDU5 FIBERMAP BINTABLE Fibermap FITS Header Units HDU0 EXTNAME = FLUX Extracted electrons[nspec, nwave] NAXIS int Number of wavelength samples NAXIS2 500 int Number of extracted spectra NIGHT str Night of observation YEARMMDD EXPID 20 int DESI exposure ID Continued on next page 1.2. DESI_SPECTRO_REDUX 25

30 Table 5 continued from previous page TILEID int DESI tile ID PROGRAM DARK str program [dark, bright,... ] FLAVOR science str Flavor [arc, flat, science, zero,... ] TELRA float Telescope pointing RA [degrees] TELDEC float Telescope pointing dec [degrees] AIRMASS float Airmass at middle of exposure EXPTIME float Exposure time [sec] SEEING float Seeing FWHM [arcsec] MOONFRAC float Moon illumination fraction 0-1; 1=full MOONALT float Moon altitude [degrees] MOONSEP float Moon:tile separation angle [degrees] DATE-OBS T02:42: str Start of exposure PASS 1 int RA float DEC float EBMV float MJD float TRANSPAR float DOSVER SIM str FEEVER SIM str BUNIT electron/angstrom str AIRORVAC vac str Vacuum wavelengths CAMERA z7 str FIBERMIN 3500 int CHECKSUM BeFKEbFIBbFIBbFI str HDU checksum DATASUM int data unit checksum Data: FITS image [float32] HDU1 EXTNAME = IVAR Inverse variance of the electrons in HDU0. Data: FITS image [float32] NAXIS int Number of wavlengths NAXIS2 500 int Number of spectra CHECKSUM BeFKEbFIBbFIBbFI str HDU checksum DATASUM int data unit checksum HDU2 EXTNAME = MASK 26 Chapter 1. The DESI Data Tree

31 Mask of spectral data; 0=good. TODO: Add link to definition of which bits mean what. NAXIS int length of original image axis NAXIS2 500 int length of original image axis BSCALE 1 int BZERO int CHECKSUM gau7izs4gas4gws4 str HDU checksum updated T16:57:58 DATASUM str data unit checksum updated T16:57:58 Data: FITS image [int32 (compressed), 2951x500] HDU3 EXTNAME = WAVELENGTH 1D array of wavelengths. NAXIS1 here is the same length as NAXIS2 of the first 2 HDUs. NAXIS int Number of wavelengths BUNIT Angstrom str CHECKSUM gau7izs4gas4gws4 str HDU checksum updated T16:57:58 DATASUM str data unit checksum updated T16:57:58 Data: FITS image [float64] HDU4 EXTNAME = RESOLUTION Resolution matrix stored as a 3D sparse matrix: Rdata[nspec, ndiag, nwave] To convert this into sparse matrices for convolving a model that is sampled at the same wavelengths as the extractions (HDU EXTNAME= WAVELENGTH ): from scipy.sparse import spdiags from astropy.io import fits import numpy as np #- read a model and its wavelength vector from somewhere #- IMPORTANT: cast them to.astype(np.float64) to get native endian (continues on next page) 1.2. DESI_SPECTRO_REDUX 27

32 #- read the resolution data resdata = fits.getdata(framefile, 'RESOLUTION').astype(np.float64) (continued from previous page) nspec, nwave = model.shape convolvedmodel = np.zeros((nspec, nwave)) diags = np.arange(10, -11, -1) for i in range(nspec): R = spdiags(resdata[i], diags, nwave, nwave) convolvedmodel[i] = R.dot(model) NAXIS int length of data axis 1 NAXIS2 21 int length of data axis 2 NAXIS3 500 int length of data axis 3 CHECKSUM gau7izs4gas4gws4 str HDU checksum updated T16:57:58 DATASUM str data unit checksum updated T16:57:58 Data: FITS image [float32] HDU5 EXTNAME = FIBERMAP Fibermap propagated from the raw data inputs; see fibermap file. NAXIS1 378 int length of dimension 1 NAXIS2 500 int length of dimension 2 ENCODING ascii str CHECKSUM UUVAVUS8UUSAUUS5 str HDU checksum updated T15:08:15 DATASUM str data unit checksum updated T15:08:15 28 Chapter 1. The DESI Data Tree

33 Required Data Table Columns Name Type Units Description OBJTYPE char[10] TARGETCAT char[20] BRICKNAME char[8] TARGETID int64 DESI_TARGET int64 BGS_TARGET int64 MWS_TARGET int64 MAG float32[5] FILTER char[200] SPECTROID int64 POSITIONER int32 LOCATION int32 DEVICE_LOC int32 PETAL_LOC int32 FIBER int32 LAMBDAREF float32 RA_TARGET float64 DEC_TARGET float64 RA_OBS float64 DEC_OBS float64 X_TARGET float32 Y_TARGET float32 X_FVCOBS float32 Y_FVCOBS float32 Y_FVCERR float32 X_FVCERR float32 Notes and Examples Add notes and examples here. You can also create links to example files. psf-camera-expid.fits Spectrograph point spread functions (PSFs) derived from individual arc exposures will be in $DESI_SPECTRO_REDUX/{prodname}/exposures/{night}/{expid}/psf*.fits. These may be combined into a nightly PSF in $DESI_SPECTRO_REDUX/{prodname}/calibnight/psf/{night}/ {expid}/psf*.fits. See that location for details on the format. regex: psf-[brz][0-9]-[0-9]{8}\.fits psfboot-camera-expid Summary psfboot file DESI_SPECTRO_REDUX 29

34 Naming Convention psfboot-camera-expid.fits, where CAMERA is e.g., b0, r1, z2, and EXPID is an 8 digit exposure number. Regex psfboot-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 64 KB This file is a first fit to the x vs. y vs. wavelength solutions and the mean Gaussian sigma wavelength dispersion per fiber. It is used as a starting point for more refined fits. Contents Number EXTNAME Type Contents HDU0 COEFFX IMAGE Legendre coefficients for x vs. wavelength HDU1 COEFFY IMAGE Legendre coefficients for y vs. wavelength HDU2 SIGMA IMAGE Gaussian sigma per fiber FITS Header Units HDU0 EXTNAME = COEFFX Legendre coefficients for x vs. wavelength. Header keywords WAVEMIN and WAVEMAX provide the mapping from wavelength to the [-1,1] domain used for the Legenre polynomials. i.e. starting with wavelength w: wx = 2.0*(w-WAVEMIN)/(WAVEMAX-WAVEMIN) #- Map to [-1,1] x[i] = Sum_j XCOEFF[i,j] L_j(wx) #- evaluate Legendre series NAXIS1 6 int Number of Legendre coefficients NAXIS2 500 int Number of spectra WAVEMIN float WAVEMAX float Data: FITS image [float64, 6x500] HDU1 EXTNAME = COEFFY Legendre coefficients for y vs. wavelength; see the HDU0 description for how to interpret these. 30 Chapter 1. The DESI Data Tree

35 NAXIS1 6 int Number of Legendre coefficients NAXIS2 500 int Number of spectra WAVEMIN float WAVEMAX float Data: FITS image [float64, 6x500] HDU2 EXTNAME = SIGMA Mean Gaussian sigma wavelength dispersion per fiber. NAXIS1 500 int Mean wavelength dispersion per fiber in Angstroms Data: FITS image [float64, 500] Notes and Examples Add notes and examples here. You can also create links to example files. sky-camera-expid.fits Summary This holds the sky model for a given camera and exposure. Naming Convention sky-{camera}-{expid}.fits, where {CAMERA} is one of the spectrograph cameras (e.g. z1) and {EXPID} is the 8-digit exposure ID. Regex sky-[brz][0-9]-[0-9]{8}\.fits File Type FITS, 17 MB Contents Number EXTNAME Type Contents HDU0 SKY IMAGE sky model in photons/bin HDU1 IVAR IMAGE inverse variance (photons/bin)^-2 HDU2 MASK IMAGE sky mask (0 = good) HDU3 WAVELENGTH IMAGE wavelength in Angstrom HDU4 STATIVAR IMAGE statistical-only inverse variance of sky model 1.2. DESI_SPECTRO_REDUX 31

36 FITS Header Units HDU0 EXTNAME = SKY sky model in photons/bin NAXIS int NAXIS2 500 int NIGHT str Night of observation YEARMMDD EXPID 20 int DESI exposure ID TILEID int DESI tile ID PROGRAM DARK str program [dark, bright,... ] FLAVOR science str Flavor [arc, flat, science, zero,... ] TELRA float Telescope pointing RA [degrees] TELDEC float Telescope pointing dec [degrees] AIRMASS float Airmass at middle of exposure EXPTIME float Exposure time [sec] SEEING float Seeing FWHM [arcsec] MOONFRAC float Moon illumination fraction 0-1; 1=full MOONALT float Moon altitude [degrees] MOONSEP float Moon:tile separation angle [degrees] DATE-OBS T02:42: str Start of exposure PASS 1 int RA float DEC float EBMV float MJD float TRANSPAR float DOSVER SIM str FEEVER SIM str BUNIT electron/angstrom str AIRORVAC vac str Vacuum wavelengths CAMERA z1 str FIBERMIN 500 int CHECKSUM 1HdW1GZU1GbU1GZU str HDU checksum updated T15:04:16 DATASUM str data unit checksum updated T15:04:16 Data: FITS image [float32, 2999x500] HDU1 EXTNAME = IVAR inverse variance of sky model (photons/bin)^-2 32 Chapter 1. The DESI Data Tree

37 NAXIS int NAXIS2 500 int CHECKSUM 6WAh6U2e6U9e6U9e str HDU checksum updated T15:04:16 DATASUM str data unit checksum updated T15:04:16 Data: FITS image [float32, 2999x500] HDU2 EXTNAME = MASK sky mask (0 = good) NAXIS int length of original image axis NAXIS2 500 int length of original image axis BSCALE 1 int BZERO int CHECKSUM 0dTm2ZQl0bQl0ZQl str HDU checksum updated T15:04:16 DATASUM str data unit checksum updated T15:04:16 Data: FITS image [int32 (compressed), 2999x500] HDU3 EXTNAME = WAVELENGTH wavelength in Angstrom NAXIS int CHECKSUM 9HSaCFQZ9FQaCFQW str HDU checksum updated T15:04:16 DATASUM str data unit checksum updated T15:04:16 Data: FITS image [float32, 2999] 1.2. DESI_SPECTRO_REDUX 33

38 HDU4 EXTNAME = STATIVAR statistical-only inverse variance of sky model NAXIS int NAXIS2 500 int BUNIT Angstrom str CHECKSUM DkSiGkPgDkPgDkPg str HDU checksum updated T15:04:16 DATASUM str data unit checksum updated T15:04:16 Data: FITS image [float32, 2999x500] Notes and Examples Add notes and examples here. You can also create links to example files. stdstars-spectrograph-expid.fits Summary This file contains the normalized standard star models fitted to the frame data. Naming Convention stdstars-{spectrograph}-{expid}.fits where {SPECTROGRAPH} is the single-digit spectrograph number 0-9, and {EXPID} is the zero-padded 8-digit exposure number. Regex stdstars-[0-9]-[0-9]{8}\.fits File Type FITS, 5 MB Contents Number EXTNAME Type Contents HDU0 FLUX IMAGE stdstar flux[nstd, nwave] in erg/s/cm^2/angstrom HDU1 WAVELENGTH IMAGE wavelength grid used, Angstroms HDU2 FIBERS IMAGE 1D array of which fibers these models correspond to HDU3 METADATA BINTABLE metadata from input standard star templates HDU4 COEFF IMAGE Linear coefficients of stdstar model fit FITS Header Units HDU0 EXTNAME = FLUX Best fit standard star model flux. 34 Chapter 1. The DESI Data Tree

39 NAXIS int NAXIS2 10 int BUNIT 1e-17 erg/(s cm2 Angstrom) str Flux units CHECKSUM 3pia4mgV3mga3mgU str HDU checksum updated T15:07:19 DATASUM str data unit checksum updated T15:07:19 Data: FITS image [float32, x10] HDU1 EXTNAME = WAVELENGTH Wavelength grid used in Angstroms. NAXIS int BUNIT Angstrom str Wavelength units CHECKSUM LNePONbMLNbMLNbM str HDU checksum updated T15:07:19 DATASUM str data unit checksum updated T15:07:19 Data: FITS image [float32, ] HDU2 EXTNAME = FIBERS Fibers used for fit. NAXIS1 10 int CHECKSUM cg56ef23cf23cf23 str HDU checksum updated T15:07:19 DATASUM str data unit checksum updated T15:07:19 Data: FITS image [int32, 10] HDU3 EXTNAME = METADATA Metadata about best fit standard star models DESI_SPECTRO_REDUX 35

40 NAXIS1 56 int length of dimension 1 NAXIS2 10 int length of dimension 2 CHECKSUM 1bNA1bN71bNA1bN7 str HDU checksum updated T15:07:19 DATASUM str data unit checksum updated T15:07:19 Required Data Table Columns Name Type Units Description LOGG float64 TEFF float64 FEH float64 CHI2DOF float64 REDSHIFT float64 DATA_G-R float64 MODEL_G-R float64 HDU4 EXTNAME = COEFF Linear coefficients of stdstar model fit TODO: add example of what that means NAXIS1 931 int NAXIS2 10 int CHECKSUM X5d6X2c6X2c6X2c6 str HDU checksum updated T15:07:19 DATASUM str data unit checksum updated T15:07:19 Data: FITS image [float64, 931x10] Notes and Examples Add notes and examples here. You can also create links to example files. spectra-nside Pixels are grouped by subdirectories of PIXGROUP = NSIDE//100 to avoid having tens of thousands of directories at the same level. 36 Chapter 1. The DESI Data Tree

41 PIXGROUP Spectra are in subdirectories by their healpix NESTED pixel number PIXNUM spectra-nside-pixnum.fits Summary DESI spectra grouped by nested healpix number Naming Convention spectra-{nside}-{pixnum}.fits, where {nside} is the healpix nside and {pixnum} is the nested scheme healpix number. Regex spectra-[0-9]+-[0-9]+\.fits File Type FITS, 1 GB Contents Number EXTNAME Type Contents HDU00 PRIMARY IMAGE Empty HDU01 FIBERMAP BINTABLE fibermap table HDU02 SCORES BINTABLE scores table HDU03 B_WAVELENGTH IMAGE Wavelength array of b-channel spectra HDU04 B_FLUX IMAGE Flux of b-channel spectra HDU05 B_IVAR IMAGE Inverse variance of b-channel spectra HDU06 B_MASK IMAGE Mask of b-channel spectra HDU07 B_RESOLUTION IMAGE Resolution matrices of b-channel spectra HDU08 R_WAVELENGTH IMAGE Wavelength array of r-channel spectra HDU09 R_FLUX IMAGE Flux of r-channel spectra HDU10 R_IVAR IMAGE Inverse variance of r-channel spectra HDU11 R_MASK IMAGE Mask of r-channel spectra HDU12 R_RESOLUTION IMAGE Resolution matrices of r-channel spectra HDU13 Z_WAVELENGTH IMAGE Wavelength array of z-channel spectra HDU14 Z_FLUX IMAGE Flux of z-channel spectra HDU15 Z_IVAR IMAGE Inverse variance of z-channel spectra HDU16 Z_MASK IMAGE Mask of z-channel spectra HDU17 Z_RESOLUTION IMAGE Resolution matrices of z-channel spectra FITS Header Units HDU00 EXTNAME = PRIMARY 1.2. DESI_SPECTRO_REDUX 37

42 CHECKSUM EAnFF7l9EAlEE5l9 str HDU checksum updated T22:45:34 DATASUM 0 str data unit checksum updated T22:45:34 Empty HDU. HDU01 EXTNAME = FIBERMAP fibermap table with two additional columns NIGHT and EXPID NAXIS1 248 int length of dimension 1 NAXIS int length of dimension 2 CHECKSUM EAnFF7l9EAlEE5l9 str HDU checksum updated T22:45:34 DATASUM 0 str data unit checksum updated T22:45:34 38 Chapter 1. The DESI Data Tree

43 Required Data Table Columns Name Type Units Description OBJTYPE char[10] Target type [ELG, LRG, QSO, STD, STAR, SKY] TARGETCAT char[20] Name/version of the target catalog BRICKNAME char[8] Brickname from target imaging TARGETID int64 Unique target ID DESI_TARGET int64 DESI dark+calib targeting bit mask BGS_TARGET int64 DESI Bright Galaxy Survey targeting bit mask MWS_TARGET int64 DESI Milky Way Survey targeting bit mask MAG float32[5] magnitudes in each of the filters FILTER char[200] SDSS_R, DECAM_Z, WISE1, etc. SPECTROID int64 Spectrograph ID [0-9] POSITIONER int32 Positioner ID [0-4999] (deprecated) LOCATION int32 Positioner location ID 1000*PETAL + DEVICE DEVICE_LOC int32 Device location on petal [0-542] PETAL_LOC int32 Petal location on focal plane [0-9] FIBER int32 Fiber ID [0-4999] LAMBDAREF float32 Reference wavelength at which to align fiber RA_TARGET float64 Target right ascension [degrees] DEC_TARGET float64 Target declination [degrees] RA_OBS float64 RA of obs from (X,Y)_FVCOBS and optics [deg] DEC_OBS float64 dec of obs from (X,Y)_FVCOBS and optics [deg] X_TARGET float64 X on focal plane derived from (RA,DEC)_TARGET Y_TARGET float64 Y on focal plane derived from (RA,DEC)_TARGET X_FVCOBS float64 X location observed by Fiber View Cam [mm] Y_FVCOBS float64 Y location observed by Fiber View Cam [mm] Y_FVCERR float32 Y location uncertainty from Fiber View Cam [mm] X_FVCERR float32 X location uncertainty from Fiber View Cam [mm] NIGHT int32 Night of exposure YYYYMMDD EXPID int32 Exposure ID TILEID int32 DESI tile ID HDU02 EXTNAME = SCORES NAXIS1 288 int width of table in bytes NAXIS int number of rows in table Required Data Table Columns 1.2. DESI_SPECTRO_REDUX 39

44 Name Type Units Description SUM_RAW_COUNT_B float64 MEDIAN_RAW_COUNT_B float64 MEDIAN_RAW_SNR_B float64 SUM_FFLAT_COUNT_B float64 MEDIAN_FFLAT_COUNT_B float64 MEDIAN_FFLAT_SNR_B float64 SUM_SKYSUB_COUNT_B float64 MEDIAN_SKYSUB_COUNT_B float64 MEDIAN_SKYSUB_SNR_B float64 SUM_CALIB_COUNT_B float64 MEDIAN_CALIB_COUNT_B float64 MEDIAN_CALIB_SNR_B float64 SUM_RAW_COUNT_R float64 MEDIAN_RAW_COUNT_R float64 MEDIAN_RAW_SNR_R float64 SUM_FFLAT_COUNT_R float64 MEDIAN_FFLAT_COUNT_R float64 MEDIAN_FFLAT_SNR_R float64 SUM_SKYSUB_COUNT_R float64 MEDIAN_SKYSUB_COUNT_R float64 MEDIAN_SKYSUB_SNR_R float64 SUM_CALIB_COUNT_R float64 MEDIAN_CALIB_COUNT_R float64 MEDIAN_CALIB_SNR_R float64 SUM_RAW_COUNT_Z float64 MEDIAN_RAW_COUNT_Z float64 MEDIAN_RAW_SNR_Z float64 SUM_FFLAT_COUNT_Z float64 MEDIAN_FFLAT_COUNT_Z float64 MEDIAN_FFLAT_SNR_Z float64 SUM_SKYSUB_COUNT_Z float64 MEDIAN_SKYSUB_COUNT_Z float64 MEDIAN_SKYSUB_SNR_Z float64 SUM_CALIB_COUNT_Z float64 MEDIAN_CALIB_COUNT_Z float64 MEDIAN_CALIB_SNR_Z float64 HDU03 EXTNAME = B_WAVELENGTH Wavelength[nwave] array in Angstroms of b-channel spectra Data: FITS image [float64, nwave] NAXIS int Number of wavelengths BUNIT Angstrom str 40 Chapter 1. The DESI Data Tree

45 HDU04 EXTNAME = B_FLUX Flux[nspec,nwave] array in 1e-17 erg/(s cm2 Angstrom) of b-channel spectra NAXIS int Number of wavelengths NAXIS int Number of spectra BUNIT 1e-17 erg/(s cm2 Angstrom) str Data: FITS image [float32, nspec x nwave] HDU05 EXTNAME = B_IVAR Inverse variance of b-channel flux array NAXIS int Number of wavelengths NAXIS int Number of spectra BUNIT 1e+34 (s2 cm4 Angstrom2) / erg2 str Data: FITS image [float32, nspec x nwave] HDU06 EXTNAME = B_MASK Mask[nspec,nwave] of b-channel flux array NAXIS int Number of wavelengths NAXIS int Number of spectra ZSIMPLE T bool This keyword probably should not be here. BZERO int BSCALE 1 int Data: FITS image [int32 (compressed), 2975x5550] 1.2. DESI_SPECTRO_REDUX 41