Data Reduction for XCOV30

Transcription

1 Data Reduction for XCOV30 1) Location of Original Data files All observers will upload their data to daedalus.dot.physics.udel.edu (note the dot ). The original observations are located in /home/data/incoming and are organized by observatory. Files in this directory are not to be changed in any way. It is very important to maintain a pristine copy of the incoming data. You may copy files from this directory to other locations, but please do not move or edit the original incoming observations. All data reduction will be done on helios. 2) Organization of files on Helios The data reduction computer is helios.dot.physics.udel.edu (please note and change the dot when logging in). The user name and password will be distributed separately. The home directory contains a subdirectory for this run called xcov30. All files and reductions will take place within that directory. The directory xcov30 has been set up with standard WET data structure as outlined below: xcov30 incoming lcs reduced working The incoming directory is a copy of the incoming directory on daedalus. Hopefully, I will have time to create a script that syncs this directory with the incoming directory on daedalus. It may be necessary to run this script occasionally. Further instructions will follow. The working directory is where the data reductions are actually performed. This directory contains subdirectories (4 letter codes) for each observatory. At the end of the run, the working directory will be removed, so it is very important to copy reductions and light curves to the appropriate final directory (see below). The reduced directory is where we will copy the final reduction directory created by Maestro. See the section outlining the use of Maestro for more details. The lcs directory is where we will put the final light curves. This directory will contain a subdirectory for each target object. It is important that each light curve be copied to the appropriate directory.

2 3) Data Reduction with Maestro A detailed instruction manual for Maestro can be found at Please read the sections on syntax and using Maestro before proceeding. I have set up a practice data reduction on helios. Please log on to helios and cd to /home/wet/practice. 1) The incoming directory contains two runs ( and ) from Krakow that are waiting to be reduced. 2) Cd to the working directory. Here you will a directory for Krakow (krak) and a second directory labeled scripts. The scripts directory contains a python script that we will use to help select apertures in a later step. 3) Cd into the krak directory. Here you will find a copy of the observations from ) Cd into the directory. Here you will find the science and calibration images. Note that, despite our best efforts, everyone uses a slightly different naming convention. For this data set, the object name (gd29-38), filter (bg40) and exposure time (20s) are included in the file name. However, the observatory and date are not. Therefore, we must be careful with this data set, so it doesn t end up confused with anything else over time. 5) The observers were careful to include bias, flat, and dark in the names of their calibration files. This makes life much easier for the data reductions. Maestro requires lists that contain the biases, darks, flats, and object images. To create these lists: ::> ls -1 *bias*.fits > biaslist ::> ls -1 *dark*.fits > darklist ;;> ls -1 *flat*.fits > flatlist ::> ls -1 *gd29*.fits > objlist You must use these names for the lists. If you use something else, you will need to explicitly tell Maestro where the lists are. Some observatories refuse to create new calibration files every night. In this case, you will need to use calibration files from previous nights. The list files must still be located in the directory in which you are working. Each list file must contain the full path name for the calibration files. You should now have in your directory all of the original images, as well as the 4 files containing the lists. 6) Maestro contains a command to allow you to check images to make sure everything is okay. This is important for the calibration files, as observers surprisingly often send flats that are either over or under exposed, and/or darks with interesting features. To run this command on the bias files, for example: ::> maestro The output of this command will show you the mean, median, and standard deviation for each bias image (note that is required). It is also possible to view the values for fits header keywords. For example, if you want to make sure that all of the images are actually biases:

3 ::> maestro inspect -k IMAGETYP This command will show you the mean, median, standard deviation, and the value of the header keyword IMAGETYP. Note that all observatories DO NOT use the same keywords, so you may have to look at the actual headers to determine the keyword you want. Run this command on all of the bias, dark, and flat list files before you do the actual reductions. It is particularly important to run this command on the flats (do so now). You will note, that for this example, the observers have sent sets of flats with different exposure times. This should be okay, because Maestro looks for the exposure times during the reductions. However, you will want to look for flats that are saturated (mean/median above 55000) or are underexposed (mean/median below about is usually a good cut off). You will want to remove any such flats from your flatlist. 6) Running Maestro for data reduction. a) The basic command is: ::> maestro reduce In this form, Maestro will look for biaslist, darklist, flatlist, and objlist. It will calibrate each image, and perform aperture photometry (for apertures between 2 and 20 pixels) for every star in the field, up to a maximum of 30 stars. The output is automatically sorted from brightest star to the faintest. b) Building a field While this sorting is sufficient for sparse fields, if the variable star is not one of the 10 brightest stars in the field, it will not be displayed with wqed (see later discussion). Even worse, if the variable is not one of the 30 brightest stars in the field, aperture photometry will not be performed. To avoid these complications, we usually construct a field file that contains the x and y coordinates of the stars that we certainly want aperture photometry for, and we usually place the variable star s coordinates as the very first entry. This will be a REQUIREMENT for NGC1501, as Maestro normally discards sources that are not round. It is also important to note that you only need to build a field file once, for each observatory. Field files are stored in /home/wet/.maestro/fields, and so can be reused. The command to build a field is: ::>maestro buildfield -w fieldname Maestro will use the images contained in objlist, and build a file that contains the x and y coordinates for each star. This file will be in /home/wet/.maestro/fields/fieldname. Fieldname is the name you give your field file. Our normal convention is object_observatorycode (for example g2938_krak). Take a look in /home/wet/.maestro/fields to see the files that are already there. Here is the original content of g2938_krak: # Reference File: "gd29-381bg4020s0032.fits" Unknown Unknown

4 Unknown Unknown Unknown The first line is the reference image used by Maestro. The remaining lines contain x, y, and counts for each star in the field. This field contained 5 stars. To identify which is g29-38, you will need to open gd29-381bg4020s0032.fits (ds9 gd29-381bg4020s0032.fits) and identify g29-38 s coordinates in the field. Once you know the coordinates, you can search the field file to identify which source is g In this case, g2938 is Unknown2. Edit the field file, and move Unknown2 to the top of the list, and relabel it as g Your field file should now look like: # Reference File: "gd29-381bg4020s0032.fits" g Unknown Unknown Unknown Unknown Building a field file for NGC1501 will be a bit more difficult, because Maestro will automatically reject the planetary nebula as a source. We will need to determine x, y, and counts level for NGC1501 and enter this information manually. You will display the reference field. The easiest way to determine x, y, and counts is to use the aperture photometry function in imexamine (iraf). and use that image to determine the x and y coordinates for the central star, or you can try a visual examination and determine a counts level by looking at the cursor display. This method may be quicker, but is not as accurate. Once you have determined the values, you will that information as the first target in the field file. Your field file should look something like (note this is only an example): # Reference File: "sspm j o.fits" ngc Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Unknown Field files must always be placed in /home/wet/.maestro/fields. Maestro is hard wired to look there. Field files must be created for each observatory set up, since field sizes differ from telescope to telescope.

5 c) Running Maestro with a field file The command is: ::> Maestro reduce -s g2938_krak The -s flag is used to identify the field file. In this example, g2938_krak is the field file name. Maestro will use the field file to order the stars, and g2938 (or which ever target is being reduced) will be the first star in the output. d) Maestro output while running. Once you have started Maestro, it will keep you informed of its progress. The first few steps involve identifying the contents of the list files (biaslist, darklist, flatlist, objlist). Maestro will check all of the files, and begin calibration. It will calculate the noise and gain, so you can check those to make sure everything looks okay. It will then search for stars on the images, and then perform aperture photometry. Normal Maestro output while running: Maestro: The Matlab Astronomy Toolkit Copyright (C) James Dalessio BUILDING LISTS OF FITS FILES Using ASCII lists to determine which FITS files to use. Bias files: 50 Dark files: 40 Flat files: 24 Object files: 606 CHECKING FILE INTEGRITY Checking integrity of 606 OBJECT FITS files. [ OK ] Checking integrity of 50 BIAS FITS files. [ OK ] Checking integrity of 40 DARK FITS files. [ OK ] Checking integrity of 24 FLAT FITS files. [ OK ] CHECKING FILE SIZES Checking size of 606 OBJECT FITS files... [ OK ] Checking size of 50 BIAS FITS files... [ OK ] Checking size of 40 DARK FITS files... [ OK ] Checking size of 24 FLAT FITS files... [ OK ] CHECKING FITS KEYWORDS Checking keywords of 606 OBJECT FITS files. [ OK ]

6 7) Output BEGINNING CALIBRATION BUILD Building a master bias? [Y] Building a master dark? [Y] Building a master flat? [Y] BUILDING MASTER BIAS Loading 50 images and checking size... [ OK ] Calculating readnoise from bias set... [ OK ] Read noise for each amplifier in ADU: (6.2003) BUILDING MASTER DARK Loading 40 images and checking size... [ OK ] BUILDING MASTER FLAT Loading 24 images and checking size... [ OK ] Calculating gain from flat fields... [ OK ] Gain for each amplifier in e-/adu: (1.3338) BUILDING MASTER FIELD Searching for stars on 606 images...[ 01% ] Note: some observatories do not send dark frames. In this case, Maestro will not build a master dark. The output will be placed in a directory named with the format yyyymmdd_nnnnnn_starfile where yyyy=year, mm=month, dd=day, nnnnnn is a number generated by Maestro, and starfile is the star field file (if used). This directory will contain three sets of files: the photometry output in the format counts_aperturesize, where aperture sizes are given in pixels, and range between 2 and 20 pixels, incremented by 0.5 pixels 2) noise files with similar format, and 3) signal files. There are several specialized files that may be used in the next steps of the reduction process: firstimage.fits is a copy of the very first science image. The header contains information that will be used by wqed. Wqed will be discussed in detail in following sections. counts_sn_optimized_aperture is Maestro s determination of the best aperture size for this star. Note that this is almost always a bit smaller than the determination from visual examination of the data. Therefore, this is almost never used. counts_dynamic_apertures is the file generated using variable apertures, where the best aperture is determined for each image. This is, for reasons we don t clearly understand, almost always noisier than the fixed aperture solution, and so is not used. 8) General notes

7 In general, Maestro is very robust and runs with very little difficulty. However, even Maestro cannot anticipate all possibilities. Here are a few we ve encountered. Cube data an increasing number of observatories save their data in cube format. Cube format is often not the optimized for photometry, since times are not usually recorded for each image. In this case, it is often necessary to write a script that alters original data headers to include needed information. A few python scripts can be found in /home/wet/xcov30/working/scripts. Fits Header sizes Some observatories (most notably PROMPT) do not pad their fits headers, so the fits files are not the same size. Maestro thinks this means that the files are corrupt, when they are not. To turn off automatic file size checking, use Maestro reduce -I -s fieldfile. Maestro does not like spaces in pathnames given in the list files. You may have to rename directories to remove spaces. Maestro does not like previously calibrated data. You may have to ask some sites for their calibration files. Maestro does not like changes in image orientation. This happens with MCAO and PROMPT, and a few other sites. In this case, the easiest thing to do is break the run into pieces, and reduce each piece separately.