| midirinfo (July2009) | gemini.midir | midirinfo (July2009) |
The tasks are designed to provide a fairly complete and flexible reduction for the purpose of assessing data quality. Real-time reductions may not be optimal for a particular science application. The tasks produce logfiles of the performed processing steps. The name of the logfile may be set in each individual task, or at the package level by setting midir.logfile. The tasks add header keywords to the output images. These header keywords contain information about the performed processing steps and timestamps for each step of the reduction.
The basic file structure for raw mid-IR data is a multi-extension FITS file with the primary header in extension 0, and one or more image extensions containing co-added T-ReCS or Michelle data frames. For CHOP or STARE mode observations there will be only one extension. For CHOP-NOD or NOD mode there will be one extension per NOD position. Each extension contains a small FITS extension header with some of the parameters specific to the individual images -- such as the start and end UT times and the airmass. (For CHOP or STARE mode with just one extension these could have been put in the primary header, but for MEF files with more than one extension the values change for each image extension.) The extension then has the images themselves. Each image is made up of one or more co-added raw frames from T-ReCS or Michelle. Each raw frame is a [320,240] array of data values of type LONG or DOUBLE. The format of the image(s) in the extension(s) depends upon the mode of the observation and on which instrument, T-ReCS or Michelle, was used to obtain the data. They all contain one or several, usually more than one, frames. For Michelle data obtained in CHOP and CHOP-NOD mode, each image extension is three dimensional: [320,240,3] where frames from each chop position are combined to form chopA and chopB position images, and these are differenced to form a dif frame. These three (chopA, chopB and dif) co-added images are delineated in the third data dimension in each image extension. In turn each of these images is a result of stacking a number of frames with exposure time of order 25 milliseconds internally in Michelle during the time at each NOD position. This "frame time" is chosen to avoid saturation of the individual frames due to the high mid-infrared background from the atmosphere and the telescope.
When chopping with T-ReCS, there are frames from each chop position. In that case a pair of co-added frames from the chop positions are called a saveset. In STARE mode or NOD mode, where there is no chopping, one has a number of co-added frames written out during the observation at a given nod position, and the image dimension is [320,320,1,N] where N stands for the number of savesets written out during the observation. This N value is determined by the T-ReCS set-up and the total exposure time. In CHOP or CHOP-NOD mode one has two chop positions per saveset and some number M of savesets. The image size is then [320,240,2,M]. The most common observation mode is CHOP-NOD mode, and each extension contains the savesets for a single NOD position. The T-ReCS NOD observations are done by starting at NOD position A and then going to NOD position B for the same length of time. With T-ReCS the telescope then returns to NOD position A and observations are taken in ABABABAB pattern. Proper NOD or CHOP-NOD observations with T-ReCS therefore should have an even number of image extensions.
As with Michelle, the actual "frame time" of T-ReCS observations is short, of order 25 milliseconds for a typical imaging observation and somewhat longer for a typical spectroscopy observation, and a number of these frames are accumulated by co-adding in internal buffers for the two chop positions in the normal case of chopping observations. At the end of the specified saveset time, normally 10 seconds, the stacked images are written out to the FITS extension. Thus there are three distinct "exposure times" for T-ReCS data: the frame time, the saveset time, and the total exposure time.
In a normal T-ReCS chop-nod mode raw data file the images in each extension have dimension [320,240,2,N] where N is the number of savesets per nod position. This value is given in the primary FITS header as the NSAVSETS value.
To calculate the number of frames co-added by T-ReCS for a saveset one uses the FRMCOADD and CHPCOADD, values in the primary header of the MEF file. The number of frames co-added is the product of the FRMCOADD and CHPCOADD values (the number of frames co-added per chop position times the number of chop cycles per saveset). This number of frames times the FRMTIME parameter gives the exposure time per saveset on the target field, in milli-seconds. For flux calibration one can directly compare T-ReCS raw difference images that have the same time on source per saveset. If the images differ in the FRMTIME, FRMCOADD, or CHPCOADD, values then one would need to scale to a common saveset exposure time before comparing images.
For Michelle the analogous keywords are EXPOSURE and NUMEXPOS, the product of which gives the observation time per nod position. This value is the exposure time on target per nod position when carrying out chop-nod mode observations. The time spent actually collecting data per nod is twice this since the target is in the (guided) beam only half the time.
When data are obtained in CHOP or STARE mode for each instrument the output data structure is the same as NOD or CHOP-NOD mode except that there is only one NOD position. A STARE mode observation is treated as a long NOD observation in NOD position A. A CHOP mode observation is treated as a long CHOP-NOD observation in NOD position A. STARE and CHOP mode observations are assumed to have only one data extension whereas those done in NOD or CHOP-NOD mode are assumed to have an even number of data extensions.
The main differences in data format between Michelle and T-ReCS are:
In order to have a common format for files from the Michelle and T-ReCS instruments, the format of the "prepared" files was defined as follows:
Currently higher resolution Michelle spectroscopy is taken in NOD-only mode. This applies to any spectroscopic observations with Michelle that do not use the low resolution N-band or Q-band gratings. When such files are prepared using the MPREPARE task the nod A and nod B positions are subtracted so that one should be left with two spectra, one positive, one negative, on the resulting stacked image. The spectral reduction routines are able to extract both the positive and negative spectra which then can be co-added for further analysis.
For Michelle the spectral flat and bias observations are done in stare mode.
TPREPARE and MPREPARE are used on the raw T-ReCS and Michelle data,
respectively, in order to collapse and reformat the images into the same data
structure. The TBACKGROUND task can be run on T-ReCS images to derive
statistics on the background flux in each chop saveset, and flag bad sets so
they are not co-added in TPREPARE. Raw T-ReCS and Michelle data frames can be
viewed and, should the user desire, examined interactively using the TVIEW and
MVIEW routines, respectively. These tasks allow the user to examine each data
image (using IMEXAMINE) and flag bad chop savesets (T-ReCS) or nods (Michelle).
The TPREPARE and MPREPARE tasks collapse the T-ReCS chop frames and reorganize
the Michelle data so the output has the same file structure.
Following the processing with the *PREPARE scripts, the data can be viewed and
examined using MIVIEW. Like TVIEW and MVIEW, MIVIEW allows the user to examine
each frame (using IMEXAMINE), derive statistics, and flag nods as BAD for
exclusion from the final data coaddition.
For CHOP-NOD and NOD mode data the nod sets for each data image are collapsed
into a single [320,240] image with the MISTACK or MIREGISTER tasks. Both of
these routines require that the input frames have been prepared either with
TPREPARE or MPREPARE. MISTACK merely averages (or sums, if the combine
task parameter is changed to "sum" from the default of "average") each frame by
co-adding the signal from each nod position and dividing by the number of
frames. MIREGISTER combines the nod frames by using the XREGISTER task
in IRAF to shift the frames before they are averaged (again, or summed if
combine is set to "sum").
The default behavior is to average the images, whether they are simply added
as is or whether they are registered first. In that case the effective
exposure time remains the same as for an individual image in the raw data,
the on-source time per saveset for T-ReCS and the on-source time per nod
position for Michelle. An alternative is to form the sum of the images
rather than the average, in which case the total on-source exposure time
applies to the final image. This behavior is selected by the combine
task parameter in MIREDUCE, TPREPARE, MISTACK, and MIREGISTER. This in
turn is passed as the combine parameter for the IMCOMBINE task which
is used to combine the images. The task MIPSF is used to measure the FWHM and
the Strehl value for either Michelle or T-ReCS images.
All of the afore-mentioned reduction steps for each T-ReCS or Michelle raw data
file can be done with one call to the MIREDUCE task. By default, MIREDUCE
will identify if a file is from T-ReCS or Michelle, run TPREPARE or MPREPARE
on the image, and, if needed on NOD or CHOP-NOD data, it will combine the nod
positions using MISTACK. The fl_view and fl_background keywords
can be flagged if the user wishes to interactively view the images or test
the background statistics (the latter for T-ReCS data, only). At the present
time, a bad pixel mask is not applied to the data in the MIREDUCE call, but the
fl_flat keyword can be set and a flat field image defined for the
MIFLAT routine to apply a flat-field correction to T-ReCS images (only). The
stackoption parameter can be set to "stack" or "register", and the
MIREDUCE routine will call the appropriate combining script, MISTACK or
MIREGISTER to combine the data. The MICLEAN task can be used to remove pattern
noise and channel-channel offsets from the combined images, if desired. The task
MISTDFLUX is used to find the in-band filter flux density of a standard star.
Imaging polarimetry observations (see below) cannnot be reduced by the
MIREDUCE task. It applies only to regular imaging observations.
Most of the midir reduction tasks can be called with multiple data images in a list
format (i.e., "@inlist"). The tasks have the option of setting a
rawpath parameter to point to the appropriate directory where
the raw data resides. The output filename can be defined for single
images, in a list for multiple images ("@outlist"), or an output prefix
can be prefixed to the name of the input file. For additional information
and syntax for each step of the midir data reduction package, see the help
file included for each task.
For spectroscopy observations the initial processing to produce a single
stacked or registered image is carried out in the same way as for imaging
observations. From there one needs to carry out a series of steps to
extract and calibrate the spectrum, assuming that there is also a spectrum
of a standard star available to use in the analysis. These steps use
various tasks in the GNIRS package. The recommended processing
steps are outlined in the next section. Some spectroscopy
tasks are currently available in the MIDIR package, all of which start with an
MS prefix. These include MSDEFRINGE, MSREDUCE, MSSLICE, MSFLATCOR, MSTELLURIC
and MSABSFLUX
If one wishes to produce a calibrated spectrum one has to run all the steps in the reduction except the last one (running MSTELLURIC) for both the observation of the science target and the observation of a standard star. Thus steps (1) to (8) below would normally be done twice to produce wavelength calibrated spectra for the two targets, and then step (9) would be done with these two spectra to produce the final calibrated spectrum of the object.
To carry out a normal spectroscopic reduction the following steps are required:
http://www.gemini.edu/sciops/instruments/midir-resources/data-reduction