You are here

Data Processing Software

Content owned by klabrie

Gemini Data Reduction Software

Gemini offers data reduction software for its facility instruments. The Gemini IRAF package and the DRAGONS platform are the official data reduction software supported by Gemini.

For the next several years we will be transitioning from the IRAF platform to our new Python-based DRAGONS platform. As time goes by, more and more instruments and modes will be supported by DRAGONS. During this transition, it is possible that users will require both platforms depending on the data they have obtained.

  • Latest version of Gemini IRAF:  v1.15  (June 2022)
  • Latest version of DRAGONS:  v3.0.3  (July 2022)

Which platform should I use?

Gemini IRAF currently still support all the instruments and modes. However for any imaging data from current instruments, we recommend using DRAGONS. DRAGONS currently support imaging data reduction, but does not support any spectroscopy, for now.

GMOS Imaging DRAGONS 
NIRI Imaging DRAGONS 
GNIRS Keyhole Imaging DRAGONS
Flamingos-2 Imaging DRAGONS 
GSAOI Imaging DRAGONS,  plus Disco-Stu
GMOS longslit spectroscopy Gemini IRAF for science reduction, DRAGONS for quicklook
Any other spectroscopy Gemini IRAF
Decommisioned Instruments Gemini IRAF

Download and Install

DRAGONS and Gemini IRAF, support software and dependencies, are all installed the same way using the Anaconda distribution system. We also make use of the STScI Astroconda suite to install common astronomy tools, like ds9.

Requirements

Gemini IRAF
  • Python 3.7 
  • 32-bit compatibility libraries
  • Linux, equilvalent to CentOS 7 and above
  • Or Mac OS X 10.10 to 10.14.0

To run Gemini IRAF on Mac OS X 10.14.1 and above, you will need to use a virtual machine.  See https://gemini-iraf-vm-tutorial.readthedocs.io for instructions.

DRAGONS
  • Python 3.7
  • Linux, equivalent to CentOS 7 and above
  • Or Mac OS X 10.10 and above
  • Intel Anaconda

Please note starting DRAGONS v3.0, Python 2.7 is no longer supported.  Python 3 is now required.

  

See the installation instructions here:

       DOWNLOAD
 


Announcements

PyRAF on Python 3
August 23, 2022

Following some collaboration with the "IRAF Community" project earlier this year, we are pleased to make available PyRAF 2.2.1, the first version to pass all our testing on Python 3. This runs roughly as fast as the Python 2 version, thanks to improvements by Ole Streicher.

This version is already included on the new IRAF VM for Apple machines (see yesterday's announcement). On Linux, it can be installed by specifying "python=3.7" instead of "python=2.7" in the "conda create" command -- see:

https://www.gemini.edu/observing/phase-iii/understanding-and-processing-data/data-processing-software/download-latest

Linux builds are also available for Python 3.8 & 3.9, but using 3.7 allows you to install it alongside the current public DRAGONS version. PyRAF 2.2.1 is NOT compatible with Python 2, where the latest version will remain at 2.1.15.

New IRAF VM for MacOS
August 22, 2022

A new virtual machine implementation for running IRAF on MacOS 10.14+ is now available. You can find updated instructions at:

https://gemini-iraf-vm-tutorial.readthedocs.io

This should provide a reliable and relatively convenient way of running Gemini IRAF on recent Apple computers with an M1/M2 processor (though M2 has not been tested yet). However, this comes with an order-of-magnitude speed penalty, because of the need to emulate an Intel CPU on ARM64, which is unavoidable without having 64-bit ports of Gemini IRAF and its dependencies (such as STSDAS). On Apple machines with an Intel CPU, the virtualization overhead is much smaller.

DRAGONS v3.0.3 Patch Release Available
July 8, 2022

A patch release of DRAGONS is available. This patch adds better handling of the GMOS-S data following the January failure of amplifier #5. For that data, the whole amplifier is automatically masked as bad pixels ensuring that those pixels are not used in calculations. The patch also includes bug fixes, small improvements, and fixes to the documentation.

For a complete list of changes since v3.0.1 see:
    https://dragons.readthedocs.io/en/v3.0.3/changes.html

Gemini IRAF v1.15 with new support for Flamingos 2 MOS
June 8, 2022

A new version of Gemini IRAF v1.15 with new support for Flamingos 2 MOS (Multi-Object Spectroscopy) data is available.

Along with Flamingos 2 MOS support, the new release includes the changes added in the v1.14 patch released two months ago, as well as several other bug fixes.

The package is distributed via conda. See the installation instructions page for details.

Revisions relative to v1.14 are summarized here: gemini_v115_rev.txt

Gemini IRAF Patch release for v1.14 to support new Flamingos 2 fitlers
April 19, 2022

A patch release for Gemini IRAF v1.14 is available.

The replacement of two filters and the reshuffling of another in Flamingos 2 necessitated additions and modifications to lookup tables and modifications to the code. This patch release contains only the affected files. The patch is to be applied on-top of an already installed Gemini IRAF v1.14 package.

Installation instructions and content of the patch are available in gemini_v114_patch1.txt.

Download: gemini_v114_patch1.tar.gz

DRAGONS v3.0.1 Patch Release Available
December 6, 2021

A patch release of DRAGONS is available. It includes bug fixes, small improvements, and fixes to the documentation.

For a complete list of changes since v3.0.0 see:
    https://dragons.readthedocs.io/en/v3.0.1/changes.html

DRAGONS v3.0 Released
October 12, 2021

A new version of DRAGONS has been released.  This version offers improvements and fixes to the imaging support.  New is this version is support for GMOS longslit spectroscopy for quicklook purposes.  This new spectroscopy support is not approved for science quality reduction, but can be use for "quicklook" inspection of new data.   A quicklook reduction is intended to check the overall quality of the data, such as confirming the presence of the target(s) and whether the expected S/N has been achieved. It is particularly useful for assessing ToO observations quickly.   We are continuing to develop DRAGONS and the next major release will support science-quality reduction of GMOS longslit data, with optional interactive tools for key reduction steps as well as additional routines such as flagging cosmic rays and performing the slit function correction.

For a complete list of changes since v2.1 see:
    https://dragons.readthedocs.io/en/v3.0.0/changes.html

Virtual machine image for running IRAF under recent MacOS releases
May 19, 2020

A CentOS 7 virtual machine image (OVA file) is now available to facilitate running Astroconda IRAF under MacOS 10.15+, which no longer supports running the necessary 32-bit binaries natively. This comes with Anaconda 2019.10, Gemini IRAF 1.14, DRAGONS 2.1.0 and other packages from Astroconda pre-installed. Users of MacOS 10.14 affected by the Tk bug that causes a desktop session logout when displaying graphics may also want to install this guest distribution as a workaround.

Please see instructions at https://gemini-iraf-vm-tutorial.readthedocs.io.

New DRAGONS Patch Release
April 20, 2020

A bug fix release of DRAGONS is now available. In release 2.1.1, we have fixed bugs and typos found by the users and ourselves since the initial release. We have also added compatibility with astropy v4. If you already have DRAGONS installed, you can update by doing conda install dragons=2.1.1. If you need to install DRAGONS for the first time, please see the Download and Installation Instructions.

This update also contains an update of disco_stu.

For information and tutorials on DRAGONS, see the "DRAGONS Information" section.

DRAGONS First Public Release!
October 31, 2019

It is with great delight that we are announcing the first public release of Gemini's new Python-base data reduction platform, DRAGONS, Data Reduction for Astronomy from Gemini Observatory North and South. This project has been many years in the making. DRAGONS offers a more streamlined approached to the data reduction of Gemini data, compared to the Gemini IRAF package.

This release, version 2.1.0, supports imaging reduction only, for the current facility instruments. For spectroscopy data, please continue to use Gemini IRAF for the time being. Work is on-going regarding spectroscopy-support in DRAGONS but it will be a while before it is publicly available for science-quality reduction.

To download DRAGONS:   Download and Installation Instructions

For information and tutorials on DRAGONS, see the "DRAGONS Information" section.


IMPORTANT: MacOS 10.14.6 and 10.15 incompatibilities with data reduction software
October 11, 2019

As of this week's v10.15 release, MacOS is no longer capable of running the 32-bit Astroconda IRAF distribution needed by Gemini IRAF. For the time being, Gemini IRAF users on Apple machines are advised to continue using MacOS 10.14 or earlier, or to install Astroconda in a virtual machine with a compatible OS. Gemini will look into providing a ready-made VM image to help with this while we are migrating our data reduction tools to Python. Furthermore, MacOS 10.14.6 suffers from a bug that can cause a desktop session logout when attempting to display plots or images with PyRAF, DS9, Matplotlib or other software that uses Tk. We suggest that PyRAF users on 10.14 avoid updating their OS until such time as this problem is resolved by Apple and/or we can determine a reliable workaround (check here for further announcements). IRAF CL is unaffected, but note that we are no longer testing it routinely and are aware of occasional failures with Gemini IRAF.

For older announcements, see the Announcements page.


Gemini IRAF Information

Use Gemini IRAF to reduce Gemini facility instrument data, including decommissioned instruments. Get to the Gemini IRAF page for a Description of the package, details of its content, and list of releases and revision history.

However, for imaging data from currently active instruments, we recommend the use of DRAGONS (below) instead.

DRAGONS Information

Use DRAGONS to reduce imaging data from the currently active Gemini facility instruments: GMOS, NIRI, Flamingos-2, GSAOI, GNIRS (keyhole). For spectroscopy data, you must use Gemini IRAF.  However, now you can obtain a quicklook reduction of GMOS longslit spectroscopy data.  A quicklook reduction is intended to check the overall quality of the data, not to be used as a science quality reduction.

The DRAGONS documentation is hosted on readthedocs.org.

DRAGONS Documentation

Science quality verification for DRAGONS imaging modes is discussed in this report.

The primary reference to be cited by users of DRAGONS is:

        K. Labrie et al. ADASS XXVII, ASP Conf. Ser. 522, eds. P. Ballester, J. Ibsen, M. Solar & K. Shortridge, p. 321

View the recording of the demo of DRAGONS 3.1 for the science quality reduction of GMOS longslit data to see what's coming next...

Other Gemini Data Reduction Software

  • Disco-Stu - Distortion Correction and Stacking Utility

    Disco-Stu is a software package for GSAOI images. This standalone package, written in python, will align and stack images that have already been processed by DRAGONS or the Gemini IRAF gareduce task. The current release is v1.3.7.
    To install:

    conda install disco_stu

    Note that disco_stu is installed by default when doing a full install of Gemini IRAF and DRAGONS with conda.

    Documentation

    Disco-Stu Manual
  • gemini_calmgr - Gemini Local Calibration Manager
     

    The Gemini Local Calibration Manager, gemini_calmgr, is a python package that DRAGONS uses to associate data to reduce with the best processed calibration available. The Local Calibration Manager handles a lightweight database where the users can upload information about the processed calibration they have produced. DRAGONS can then use the calibration association rules to identify and retrieve automatically the master bias, the master flat, etc. that the data requires. The calibration association rules are the same as those used in the Gemini Observatory Archive.

    The package gemini_calmgr is installed automatically as part of a DRAGONS installation.


Need help?

To get help with Gemini IRAF or DRAGONS, please use the Gemini Helpdesk system. Use the Gemini IRAF category even for DRAGONS questions.

     Submit Helpdesk Ticket

If you find a bug with DRAGONS, please consider reporting on the DRAGONS Github issues portal. You will need a Github account.

     DRAGONS Code Issues

For comment, suggestions, and general feedback on DRAGONS, please add a comment to this decidicated post on the Gemini Data Reduction User Forum.

     DRAGONS Feedback Forum