1.1 New data notification Emails

We send automatic emails in the morning following observing to notify PIs and/or Co-Is that new data were taken for their program. This facility is provided by our internal data handling systems. Email addresses to send notifications to are updated from the Observing Database daily. If you would like to add email addresses to receive the notifications, please simply add them to the "PI / PC Email" field in the top level of your Gemini Science Program in the OT and store your program to the database.

Email notifications for data taken during the night go out at 8:00am in the morning following the observing night. The email contains a summary table of all the relevant data files.

These Emails are triggered purely on the presence of science data files from night time observing. These data will not generally have been Quality Assessed, or their metadata validated, at the time the Email notifications go out. Data Quality and FITS header information in these files will be updated as QA proceeds. Data should be retrieved from the Gemini Observatory Archive in the normal manner.

1.2 Gemini Observatory Archive

The Gemini Observatory Archive (GOA) located at https://archive.gemini.edu is the primary conduit to obtain data from Gemini Observatory. The Help and About pages also linked at the top of the archive page give more information.

The GOA provides a simple yet powerful interface to search for and download science data, calibrations and observation logs both for Gemini program PIs and Co-Is, and also for people searching for Gemini data by observation details such as instrument configuration, sky coordinates, observing date etc. Paul Hirst is the GOA project lead.

Data Flow and Distribution

All Gemini data distribution to users is via the archive web interface. Raw data are transferred to the archive automatically during observing, and each file is typically available for download from the archive within a minute or two of the observation completing. Data Quality Assessment proceeds over the following days and the archive is updated if QA states or other metadata are updated.

The archive will automatically find calibration data associated with search results, and presents this in the associated calibrations tab in the search form. Additionally, it's possible to search for calibration data directly in the search form simply by setting the search fields appropriately.

Gemini metadata, including sky coordinates, target name and instrument configuration are in general immediately world public, and the full FITS headers of proprietary data files can be viewed by any archive user. This is a deliberate policy and allows people to avoid writing proposals that would duplicate observations whose data are still proprietary. The actual pixel values are proprietary and only made available to the project they were observed for, typically for 12 months (18 prior to semester 2016A). A few projects (for example some of the exoplanet campaign projects with NICI and GPI) have been granted a proprietary metadata status where certain items of metadata (sky coordinates and target name), and the full FITS header text, assume the same proprietary privacy restrictions as the pixel values.

Downloading proprietary data

The archive is open to the public to search for data, and download any data which is no longer proprietary. If you are a gemini PI or Co-I and would like to access proprietary data for your program you will need to complete the following steps:

• Create an account on the archive
• Register your Gemini program with your archive account. To do this you need the program ID and the ODB key you were given in your time award notification email.

More details on how do to this are provided in the User Accounts section of the archive help page

Getting data automatically (e.g., as part of reduction pipeline) 

Please see the GOA API documentation, which gives some example python code for acessing the APIs. You can also find an example of a script from the AAT Observational Techniques Workshop 2016.

Reduced data in the GOA

The archive currently contains mostly raw data. Some data processed by the observatory or by visitor instrument teams is available in the archive. Notably, this includes GMOS bias frames and imaging twilight flat fields, along with MOS preimaging data. Other processed data has traditionally been produced and archived on a somewhat ad-hoc basis. We are working to provide automatic DRAGONS reduced data products in the archive and expect these to strt becoming available in late 2025.

Processed (ie reduced) data files in the archive generally have a processing classification specified as Quick-Look or Science-Quality. Quick-Look data are intended for doing a quick evaluation of those observation(s). Science-Quality data are intended to be suitable for science use, however in most cases they have been generated automatically and have not been reviewed by an expert. There will likely be cases where the automatic reduction does not provide good results and the onus is on the end user to verify that the data and reduction meets their requirements. They are reduced in a general manner which we believe is applicable to most science cases, however some science cases will require that the data be re-reduced with reduction techniques specific to the particular scientific use case. Our working definitions of these terms are in the archive documentation here.