Automated data reduction workflows for astronomy
By W. Freudling, M. Romaniello, D. Bramich, P. Ballester, V. Forchì, C. E. Garcia-Dablo, S. Moehler, M. Neeser. 2013
454 citations recorded by the source corpus.
Data from complex modern astronomical instruments often consist of a large number of different science and calibration files, and their reduction requires a variety of software tools. The execution chain of the tools represents a complex workflow that needs to be tuned and supervised, often by individual researchers that are not necessarily experts for any specific instrument. The efficiency of data reduction can be improved by using automatic workflows to organise data and execute the sequence of data reduction steps. To realize such efficiency gains, we designed a system that allows intuitive representation, execution and modification of the data reduction workflow, and has facilities for inspection and interaction with the data. The European Southern Observatory (ESO) has developed Reflex, an environment to automate data reduction workflows. Reflex is implemented as a package of customized components for the Kepler workflow engine. Kepler provides the graphical user interface to create an executable flowchart-like representation of the data reduction process. Key features of Reflex are a rule-based data organiser, infrastructure to re-use results, thorough book-keeping, data progeny tracking, interactive user interfaces, and a novel concept to exploit information created during data organisation for the workflow execution. Reflex includes novel concepts to increase the efficiency of astronomical data processing. While Reflex is a specific implementation of astronomical scientific workflows within the Kepler workflow engine, the overall design choices and methods can also be applied to other environments for running automated science workflows.
Subfields: Computational Mechanics, Information Systems and Management, Instrumentation
View Paper: https://www.semanticscholar.org/paper/ad4f067304551c20b50a3c0f7e02e3e9d6946003
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Computational Mechanics
In Instrumentation
Citations in the corpus
- A catalogue of masses, structural parameters, and velocity dispersion profiles of 112 Milky Way globular clusters (2018, methodology): “In order to derive radial velocities from FORS2 spectra, we reduced the raw data with the help of the ESO Reflex pipeline (Freudling et al. 2013).”
- SPHERE: the exoplanet imager for the Very Large Telescope (2019, methodology): “In addition, the more recent Reflex system (Freudling et al. 2013) is now available to execute recipes and reflex workflows are now part of the SPHERE pipeline package11.”
- X-shooter reveals powerful outflows in z ∼ 1.5 X-ray selected obscured quasi-stellar objects (2014, methodology): “MB, AB and GC acknowledge useful discussion with N. Neumeyer, A. Modigliani, M. Romaniello and the ESO USD team, for Reflex support, and are grateful to A. Mucciarelli and S. Piranomonte for help on X-shooter data reduction and flux calibration.”
- A Multimessenger Picture of the Flaring Blazar TXS 0506+056: Implications for High-energy Neutrino Emission and Cosmic-Ray Acceleration (2018, methodology): “22.1), X-shooter pipeline (v.2.9.3; Goldoni et al. 2006; Modigliani et al. 2010), Reflex (Freudling et al. 2013), Molecfit (Kausch et al. 2015; Smette et al. 2015), Fermi Science Tools (https://fermi. gsfc.nasa.gov/ssc/data/analysis/software/), SOPHIA (Mücke et al. 2000).”
- Characterizing the Fast Radio Burst Host Galaxy Population and its Connection to Transients in the Local and Extragalactic Universe (2021, methodology): “The individual frames were bias subtracted and flatfielded using the ESOReflex package (Freudling et al. 2013).”