DiFX: A Software Correlator for Very Long Baseline Interferometry Using Multiprocessor Computing Environments

By A. Deller, Steven J. Tingay, M. Bailes, C. West. 2007

334 citations recorded by the source corpus.

We describe the development of an FX‐style correlator for very long baseline interferometry (VLBI), implemented in software and intended to run in multiprocessor computing environments, such as large clusters of commodity machines (Beowulf clusters) or computers specifically designed for high‐performance computing, such as multiprocessor shared‐memory machines. We outline the scientific and practical benefits for VLBI correlation, these chiefly being due to the inherent flexibility of software and the fact that the highly parallel and scalable nature of the correlation task is well suited to a multiprocessor computing environment. We suggest scientific applications where such an approach to VLBI correlation is most suited and will give the best returns. We report detailed results from the Distributed FX (DiFX) software correlator running on the Swinburne supercomputer (a Beowulf cluster of ∼300 commodity processors), including measures of the performance of the system. For example, to correlate all Stokes products for a 10 antenna array with an aggregate bandwidth of 64 MHz per station, and using typical time and frequency resolution, currently requires an order of 100 desktop‐class compute nodes. Due to the effect of Moore’s law on commodity computing performance, the total number and cost of compute nodes required to meet a given correlation task continues to decrease rapidly with time. We show detailed comparisons between DiFX and two existing hardware‐based correlators: the Australian Long Baseline Array S2 correlator and the NRAO Very Long Baseline Array correlator. In both cases, excellent agreement was found between the correlators. Finally, we describe plans for the future operation of DiFX on the Swinburne supercomputer for both astrophysical and geodetic science.

Subfields: Aerospace Engineering, Astronomy and Astrophysics, Environmental Engineering

View Paper: https://www.semanticscholar.org/paper/5b77382d10ae1488910bf8528b39f57e8dded18a

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Aerospace Engineering

In Astronomy and Astrophysics

In Environmental Engineering

Citations in the corpus

DiFX-2: A More Flexible, Efficient, Robust, and Powerful Software Correlator (2011, methodology): “In DiFX2 an appropriate phase compensation is made after channelization in the same operation as fractional sample correction (see Deller et al. 2007).”
Science with ASKAP (2008, methodology): “The capabilities for e-VLBI in the form of PAMHELA are currently under development as a collaboration between ATNF and the Swinburne University of Technology, who provide a software correlator [76].”
ON THE NATURE OF THE LOCAL SPIRAL ARM OF THE MILKY WAY (2013, methodology): “All data were processed on the DiFX2 correlator in Socorro, NM (Deller et al. 2007).”
The Murchison Widefield Array: The Square Kilometre Array Precursor at Low Radio Frequencies (2012, methodology): “The recorded voltages can be correlated using existing software correlation solutions, such as DiFX (Deller et al. 2011, 2007).”
FERMI LARGE AREA TELESCOPE VIEW OF THE CORE OF THE RADIO GALAXY CENTAURUS A (2010, methodology): “…were part of the TANAMI monitoring of a radio and γ-ray selected sample of 65 blazars266 at 8.4 GHz and 22.3 GHz with observations approximately every two months.267 TANAMI data are correlated on the DiFX software correlator (Deller et al. 2007) at Curtin268 University in Perth, Western Australia.”
The limb-brightened jet of M87 down to the 7 Schwarzschild radii scale (2018, methodology): “The 2015 data were correlated with the DiFX correlator (Deller et al. 2007) at the MPIfR.”
The Karl G. Jansky Very Large Array Sky Survey (VLASS). Science Case and Survey Design (2019, methodology): “…(NRL), VLITE uses splitters, dedicated samplers and spare fibers to make available 64 MHz of bandwidth covering 320–384 MHz from the new broadband low frequency receivers (Clarke et al. 2011) to a custom-designed real-time DiFX correlator producing full polarization products (Deller et al. 2007).”