From AAS NOVA: “Prepping for Even Bigger Data in the Era of Interferometry”



26 April 2019
Kerry Hensley

ESO/NRAO/NAOJ ALMA Array in Chile in the Atacama at Chajnantor plateau, at 5,000 metres

The Atacama Large Millimeter/submillimeter Array (ALMA), with its collection of 66 radio dishes, is one of several telescope arrays capable of capturing images of the Sun at radio wavelengths. [Y. Beletsky (LCO)/ESO]

Interferometric arrays collect massive amounts of information, leaving astronomers with a happy problem: too much data! How can we handle mountains of data in an efficient way?

SKA Murchison Widefield Array, Boolardy station in outback Western Australia, at the Murchison Radio-astronomy Observatory (MRO)

One of many tiles comprising the Murchison Widefield Array (MWA). Radio interferometric arrays like MWA generate vast amounts of data. [Dr. John Goldsmith/Celestial Visions]

Too Much of a Good Thing?

Astronomers have come a long way from the early days of manually cataloging stars and sketching sunspots by hand. Even though today’s data sets are larger and more complex, many astronomers still manually calibrate and process their data.

This hands-on data processing won’t always be feasible, though; interferometry — the process of linking together tens to thousands of telescopes or antennae to produce images with ever-finer angular resolution — generates far more data than humans could hope to handle manually. Just one minute’s worth of data from the Murchison Widefield Array (MWA), a radio interferometer made up of 4,096 antennae, yields roughly 10,000 images!

With the number of interferometers increasing, we’ll need to be smart about how we process all that data to minimize computing hours while maximizing the quality of the output. Among the many detectors requiring novel data-processing techniques is the planned Square Kilometer Array (SKA), which will comprise a million antennae and 2,000 radio telescopes. How can we get a handle on all this data without getting too hands-on?

Australian Square Kilometre Array Pathfinder (ASKAP) is a radio telescope array located at Murchison Radio-astronomy Observatory (MRO) in the Australian Mid West. ASKAP consists of 36 identical parabolic antennas, each 12 metres in diameter, working together as a single instrument with a total collecting area of approximately 4,000 square metres.

SKA Hera at SKA South Africa

SKA Meerkat telescope(s), 90 km outside the small Northern Cape town of Carnarvon, SA

An illustration of how increasing numbers of detectors are included in the model of the target for self-calibration. The first step includes only the blue detectors near the center, and subsequent steps add the red, teal, black, and yellow detectors to increase the complexity of the model. [Mondal et al. 2019]

Dealing with Data Pileup

To tackle this problem, a team led by Surajit Mondal (Tata Institute of Fundamental Research, India) developed an automated processing pipeline for interferometric data — the Automated Imaging Routine for Compact Arrays for the Radio Sun (AIRCARS). They focused on processing solar radio images, which need to capture a huge dynamic range — from extremely bright active regions to faint, wispy filaments.

One of the challenges in radio interferometry is removing the effects of instrumental artifacts and the plasma in Earth’s atmosphere. Most radio interferometry data are corrected with a self-calibration process that treats the instrumental artifacts and the brightness of the target as free parameters and iteratively minimizes the difference between the observations and a model of the target.

AIRCARS works especially well when applied to a compact array — one with many detectors clustered in the center and fewer near the outskirts. This configuration allows the pipeline to start with relatively little information about the target from just a few central detectors and gradually build a complex model of the target to be used in its self-calibration routine.

An example of the improvement of the dynamic range of MWA images through the self-calibration process. The number of iterations increases from top to bottom and left to right. The dashed circle indicates the location of the Sun’s disk. [Mondal et al. 2019]

AIRCARS in Our Future

In their tests on MWA data, the authors find that AIRCARS is capable of capturing a dynamic range up to 100,000:1 — a huge improvement over previous processing methods.

Mondal and collaborators note that AIRCARS can be configured to attain the maximum possible dynamic range without constraints on computing time, or to accept user-imposed time limits to rapidly process large amounts of data, depending on the user’s computational requirements.

Because the pipeline needs no human supervision, astronomers can take a step back from processing the vast amount of incoming data and focus instead on the exciting science we can do with interferometry.


“Unsupervised Generation of High Dynamic Range Solar Images: A Novel Algorithm for Self-calibration of Interferometry Data,” Surajit Mondal et al 2019 ApJ 875 97.

See the full article here .


Please help promote STEM in your local schools.

Stem Education Coalition


AAS Mission and Vision Statement

The mission of the American Astronomical Society is to enhance and share humanity’s scientific understanding of the Universe.

The Society, through its publications, disseminates and archives the results of astronomical research. The Society also communicates and explains our understanding of the universe to the public.
The Society facilitates and strengthens the interactions among members through professional meetings and other means. The Society supports member divisions representing specialized research and astronomical interests.
The Society represents the goals of its community of members to the nation and the world. The Society also works with other scientific and educational societies to promote the advancement of science.
The Society, through its members, trains, mentors and supports the next generation of astronomers. The Society supports and promotes increased participation of historically underrepresented groups in astronomy.
The Society assists its members to develop their skills in the fields of education and public outreach at all levels. The Society promotes broad interest in astronomy, which enhances science literacy and leads many to careers in science and engineering.

Adopted June 7, 2009