From Science Node: “XSEDE cuts through the noise”

Science Node bloc
Science Node

06 June, 2017
Alisa Alering

Courtesy LIGO; Caltech; MIT; Sonoma State; Aurore Simonnet.

Over two billion years ago, when multicellular life had only just begun to evolve on Earth, two black holes collided and merged to form a new black hole.

With a mass 49 times that of our sun, the massive collision set off ripples in space-time that radiated from the event like waves from a stone thrown into a pond. Predicted by Albert Einstein in 1916 and known as gravitational waves, those ripples are still traveling.

Surf’s up! The Extreme Science and Engineering Discovery Environment (XSEDE) provides the HPC resources required to pluck gravitational waves from the noise found on LIGO detectors. Courtesy XSEDE.

Able to pass through dust, matter, or anything else without being distorted, gravitational waves carry unique information about cosmic events that can’t be obtained in any other way. When the waves reach Earth, they give astrophysicists a completely new way to explore the universe.

The first such waves were detected on September 14, 2015 by the Laser Interferometer Gravitational-Wave Observatory (LIGO) Scientific Collaboration. In the months since, two more gravitational wave events have been confirmed, one in December 2015 and the most recent on January 4, 2017.

Caltech/MIT Advanced aLigo Hanford, WA, USA installation

Caltech/MIT Advanced aLigo detector installation Livingston, LA, USA

Cornell SXS, the Simulating eXtreme Spacetimes (SXS) project

Gravitational waves. Credit: MPI for Gravitational Physics/W.Benger-Zib

ESA/eLISA the future of gravitational wave research

Signal from noise

When gravitational waves pass by, they change the distance between objects. The change is so infinitesimal that it can’t be felt, or seen with a microscope. But incredibly sensitive scientific instruments—interferometers—can detect a change that is a thousand times smaller than a proton.

The LIGO Scientific Collaboration, a body of more than 1,000 international scientists who collectively perform LIGO research, operates two interferometers located over 2000 miles apart in Washington and Louisiana, USA.

Despite the sensitivity of the instruments, it’s not easy to detect a gravitational wave. When a signal is received, scientists must determine what it means and how likely it is to be noise or a real gravitational wave. Making that determination requires high-performance computing.

Since 2013, LIGO has collaborated with the Extreme Science and Engineering Discovery Environment (XSEDE), a National Science Foundation (NSF)-funded cyberinfrastructure network that includes not just high-performance computing systems but also experts who help researchers move projects forward.

Better, faster, cheaper

In order to validate the discovery of a gravitational wave, researchers measure the significance of the signal by calculating a false alarm rate for the event.

Making waves, taking names. The top part of the animation shows two black holes orbiting each other until they merge, and the lower part shows the two distinct gravitational waves emitted. Thanks to supercomputers at TACC and SDSC, researchers can pick out these waves from other detector noise. Courtesy Simulating eXtreme Spacetimes collaboration.

TACC Maverick HP NVIDIA supercomputer

TACC Lonestar Cray XC40 supercomputer

Dell Poweredge U Texas Austin Stampede Supercomputer. Texas Advanced Computer Center 9.6 PF

TACC HPE Apollo 8000 Hikari supercomputer

TACC Maverick HP NVIDIA supercomputer

SDSC Triton HP supercomputer

SDSC Gordon-Simons supercomputer

SDSC Dell Comet supercomputer

Once confirmed, further supercomputer analysis is used to extract precise estimates of the physical properties of the event, including the masses of the colliding objects, position, orientation, and distance from the Earth, carefully checking millions of combinations of these characteristics and testing how well the predicted waveform matches the signal detected by LIGO.

To draw larger conclusions about the nature of black holes requires careful modeling based on the received data. Each simulation can take from a week to one month to complete, depending upon the complexity.

Such intensive data analysis requires large scale high-throughput computing with parallel workflows at the scale of tens of thousands of cores for long periods of time. LIGO has been allocated millions of hours on XSEDE’s high-performance computers, including Stampede at the Texas Advanced Computing Center (TACC) and Comet at the San Diego Supercomputer Center (SDSC).

Over the first year of XSEDE’s collaboration with LIGO, XSEDE worked to increase the speed of the applications, making them 8-10x faster on average.

“The strategic collaboration between the two NSF-funded projects allows for accelerated scientific discovery which also translates into cost-savings for LIGO on the order of tens of millions of dollars so far,” says Pedro Marronetti, Gravitational Physics program director at the NSF.

Waves of the future

LIGO plans to upgrade its observatories and improve the sensitivity of its detectors before the next observational period begins in late 2018. LIGO predicts that once its observatories reach their most sensitive state, they may able to detect as many as 40 gravitational waves per year.


More instruments like LIGO will soon be listening for waves around the world in Italy, Japan, and India. Scientists also hope to place interferometers in orbit in order to avoid interference from Earth noise.

And that means much more computing power will be required to verify the signals and extract information about the nature and origins of our universe.

See the full article here .

Please help promote STEM in your local schools.

Stem Education Coalition

Science Node is an international weekly online publication that covers distributed computing and the research it enables.

“We report on all aspects of distributed computing technology, such as grids and clouds. We also regularly feature articles on distributed computing-enabled research in a large variety of disciplines, including physics, biology, sociology, earth sciences, archaeology, medicine, disaster management, crime, and art. (Note that we do not cover stories that are purely about commercial technology.)

In its current incarnation, Science Node is also an online destination where you can host a profile and blog, and find and disseminate announcements and information about events, deadlines, and jobs. In the near future it will also be a place where you can network with colleagues.

You can read Science Node via our homepage, RSS, or email. For the complete iSGTW experience, sign up for an account or log in with OpenID and manage your email subscription from your account preferences. If you do not wish to access the website’s features, you can just subscribe to the weekly email.”