From Science Node: “Putting neutrinos on ice”

Science Node bloc
From Science Node

29 Aug, 2018
Ken Chiacchia
Jan Zverina

IceCube Collaboration/Google Earth: PGC/NASA U.S. Geological Survy Data SIO,NOAA, U.S. Navy, NGA, GEBCO Landsat/Copernicus.

Identification of cosmic-ray source by IceCube Neutrino Observatory depends on global collaboration.

Four billion years ago—before the first life had developed on Earth—a massive black hole shot out a proton at nearly the speed of light.

Fast forward—way forward—to 45.5 million years ago. At that time, the Antarctic continent had started collecting an ice sheet. Eventually Antarctica would capture 61 percent of the fresh water on Earth.

Thanks to XSEDE resources and help from XSEDE Extended Collaborative Support Service (ECSS) experts, scientists running the IceCube Neutrino Observatory in Antarctica and their international partners have taken advantage of those events to answer a hundred-year-old scientific mystery: Where do cosmic rays come from?

U Wisconsin IceCube neutrino observatory

U Wisconsin ICECUBE neutrino detector at the South Pole

IceCube employs more than 5000 detectors lowered on 86 strings into almost 100 holes in the Antarctic ice NSF B. Gudbjartsson, IceCube Collaboration

Lunar Icecube

IceCube DeepCore annotated

IceCube PINGU annotated

DM-Ice II at IceCube annotated

Making straight the path

First identified in 1912, cosmic rays have puzzled scientists. The higher in the atmosphere you go, the more of them you can measure. The Earth’s thin shell of air, scientists came to realize, was protecting us from potentially harmful radiation that filled space. Most cosmic ray particles consist of a single proton. That’s the smallest positively charged particle of normal matter.

Cosmic ray particles are ridiculously powerful. Gonzalo Merino, computing facilities manager for the Wisconsin IceCube Particle Astrophysics Center at the University of Wisconsin-Madison (UW), compares the force of a proton accelerated by the LHC, the world’s largest atom-smasher, as similar to the force of a mosquito flying into a person.


CERN map


CERN LHC particles

By comparison, the “Oh-My-God” cosmic ray particle detected by the University of Utah in 1991 hit with the force of a baseball flying at 58 miles per hour.

Because cosmic-ray particles are electrically charged, they would be pushed and pulled by every magnetic field they encounter along the way. Cosmic rays would not travel in a straight line, particularly if they came from some powerful object far away in the Universe. You can’t figure out where they originated from by their direction when they hit Earth.

Particle-physics theorists came to the rescue.

“If cosmic rays hit any matter around them, the collision will generate secondary products,” Merino says. “A byproduct of any high-energy interaction with the protons that make up much of a cosmic ray will be neutrinos.”

Neutrinos respond to gravity and to what’s known as the weak subatomic force, like most matter. But they aren’t affected by the electromagnetic forces that send cosmic rays on a drunkard’s walk. Scientists realized that the intense showers of protons at the source of cosmic rays had to be hitting matter nearby, producing neutrinos that can be tracked back to their source.

The shape of water

But if the matter that makes up your instrument can’t interact with an incoming neutrino, how are you going to detect it? The answer lay in making the detector big.

“The probability that a neutrino will interact with matter is extremely low, but not zero,” Merino explains. “If you want to see neutrinos, you need to build a huge detector so that they collide with matter at a reasonable rate.”

Multimessenger astronomy combines information from different cosmic messenger—cosmic rays, neutrinos, gamma rays, and gravitational waves—to learn about the distant and extreme universe. Courtesy IceCube Collaboration.

Enter the Antarctic ice shelf. The ice here is nearly pure water and could be used as a detector. From 2005 through 2010, a UW-led team created the IceCube Neutrino Observatory by drilling 86 holes deep in the ice, re-freezing detectors in the holes. Their new detector consisted of 5,160 detectors suspended in a huge ice cube six-tenths of a mile on each side.

The IceCube scientists weren’t quite ready to detect cosmic-ray-associated neutrinos yet. While the IceCube observatory was nearly pure water, it wasn’t completely pure. As a natural formation, its transparency might differ a bit from spot to spot, which could affect detection.

“Progress in understanding the precise optical properties of the ice leads to increasing complexity in simulating the propagation of photons in the instrument and to a better overall performance of the detector,” says Francis Halzen, a UW professor of physics and the lead scientist for the IceCube Neutrino Observatory.

GPUs to the rescue

The collaborators simulated the effects of neutrinos hitting the ice using traditional supercomputers containing standard central processing units (CPUs). They realized, though, that portions of their computations would instead work faster on graphics-processing units (GPUs), invented to improve video-game animation.

“We realized that a part of the simulation is a very good match for GPUs,” Merino says. “These computations run 100 to 300 times faster on GPUs than on CPUs.”

Madison’s own GPU cluster and collaborators’ campuses’ GPU systems helped, but it wasn’t enough.


Then Merino had a talk with XSEDE ECSS expert Sergiu Sanielevici from the Pittsburgh Supercomputing Center (PSC), lead of XSEDE’s Novel and Innovative Projects.

Pittsburgh Supercomputing Center 3000 cores, 6 TFLOPS

Sanielevici filled him in on the large GPU capability of XSEDE supercomputing systems. The IceCube team wound up using a number of XSEDE machines for GPU and CPU computations: Bridges at PSC, Comet at the San Diego Supercomputer Center (SDSC), XStream at Stanford University and the collection of clusters available through the Open Science Grid Consortium.

Bridges at PSC

SDSC Dell Comet supercomputer at San Diego Supercomputer Center (SDSC)

Stanford U Cray Xstream supercomputer

The IceCube scientists could not assume that their computer code would run well in the XSEDE system. Their massive and complex flow of calculations could have slowed down considerably had the new machines conflicted with it. ECSS expertise was critical to making the join-up smooth.

“XSEDE’s resources integrated seamlessly; that was very important for us,” Merino says. “XSEDE has been very collaborative, extremely open in facilitating that integration.”

Their detector built and simulated, the IceCube scientists had to wait for it to detect a cosmic neutrino. On Sept. 22, 2017, it happened. An automated system tuned to the signature of a cosmic-ray neutrino sent a message to the members of the IceCube Collaboration, an international team with more than 300 scientists in 12 countries.

This was important. A single neutrino detection would not have been proof by itself. Scientists at observatories that detect other types of radiation expected from cosmic rays needed to look at the same spot in the sky.

Blazars are a type of active galaxy with one of its jets pointing toward us. It emits both neutrinos and gamma rays that could be detected by the IceCube Neutrino Observatory as well as by other telescopes on Earth and in space. Courtesy IceCube/NASA.

They found multiple types of radiation coming from the same spot in the sky as the neutrino. At this spot was a “blazar” called TXS 0506+056, about 4 billion light years from Earth. A type of active galactic nucleus (AGN), a blazar is a huge black hole sitting in the center of a distant galaxy, flaring as it eats the galaxy’s matter. Blazars are AGNs that happen to be pointed straight at us.

The scientists think that the vast forces surrounding the black hole are likely the catapult that shot cosmic-ray particles on their way toward Earth. After a journey of 4 billion years across the vastness of space, one of the neutrinos created by those particles blazed a path through IceCube’s detector.

The IceCube scientists went back over nine and a half years of detector data, before they’d set up their automated warning. They found several earlier detections from TXS 0506+056, greatly raising their confidence.

The findings led to papers in the prestigious journal Science and Science in July 2018. Future work will focus on confirming that blazars are the source—or at least a major source—of the high-energy particles that fill the Universe.

See the full article here .

Please help promote STEM in your local schools.

Stem Education Coalition

Science Node is an international weekly online publication that covers distributed computing and the research it enables.

“We report on all aspects of distributed computing technology, such as grids and clouds. We also regularly feature articles on distributed computing-enabled research in a large variety of disciplines, including physics, biology, sociology, earth sciences, archaeology, medicine, disaster management, crime, and art. (Note that we do not cover stories that are purely about commercial technology.)

In its current incarnation, Science Node is also an online destination where you can host a profile and blog, and find and disseminate announcements and information about events, deadlines, and jobs. In the near future it will also be a place where you can network with colleagues.

You can read Science Node via our homepage, RSS, or email. For the complete iSGTW experience, sign up for an account or log in with OpenID and manage your email subscription from your account preferences. If you do not wish to access the website’s features, you can just subscribe to the weekly email.”