From insideHPC: “NERSC taps NVIDIA compiler team for Perlmutter Supercomputer”

From insideHPC

March 22, 2019

Dr. Saul Perlmutter (left) holds an animated conservation with John Kirkley at SC13. Photo by Sharan Kalwani, Fermilab

NERSC has signed a contract with NVIDIA to enhance GPU compiler capabilities for Berkeley Lab’s next-generation Perlmutter supercomputer.

Cray Shasta Perlmutter SC18 AMD Epyc Nvidia pre-exascale supeercomputer


NERSC Cray Cori II supercomputer at NERSC at LBNL, named after Gerty Cori, the first American woman to win a Nobel Prize in science

NERSC Hopper Cray XE6 supercomputer

LBL NERSC Cray XC30 Edison supercomputer

The Genepool system is a cluster dedicated to the DOE Joint Genome Institute’s computing needs. Denovo is a smaller test system for Genepool that is primarily used by NERSC staff to test new system configurations and software.


PDSF is a networked distributed computing cluster designed primarily to meet the detector simulation and data analysis requirements of physics, astrophysics and nuclear science collaborations.

DOE and Cray announced on Oct. 30, 2018 that NERSC’s next supercomputer will be a Cray pre-exascale system to be delivered in 2020.

To highlight NERSC’s commitment to advancing research, the new system will be named “Perlmutter” in honor of Saul Perlmutter, an astrophysicist at Berkeley Lab and a professor of physics at the University of California, Berkeley who shared the 2011 Nobel Prize in Physics for his contributions to research showing that the expansion of the universe is accelerating. Dr. Perlmutter is also director of the Berkeley Institute for Data Science and leads the international Supernova Cosmology Project. He has been a NERSC user for many years, and part of his Nobel Prize winning work was carried out on NERSC machines.

Perlmutter, a Cray system code-named “Shasta”, will be a heterogeneous system comprising both CPU-only and GPU-accelerated nodes, with a performance of more than 3 times Cori, NERSC’s current platform. It will include a number of innovations designed to meet the diverse computational and data analysis needs of NERSC’s user base and speed their scientific productivity. The new system derives performance from advances in hardware and software, including a new Cray system interconnect, code-named Slingshot, which is designed for data-centric computing. Slingshot’s Ethernet compatibility, advanced adaptive routing, first-of-a-kind congestion control, and sophisticated quality of service capabilities improve system utilization and performance and scalability of supercomputing and AI applications and workflows. The system will also feature NVIDIA GPUs with new Tensor Core technology, direct liquid cooling and will be NERSC’s first supercomputer with an all-flash scratch filesystem. Developed by Cray to accelerate I/O, the 30-petabyte Lustre filesystem will move data at a rate of more than 4 terabytes/sec.

“We are excited to work with NVIDIA to enable OpenMP GPU computing using their PGI compilers,” said Nick Wright, the Perlmutter chief architect. “Many NERSC users are already successfully using the OpenMP API to target the manycore architecture of the NERSC Cori supercomputer. This project provides a continuation of our support of OpenMP and offers an attractive method to use the GPUs in the Perlmutter supercomputer. We are confident that our investment in OpenMP will help NERSC users meet their application performance portability goals.”

Under the new non-recurring engineering contract with NVIDIA, worth approximately $4 million, Berkeley Lab researchers will work with NVIDIA engineers to enhance the NVIDIA’s PGI C, C++ and Fortran compilers to enable OpenMP applications to run on NVIDIA GPUs. This collaboration will help NERSC users, and the HPC community as a whole, efficiently port suitable applications to target GPU hardware in the Perlmutter system.

Programming using compiler directives of any form are an important part of code portability and developer productivity. NERSC participation in both OpenMP and OpenACC organizations helps advance the entire ecosystem of important tools and the specifications on which they rely.

“Together with OpenACC, this OpenMP collaboration gives HPC developers more options for directives-based programming from a single compiler on GPUs and CPUs,” said Doug Miles, senior director of PGI compilers and tools at NVIDIA. “Our joint effort on programming tools for the Perlmutter supercomputer highlights how NERSC and NVIDIA are simplifying migration and development of science and engineering applications to pre-exascale systems and beyond.”

The Perlmutter Supercomputer will be based on the Cray Shasta architecture.

In addition, through this partnership, NERSC and NVIDIA will develop a set of GPU-based high performance data analytic tools using Python, the primary language used for data analytics at NERSC and a robust platform for machine learning and deep learning libraries. The new Python tools will allow NERSC to train staff and users through hack-a-thons where NERSC users will be able to work directly with NVIDIA personnel on their codes.

“NERSC supports thousands of researchers in diverse sciences at universities, national laboratories, and in industry,” commented Data Architect Rollin Thomas, who is leading the partnership at NERSC. “Our users increasingly want productive high-performance tools for interacting with their data, whether it comes from a massively parallel simulation or an experimental or observational science facility like a particle accelerator, astronomical observatory, or genome sequencer. We look forward to working with NVIDIA to accelerate discovery across all these disciplines.”

See the full article here .


Please help promote STEM in your local schools.

Stem Education Coalition

Founded on December 28, 2006, insideHPC is a blog that distills news and events in the world of HPC and presents them in bite-sized nuggets of helpfulness as a resource for supercomputing professionals. As one reader said, we’re sifting through all the news so you don’t have to!

If you would like to contact me with suggestions, comments, corrections, errors or new company announcements, please send me an email at Or you can send me mail at:

2825 NW Upshur
Suite G
Portland, OR 97239

Phone: (503) 877-5048