Tagged: Supercomputing Toggle Comment Threads | Keyboard Shortcuts

  • richardmitnick 12:41 pm on February 12, 2020 Permalink | Reply
    Tags: Green AI Hackathon, Green AI Hackathon-shrinking the carbon footprint of artificial intelligence models., IBM Satori- MIT’s new supercomputer., , Supercomputing   

    From MIT News: “Brainstorming energy-saving hacks on Satori, MIT’s new supercomputer” 

    MIT News

    From MIT News

    February 11, 2020
    Kim Martineau | MIT Quest for Intelligence

    1
    IBM Satori Supercomputer

    Three-day hackathon explores methods for making artificial intelligence faster and more sustainable.

    Mohammad Haft-Javaherian planned to spend an hour at the Green AI Hackathon — just long enough to get acquainted with MIT’s new supercomputer, Satori. Three days later, he walked away with $1,000 for his winning strategy to shrink the carbon footprint of artificial intelligence models trained to detect heart disease.

    “I never thought about the kilowatt-hours I was using,” he says. “But this hackathon gave me a chance to look at my carbon footprint and find ways to trade a small amount of model accuracy for big energy savings.”

    Haft-Javaherian was among six teams to earn prizes at a hackathon co-sponsored by the MIT Research Computing Project and MIT-IBM Watson AI Lab Jan. 28-30. The event was meant to familiarize students with Satori, the computing cluster IBM donated to MIT last year, and to inspire new techniques for building energy-efficient AI models that put less planet-warming carbon dioxide into the air.

    The event was also a celebration of Satori’s green-computing credentials. With an architecture designed to minimize the transfer of data, among other energy-saving features, Satori recently earned fourth place on the Green500 list of supercomputers. Its location gives it additional credibility: It sits on a remediated brownfield site in Holyoke, Massachusetts, now the Massachusetts Green High Performance Computing Center, which runs largely on low-carbon hydro, wind and nuclear power.

    A postdoc at MIT and Harvard Medical School, Haft-Javaherian came to the hackathon to learn more about Satori. He stayed for the challenge of trying to cut the energy intensity of his own work, focused on developing AI methods to screen the coronary arteries for disease. A new imaging method, optical coherence tomography, has given cardiologists a new tool for visualizing defects in the artery walls that can slow the flow of oxygenated blood to the heart. But even the experts can miss subtle patterns that computers excel at detecting.

    At the hackathon, Haft-Javaherian ran a test on his model and saw that he could cut its energy use eight-fold by reducing the time Satori’s graphics processors sat idle. He also experimented with adjusting the model’s number of layers and features, trading varying degrees of accuracy for lower energy use.

    A second team, Alex Andonian and Camilo Fosco, also won $1,000 by showing they could train a classification model nearly 10 times faster by optimizing their code and losing a small bit of accuracy. Graduate students in the Department of Electrical Engineering and Computer Science (EECS), Andonian and Fosco are currently training a classifier to tell legitimate videos from AI-manipulated fakes, to compete in Facebook’s Deepfake Detection Challenge. Facebook launched the contest last fall to crowdsource ideas for stopping the spread of misinformation on its platform ahead of the 2020 presidential election.

    If a technical solution to deepfakes is found, it will need to run on millions of machines at once, says Andonian. That makes energy efficiency key. “Every optimization we can find to train and run more efficient models will make a huge difference,” he says.

    To speed up the training process, they tried streamlining their code and lowering the resolution of their 100,000-video training set by eliminating some frames. They didn’t expect a solution in three days, but Satori’s size worked in their favor. “We were able to run 10 to 20 experiments at a time, which let us iterate on potential ideas and get results quickly,” says Andonian.

    As AI continues to improve at tasks like reading medical scans and interpreting video, models have grown bigger and more calculation-intensive, and thus, energy intensive. By one estimate, training a large language-processing model produces nearly as much carbon dioxide as the cradle-to-grave emissions from five American cars. The footprint of the typical model is modest by comparison, but as AI applications proliferate its environmental impact is growing.

    One way to green AI, and tame the exponential growth in demand for training AI, is to build smaller models. That’s the approach that a third hackathon competitor, EECS graduate student Jonathan Frankle, took. Frankle is looking for signals early in the training process that point to subnetworks within the larger, fully-trained network that can do the same job. The idea builds on his award-winning Lottery Ticket Hypothesis paper from last year that found a neural network could perform with 90 percent fewer connections if the right subnetwork was found early in training.

    The hackathon competitors were judged by John Cohn, chief scientist at the MIT-IBM Watson AI Lab, Christopher Hill, director of MIT’s Research Computing Project, and Lauren Milechin, a research software engineer at MIT.

    The judges recognized four other teams: Department of Earth, Atmospheric and Planetary Sciences (EAPS) graduate students Ali Ramadhan, Suyash Bire, and James Schloss, for adapting the programming language Julia for Satori; MIT Lincoln Laboratory postdoc Andrew Kirby, for adapting code he wrote as a graduate student to Satori using a library designed for easy programming of computing architectures; and Department of Brain and Cognitive Sciences graduate students Jenelle Feather and Kelsey Allen, for applying a technique that drastically simplifies models by cutting their number of parameters.

    IBM developers were on hand to answer questions and gather feedback. “We pushed the system — in a good way,” says Cohn. “In the end, we improved the machine, the documentation, and the tools around it.”

    Going forward, Satori will be joined in Holyoke by TX-Gaia, Lincoln Laboratory’s new supercomputer. Together, they will provide feedback on the energy use of their workloads. “We want to raise awareness and encourage users to find innovative ways to green-up all of their computing,” says Hill.

    See the full article here .


    five-ways-keep-your-child-safe-school-shootings
    Please help promote STEM in your local schools.


    Stem Education Coalition

    MIT Seal

    The mission of MIT is to advance knowledge and educate students in science, technology, and other areas of scholarship that will best serve the nation and the world in the twenty-first century. We seek to develop in each member of the MIT community the ability and passion to work wisely, creatively, and effectively for the betterment of humankind.

    MIT Campus

     
  • richardmitnick 3:06 pm on February 4, 2020 Permalink | Reply
    Tags: , , Fujitsu PRIMEHPC FX1000 supercomputer at Nagoya University, , Supercomputing   

    From insideHPC: “Fujitsu to Deploy Arm-based Supercomputer at Nagoya University” 

    From insideHPC

    February 4, 2020
    Rich Brueckner

    Today Fujitsu announced that it has received an order for an Arm-based supercomputer system from Nagoya University’s Information Technology Center. The system is scheduled to start operation in July 2020.

    1
    Fujitsu PRIMEHPC FX1000

    “For the first time in the world, this system will adopt 2,304 nodes of the Fujitsu Supercomputer PRIMEHPC FX1000, which utilizes the technology of the supercomputer Fugaku developed jointly with RIKEN. In addition, a cluster system, connecting 221 nodes of the latest x86 servers Fujitsu Server PRIMERGY CX2570 M5 in parallel, as well as storage systems are connected by a high-speed interconnect. The sum of the theoretical computational performance of the entire system is 15.88 petaflops, making it one of the highest performing systems in Japan.”

    As a national joint usage/research center, the Information Technology Center of Nagoya University provides computing resources for academic use to researchers and private companies nationwide. It is currently operating a supercomputer system consisting of Fujitsu Supercomputer PRIMEHPC FX100 and other components. This time, the Center is planning to innovate the system in order to fulfill the large-scale calculation demand from researchers in joint usage nationwide, as well as the new calculation requirement for supercomputers represented by data science. Fujitsu won the order for this system in recognition of its proposal that concentrates the technical capabilities of Fujitsu and Fujitsu Laboratories Ltd.

    With the new system, Nagoya University’s Information Technology Center will contribute to various research and development activities. These include the conventional simulation of numerical computation to unravel the mechanism of typhoons and design new drugs. Moreover, the new system will develop a technology in the medical field that supports diagnoses and treatment, while apply AI in developing automatic driving technology.

    Fujitsu will continue to support the activities of the Center with its technology and experience nurtured through the development and offering of world-class supercomputers. By providing PRIMEHPC FX1000 worldwide, the company will also contribute to solving social issues, accelerating leading-edge research, and strengthening corporate advantages.

    “In recent years, the digitization of university education and research activities has increased the demand for computing,” said Kensaku Mori, Director, The Information Technology Center of Nagoya University. “In addition to such areas as extreme weather including super typhoons, earthquakes, and tsunamis, which are closely related to the safety and security of people’s lives, chemical fields such as molecular structure and drug discovery, and simulations in basic sciences such as space and elementary particles, there is an ever-increasing demand for computing in the fields of medicine and mobility, including artificial intelligence and machine learning. Also important are the data consumed and generated in computing, the networks that connect them, and the visualization of knowledge discovery from computing and data. Equipped with essential functions for such digital science in universities, the new supercomputer will be offered not only to Nagoya University but also to universities and research institutes nationwide, contributing to the further development of academic research in Japan.”

    See the full article here .

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Founded on December 28, 2006, insideHPC is a blog that distills news and events in the world of HPC and presents them in bite-sized nuggets of helpfulness as a resource for supercomputing professionals. As one reader said, we’re sifting through all the news so you don’t have to!

    If you would like to contact me with suggestions, comments, corrections, errors or new company announcements, please send me an email at rich@insidehpc.com. Or you can send me mail at:

    insideHPC
    2825 NW Upshur
    Suite G
    Portland, OR 97239

    Phone: (503) 877-5048

     
  • richardmitnick 5:52 pm on January 30, 2020 Permalink | Reply
    Tags: ALCF will deploy the new Cray ClusterStor E1000 as its parallel storage solution., ALCF’s two new storage systems which it has named “Grand” (150 PB of center-wide storage) and “Eagle” (50 PB community file system) are using the Cray ClusterStor E1000 system., , , , , Supercomputing, This is in preparation for the pending Aurora exascale supercomputer.   

    From insideHPC: “Argonne to Deploy Cray ClusterStor E1000 Storage System for Exascale” 

    From insideHPC

    January 30, 2020
    Rich Brueckner

    1
    Cray ClusterStor E1000

    Today HPE announced that ALCF will deploy the new Cray ClusterStor E1000 as its parallel storage solution.

    The new collaboration supports ALCF’s scientific research in areas such as earthquake seismic activity, aerospace turbulence and shock-waves, physical genomics and more.

    The latest deployment will expand storage capacity for ALCF’s workloads that require converged modeling, simulation, AI and analytics workloads, in preparation for the pending Aurora exascale supercomputer.

    Depiction of ANL ALCF Cray Intel SC18 Shasta Aurora exascale supercomputer

    Powered by HPE and Intel, Aurora is a Cray Shasta system planned for delivery in 2021.

    The Cray ClusterStor E1000 system utilizes purpose-built software and hardware features to meet high-performance storage requirements of any size with significantly fewer drives. Designed to support the Exascale Era, which is characterized by the explosion of data and converged workloads, the Cray ClusterStor E1000 will power ALCF’s future Aurora supercomputer to target a multitude of data-intensive workloads required to make breakthrough discoveries at unprecedented speed.

    “ALCF is leveraging Exascale Era technologies by deploying infrastructure required for converged workloads in modeling, simulation, AI and analytics,” said Peter Ungaro, senior vice president and general manager, HPC and AI, at HPE. “Our recent introduction of the Cray ClusterStor E1000 is delivering ALCF unmatched scalability and performance to meet next-generation HPC storage needs to support emerging, data-intensive workloads. We look forward to continuing our collaboration with ALCF and empowering its research community to unlock new value.”

    ALCF’s two new storage systems, which it has named “Grand” and “Eagle,” are using the Cray ClusterStor E1000 system to gain a completely new, cost-effective high-performance computing (HPC) storage solution to effectively and efficiently manage growing converged workloads that today’s offerings cannot support.

    “When Grand launches, it will benefit ALCF’s legacy petascale machines, providing increased capacity for the Theta compute system and enabling new levels of performance for not just traditional checkpoint-restart workloads, but also for complex workflows and metadata-intensive work,” said Mark Fahey, director of operations, ALCF.”

    “Eagle will help support the ever-increasing importance of data in the day-to-day activities of science,” said Michael E. Papka, director, ALCF. “By leveraging our experience with our current data-sharing system, Petrel, this new storage will help eliminate barriers to productivity and improve collaborations throughout the research community.”

    The two new systems will gain a total of 200 petabyes (PB) of storage capacity, and through the Cray ClusterStor E1000’s intelligent software and hardware designs, will more accurately align data flows with target workloads. ALCF’s Grand and Eagle systems will help researchers accelerate a range of scientific discoveries across disciplines, and are each assigned to address the following:

    Computational capacity – ALCF’s “Grand” provides 150 PB of center-wide storage and new levels of input/output (I/O) performance to support massive computational needs for its users.
    Simplified data-sharing – ALCF’s “Eagle” provides a 50 PB community file system to make data-sharing easier than ever among ALCF users, their collaborators and with third parties.

    ALCF plans to deliver its Grand and Eagle storage systems in early 2020. The systems will initially connect to existing ALCF supercomputers powered by HPE HPC systems: Theta, based on the Cray XC40-AC and Cooley, based on the Cray CS-300. ALCF’s Grand, which is capable of 1 terabyte per second (TB/s) bandwidth, will be optimized to support converged simulation science and data-intensive workloads once the Aurora exascale supercomputer is operational.

    See the full article here .

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Founded on December 28, 2006, insideHPC is a blog that distills news and events in the world of HPC and presents them in bite-sized nuggets of helpfulness as a resource for supercomputing professionals. As one reader said, we’re sifting through all the news so you don’t have to!

    If you would like to contact me with suggestions, comments, corrections, errors or new company announcements, please send me an email at rich@insidehpc.com. Or you can send me mail at:

    insideHPC
    2825 NW Upshur
    Suite G
    Portland, OR 97239

    Phone: (503) 877-5048

     
  • richardmitnick 4:40 pm on January 24, 2020 Permalink | Reply
    Tags: "Simulations Reveal Galaxy Clusters Details", , Astrophysicists have developed cosmological computer simulations called RomulusC where the ‘C' stands for galaxy cluster., , , , RomulusC has produced some of the highest resolution simulations ever of galaxy clusters which can contain hundreds or even thousands of galaxies., Supercomputing,   

    From Texas Advanced Computing Center: “Simulations Reveal Galaxy Clusters Details” 

    TACC bloc

    From Texas Advanced Computing Center

    January 23, 2020
    Jorge Salazar

    Galaxy clusters probed with Stampede2, Comet supercomputers [and others-see below]

    1
    RomulusC has produced some of the highest resolution simulations ever of galaxy clusters, which can contain hundreds or even thousands of galaxies. The galaxy cluster simulations generated by supercomputers are helping scientists map the unknown universe. Credit: Butsky et al.

    Inspired by the science fiction of the spacefaring Romulans of Star Trek, astrophysicists have developed cosmological computer simulations called RomulusC, where the ‘C’ stands for galaxy cluster. With a focus on black hole physics, RomulusC has produced some of the highest resolution simulations ever of galaxy clusters, which can contain hundreds or even thousands of galaxies.

    On Star Trek, the Romulans powered their spaceships with an artificial black hole. In reality, it turns out that black holes can drive the formation of stars and the evolution of whole galaxies. And this galaxy cluster work is helping scientists map the unknown universe.

    An October 2019 study yielded results from RomulusC simulations, published in the Monthly Notices of the Royal Astronomical Society. It probed the ionized gas of mainly hydrogen and helium within and surrounding the intracluster medium, which fills the space between galaxies in a galaxy cluster.

    Hot, dense gas of more than a million degrees Kelvin fills the inner cluster with roughly uniform metallicity. Cool-warm gas between ten thousand and a million degrees Kelvin lurks in patchy distributions at the outskirts, with greater variety of metals. Looking like the tail of a jellyfish, the cool-warm gas traces the process of galaxies falling into the cluster and losing their gas. The gas gets stripped from the falling galaxy and eventually mixes with the inner region of the galaxy cluster.

    “We find that there’s a substantial amount of this cool-warm gas in galaxy clusters,” said study co-author Iryna Butsky, a PhD Student in the Department of Astronomy at the University of Washington. “We see that this cool-warm gas traces at extremely different and complementary structures compared to the hot gas. And we also predict that this cool-warm component can be observed now with existing instruments like the Hubble Space Telescope Cosmic Origins Spectrograph.”

    Scientists are just beginning to probe the intracluster medium, which is so diffuse that its emissions are invisible to any current telescopes. Scientists are using RomulusC to help see clusters indirectly using the ultraviolet (UV) light from quasars, which act like a beacon shining through the gas. The gas absorbs UV light, and the resulting spectrum yields density, temperature, and metallicity profiles when analyzed with instruments like the Cosmic Origins Spectrograph aboard the Hubble Space Telescope (HST).

    NASA Hubble Cosmic Origins Spectrograph

    NASA/ESA Hubble Telescope

    2
    A 5×5 megaparsec (~18.15 light years) snapshot of the RomulusC simulation at redshift z = 0.31. The top row shows density-weighted projections of gas density, temperature, and metallicity. The bottom row shows the integrated X-ray intensity, O VI column density, and H I column density. Credit: Butsky et al.

    “One really cool thing about simulations is that we know what’s going on everywhere inside the simulated box,” Butsky said. “We can make some synthetic observations and compare them to what we actually see in absorption spectra and then connect the dots and match the spectra that’s observed and try to understand what’s really going on in this simulated box.”

    They applied a software tool called Trident developed by Cameron Hummels of Caltech and colleagues that takes the synthetic absorption line spectra and adds a bit of noise and instrument quirks known about the HST.

    “The end result is a very realistic looking spectrum that we can directly compare to existing observations,” Butsky said. “But what we can’t do with observations is reconstruct three-dimensional information from a one-dimensional spectrum. That’s what’s bridging the gap between observations and simulations.”

    One key assumption behind the RomulusC simulations supported by the latest science is that the gas making up the intracluster medium originates at least partly in the galaxies themselves. “We have to model how that gas gets out of the galaxies, which is happening through supernovae going off, and supernovae coming from young stars,” said study co-author Tom Quinn, a professor of astronomy at the University of Washington. That means a dynamic range of more than a billion to contend with.

    What’s more, clusters don’t form in isolation, so their environment has to be accounted for.

    Then there’s a computational challenge that’s particular to clusters. “Most of the computational action is happening in the very center of the cluster. Even though we’re simulating a much larger volume, most of the computation is happening at a particular spot. There’s a challenge of, as you’re trying to simulate this on a large supercomputer with tens of thousands of cores, how do you distribute that computation across those cores?” Quinn said.

    Quinn is no stranger to computational challenges. Since 1995, he’s used computing resources funded by the National Science Foundation (NSF), most recently those that are part of XSEDE, the Extreme Science and Engineering Discovery Environment.

    “Over the course of my career, NSF’s ability to provide high-end computing has helped the overall development of the simulation code that produced this,” said Quinn. “These parallel codes take a while to develop. And XSEDE has been supporting me throughout that development period. Access to a variety of high-end machines has helped with the development of the simulation code.”

    RomulusC started out as a proof-of-concept with friendly user time on the Stampede2 [below] system at the Texas Advanced Computing Center (TACC), when the Intel Xeon Phil (“Knights Landing”) processors first became available. “I got help from the TACC staff on getting the code up and running on the many-core, 68 core per chip machines.”

    Quinn and colleagues eventually scaled up RomulusC to 32,000 processors and completed the simulation on the Blue Waters system of the National Center for Supercomputing Applications.

    NCSA U Illinois Urbana-Champaign Blue Waters Cray Linux XE/XK hybrid machine supercomputer

    Along the way, they also used the NASA Pleiades supercomputer and the XSEDE-allocated Comet system at the San Diego Supercomputer Center, an Organized Research Unit of the University of California San Diego.

    NASA SGI Intel Advanced Supercomputing Center Pleiades Supercomputer

    SDSC Dell Comet supercomputer at San Diego Supercomputer Center (SDSC)

    “Comet fills a particular niche,” Quinn said. “It has large memory nodes available. Particular aspects of the analysis, for example identifying the galaxies, is not easily done on a distributed memory machine. Having the large shared memory machine available was very beneficial. In a sense, we didn’t have to completely parallelize that particular aspect of the analysis.”

    4
    The Stampede2 supercomputer at the Texas Advanced Computing Center (left) and the Comet supercomputer at the San Diego Supercomputer Center (right) are allocated resources of the Extreme Science and Engineering Discovery Environment (XSEDE) funded by the National Science Foundation (NSF). Credit: TACC, SDSC.

    “Without XSEDE, we couldn’t have done this simulation,” Quinn recounted. “It’s essentially a capability simulation. We needed the capability to actually do the simulation, but also the capability of the analysis machines.”

    The next generation of simulations are being made using the NSF-funded Frontera [below] system, the fastest academic supercomputer and currently the #5 fastest system in the world, according to the November 2019 Top500 List.

    “Right now on Frontera, we’re doing runs at higher resolution of individual galaxies,” Quinn said. “Since we started these simulations, we’ve been working on proving how we model the star formation. And of course we have more computational power, so just purely higher mass resolution, again, to make our simulations of individual galaxies more realistic. More and bigger clusters would be good too,” Quinn added.

    Said Butsky: “What I think is really cool about using supercomputers to model the universe is that they play a unique role in allowing us to do experiments. In many of the other sciences, you have a lab where you can test your theories. But in astronomy, you can come up with a pen and paper theory and observe the universe as it is. But without simulations, it’s very hard to run these tests because it’s hard to reproduce some of the extreme phenomena in space, like temporal scales and getting the temperatures and densities of some of these extreme objects. Simulations are extremely important in being able to make progress in theoretical work.”

    The study,”Ultraviolet Signatures of the Multiphase Intracluster and Circumgalactic Media in the RomulusC Simulation,” was published in October of 2019 in the Monthly Notices of the Royal Astronomical Society. The study co-authors are Iryna S. Butsky, Thomas R. Quinn, and Jessica K. Werk of the University of Washington; Joseph N. Burchett of UC Santa Cruz, and Daisuke Nagai and Michael Tremmel of Yale University. Study funding came from the NSF and NASA.

    See the full article here .

    Please help promote STEM in your local schools.


    Stem Education Coalition

    The Texas Advanced Computing Center (TACC) designs and operates some of the world’s most powerful computing resources. The center’s mission is to enable discoveries that advance science and society through the application of advanced computing technologies.

    TACC Maverick HP NVIDIA supercomputer

    TACC Lonestar Cray XC40 supercomputer

    Dell Poweredge U Texas Austin Stampede Supercomputer. Texas Advanced Computer Center 9.6 PF

    TACC HPE Apollo 8000 Hikari supercomputer

    TACC Maverick HP NVIDIA supercomputer

    TACC DELL EMC Stampede2 supercomputer


    TACC Frontera Dell EMC supercomputer fastest at any university

     
  • richardmitnick 3:36 pm on January 24, 2020 Permalink | Reply
    Tags: , , , Microway supercomputer being installed, Supercomputing, The new cluster from Microway affords the university five times the compute performance its researchers enjoyed previously with over 85% more total memory and over four times the aggregate memory band, The UMass Dartmouth cluster reflects a hybrid design to appeal to a wide array of the campus’ workloads.,   

    From insideHPC: “UMass Dartmouth Speeds Research with Hybrid Supercomputer from Microway” 

    From insideHPC

    Today Microway announced that research activities are accelerating at the University of Massachusetts Dartmouth since the installation of a new supercomputing cluster.

    “UMass Dartmouth’s powerful new cluster from Microway affords the university five times the compute performance its researchers enjoyed previously, with over 85% more total memory and over four times the aggregate memory bandwidth. It includes a heterogeneous system architecture featuring a wide array of computational engines.”

    2

    The UMass Dartmouth cluster reflects a hybrid design to appeal to a wide array of the campus’ workloads.

    Over 50 nodes include Intel Xeon Scalable Processors, DDR4 memory, SSDs and Mellanox ConnectX-5 EDR 100Gb InfiniBand. A subset of systems also feature NVIDIA V100 GPU Accelerators for GPU computing applications.

    Equally important are a second subset of POWER9 with 2nd Generation NVLink- based- IBM Power Systems AC922 Compute nodes. These systems are similar to those utilized in the world’s #1 and #2 most powerful Summit and Sierra supercomputers at ORNL and LLNL. The advanced NVIDIA NVLink interfaces built into POWER9 CPU and NVIDIA GPU ensure a broad pipeline between CPU:GPU for data intensive workloads.

    The deployment of the hybrid architecture system was critical to meeting the users’ needs. It also allowed those on the UMass Dartmouth campus to apply to test workloads onto the larger national laboratory systems at ORNL.

    Microway was one of the few vendors able to deliver a unified system with a mix of x86 and POWER9 systems, complete software integration across both kinds of nodes in the cluster, and offer a single point of sale and warranty coverage.

    Microway was selected as the vendor for the new cluster through an open bidding process. “They not only competed well on the price,” says Khanna, “but they were also the only company that could deliver the kind of heterogeneous system we wanted with a mixture of architecture.”

    For more information about the UMass Dartmouth Center for Scientific Computing and Visualization Research please navigate to: http://cscvr1.umassd.edu/

    This new cluster purchase was funded through an Office of Naval Research (ONR) DURIP grant award.

    Serving Users Across a Research Campus

    The deployment has helped continue to serve, attract and retain faculty, undergraduate students, and those seeking advance degrees to the UMass Dartmouth campus. The Center for Scientific Computing and Visualization Research administers the new compute resource.

    With its new cluster, CSCVR is undertaking cutting edge work. Mathematics researchers are developing new numerical algorithms on the new deployment. A primary focus is in astrophysics: with focus on the study of black holes and stars.

    “Our engineering researchers,” says Gaurav Khanna, Co-Director of UMass Dartmouth’s Center for Scientific Computing & Visualization Research, “are very actively focused on computational engineering, and there are people in mechanical engineering who look at fluid and solid object interactions.” This type of research is known as two-phase fluid flow. Practical applications can take the form of modelling windmills and coming up with a better design for the materials on the windmill such as the coatings on the blade, as well as improved designs for the blades themselves.

    This team is also looking at wave energy converters in ocean buoys. “As buoys bob up and down,” Khanna explains, “you can use that motion to generate electricity. You can model that into the computation of that environment and then try to optimize the parameters needed to have the most efficient design for that type of buoy.”

    A final area of interest to this team is ocean weather systems. Here, UMass Dartmouth researchers are building large models to predict regional currents in the ocean, weather patterns, and weather changes.

    2

    A Hybrid Architecture for a Broad Array of Workloads

    The UMass Dartmouth cluster reflects a hybrid design to appeal to a wide array of the campus’ workloads.

    The deployment of the hybrid architecture system was critical to meeting the users’ needs. It also allowed those on the UMass Dartmouth campus to apply to test workloads onto the larger national laboratory systems at ORNL.

    Microway was one of the few vendors able to deliver a unified system with a mix of x86 and POWER9 systems, complete software integration across both kinds of nodes in the cluster, and offer a single point of sale and warranty coverage.

    “Microway was selected as the vendor for the new cluster through an open bidding process. “They not only competed well on the price,” says Khanna, “but they were also the only company that could deliver the kind of heterogeneous system we wanted with a mixture of architecture.”

    See the full article here .

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Founded on December 28, 2006, insideHPC is a blog that distills news and events in the world of HPC and presents them in bite-sized nuggets of helpfulness as a resource for supercomputing professionals. As one reader said, we’re sifting through all the news so you don’t have to!

    If you would like to contact me with suggestions, comments, corrections, errors or new company announcements, please send me an email at rich@insidehpc.com. Or you can send me mail at:

    insideHPC
    2825 NW Upshur
    Suite G
    Portland, OR 97239

    Phone: (503) 877-5048

     
  • richardmitnick 1:46 pm on January 22, 2020 Permalink | Reply
    Tags: "Beyond the tunnel", (LES)-large-eddy simulation, , , , , How turbulence affects aircraft during flight, , Stanford-led team turns to Argonne’s Mira to fine-tune a computational route around aircraft wind-tunnel testing., Supercomputing   

    From ASCR Discovery: “Beyond the tunnel” 

    From ASCR Discovery

    Stanford-led team turns to Argonne’s Mira to fine-tune a computational route around aircraft wind-tunnel testing.

    MIRA IBM Blue Gene Q supercomputer at the Argonne Leadership Computing Facility

    1
    The white lines represent simulated air-flow on a wing surface, including an eddy (the circular pattern at the tip). Engineers can use supercomputing, in particular large-eddy simulation (LES), to study how turbulence affects flight. LES techniques applied to commercial aircraft promise a cost-effective alternative to wind-tunnel testing. Image courtesy of Stanford University and Cascade Technologies.

    For aircraft designers, modeling and wind-tunnel testing one iteration after another consumes time and may inadequately recreate the conditions planes encounter during free flight – especially take-off and landing. “Prototyping that aircraft every time you change something in the design would be infeasible and expensive,” says Parviz Moin, a Stanford University professor of mechanical engineering.

    Over the past five years, researchers have explored the use of high-fidelity numerical simulations to predict unsteady airflow and forces such as lift and drag on commercial aircraft, particularly under challenging operating conditions such as takeoff and landing.

    Moin has led much of this research as founding director of the Center for Turbulence Research at Stanford. With help from the Department of Energy’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) grants, he and colleagues at Stanford and nearby Cascade Technologies in Palo Alto, California, have used supercomputing to see whether large-eddy simulation (LES) of commercial aircraft is both cost effective and sufficiently accurate to help designers. They’ve used 240 million core-hours on Mira, the Blue Gene/Q at the Argonne Leadership Computing Facility, a DOE user facility at Argonne National Laboratories, to conduct these simulations. The early results are “very encouraging,” Moin says.

    Specifically, Moin and colleagues – including INCITE co-investigators George Park of the University of Pennsylvania and Cascade Technologies’ Sanjeeb Bose, a DOE Computational Science Graduate Fellowship (DOE CSGF) alumnus – study how turbulence affects aircraft during flight. Flow of air about a plane in flight is always turbulent, wherein patches of swirling fluid – eddies – move seemingly at random.

    Because it happens on multiple scales, engineers and physicists find aircraft turbulence difficult to describe mathematically. The Navier-Stokes equations are known to govern all flows of engineering interest, Moin explains, including those involving gases and liquids and flows inside or outside a given object. Eddies can be several meters long in the atmosphere but only microns big in the aircraft surface’s vicinity. This means computationally solving the Navier-Stokes equations to describe all the fluid-motion scales would be prohibitively expensive and computationally taxing. For years, engineers have used Reynolds-averaged Navier-Stokes (RANS) equations to predict averaged quantities of engineering interest such as lift and drag forces. RANS equations, however, contain certain modeling assumptions that are not based on first principles, which can result in inaccurate predictions of complex flows.

    LES, on the other hand, offers a compromise, Moin says, between capturing the spectrum of eddy motions or ignoring them all. With LES, researchers can compute the effect of energy-containing eddies on an aircraft while modeling small-scale motions. Although LES predictions are more accurate than RANS approaches, the computational cost of LES has so far been a barrier to widespread use. But supercomputers and recent algorithm advances have rendered LES computations and specialized variations of them more feasible recently, Moin says.

    Eddies get smaller and smaller as they approach a wall – or a surface like an aircraft wing – and capturing these movements has historically been computationally challenging. To avoid these issues, Moin and his colleagues instead model the small-scale near-wall turbulence using a technique they call wall-modeled LES. In wall-modeled LES, the near-wall-eddy effect on the large-scale motions away from the wall are accounted for by a simpler system model.

    Moin and his colleagues have used two commercial aircraft models to validate their large-eddy simulation results: NASA’s Common Research Model and the Japan Aerospace Exploration Agency’s (JAXA’s) Standard Model. They’ve studied each at about a dozen operating conditions to see how the simulations agreed with physical measurements in wind tunnels.

    These early results show that the large-eddy simulations are capable of predicting quantities of engineering interest resulting from turbulent flow around an aircraft. This proof of concept, Moin says, is the first step. “We can compute these flows without tuning model parameters and predict experimental results. Once we have confidence as we compute many cases, then we can start looking into manipulating the flow using passive or active flow-control strategies.” The speed and accuracy of the computations, Moin notes, have been surprising. Researchers commonly thought the calculations could not have been realized until 2030, he says.

    Ultimately, these simulations will help engineers to make protrusions or other modifications of airplane wing surfaces to increase lift during take-off conditions or to design more efficient engines.

    Moin is eager to see more engineers use large eddy simulations and supercomputing to study the effect of turbulence on commercial aircraft and other applications.

    “The future of aviation is bright and needs more development,” he says. “I think with time – and hopefully it won’t take too long – aerospace engineers will start to see the advantage of these high-fidelity computations in engineering analysis and design.”

    See the full article here.


    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    ASCRDiscovery is a publication of The U.S. Department of Energy

     
  • richardmitnick 9:45 am on January 21, 2020 Permalink | Reply
    Tags: , , New Supercomputer AiMOS, , Supercomputing   

    From Rensselaer Polytechnic Institute: “Everything You Need to Know About Supercomputer AiMOS” 

    From Rensselaer Polytechnic Institute

    December 6, 2019
    Reeve Hamilton
    Mary Martialay

    1
    2
    2 images of AiMOS supercomputer provided by Rensselaer

    One of the most powerful new supercomputers in the world — and the most powerful supercomputer in New York state — was recently installed at the Rensselaer Polytechnic Institute Center for Computational Innovations. Part of a collaboration between Rensselaer, IBM, and New York state, it is configured to enable users to explore new AI applications and accelerate economic development from New York’s smallest startups to its largest enterprises.

    Here is everything you need to know about the machine, which is named AiMOS (short for Artificial Intelligence Multiprocessing Optimized System), and how it fits into the research ecosystem at Rensselaer and beyond:

    Quick Facts

    According to the November 2019 Top500 and Green500 supercomputer rankings, AiMOS is the…

    #1 most powerful supercomputer housed at a private university;
    #1 most energy efficient supercomputer in the U.S. (#3 in the world);
    And the #24 most powerful supercomputer in the world.

    How was AiMOS built?

    Built using the same IBM Power Systems technology as the world’s smartest supercomputers, Summit and Sierra, AiMOS uses a heterogeneous system architecture that includes IBM POWER9 CPUs connected to NVIDIA GPUs with the industry’s only CPU-to-GPU NVIDIA NVLink interface, which increases GPU memory bandwidth up to 5.6x.

    ORNL IBM AC922 SUMMIT supercomputer, No.1 on the TOP500. Credit: Carlos Jones, Oak Ridge National Laboratory/U.S. Dept. of Energy

    LLNL IBM NVIDIA Mellanox ATS-2 Sierra Supercomputer, NO.2 on the TOP500

    How powerful is AiMOS?

    AiMOS can deliver sustained performance of 8 petaflops. It has a peak performance of 12 petaflops.

    What are petaflops?

    FLOPS are floating-point operations per second. A floating-point operation is any mathematical operation that involves floating-point numbers, which are numbers that have decimal points in them. A petaflop is the ability to do 1 quadrillion — or one thousand million million — FLOPS.

    Is there an easier way to explain 8 petaflops?

    You could say that, in order to keep up with AiMOS, each of the roughly 8 billion people on our planet would have to perform 1 million calculations per second.

    Why is it called AiMOS?

    Amos Eaton, the co-founder and first senior professor at Rensselaer, has historically been the namesake of supercomputers at the Institute. While continuing to honor his legacy, the name has been tweaked slightly for this new machine to reflect its specialized configuration to enable advances in artificial intelligence research.

    How does AiMOS compare with previous supercomputers at Rensselaer?

    AiMOS is roughly 10 times more powerful than its immediate predecessor, the Blue Gene system named AMOS, and about 120 times more powerful than the first Rensselaer Blue Gene L supercomputer, activated 12 years ago.

    4
    AMOS IBM Blue Gene/Q supercomputer

    How will AiMOS be used?

    AiMOS will serve as the test bed for the New York State-IBM Research AI Hardware Center. A partnership between Rensselaer, IBM, and New York state, the AI Hardware Center is advancing development of computing chips and systems that are designed and optimized for AI workloads and are pushing the boundaries of AI performance. AiMOS will provide the modeling, simulation, and computation necessary to support the development of this hardware.

    AiMOS will also be accessible to collaborators from IBM Research, SUNY, and other public and private industry partners. It will also be used by Rensselaer faculty, students, and staff engaged in ongoing research collaborations that employ and advance AI technology, many of which involve IBM Research. These initiatives include the Rensselaer-IBM Artificial Intelligence Research Collaboration, which brings researchers at both institutions together to explore new frontiers in AI; the Cognitive and Immersive Systems Lab; and The Jefferson Project, which combines Internet of Things technology and powerful analytics to help manage and protect one of New York’s largest lakes, while creating a data-based blueprint for preserving bodies of fresh water around the globe.

    See the full article here .

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Rensselaer Campus

    Founded in 1824, Rensselaer Polytechnic Institute is America’s first technological research university. Rensselaer encompasses five schools, 32 research centers, more than 145 academic programs, and a dynamic community made up of more than 7,900 students and more than 100,000 living alumni. Rensselaer faculty and alumni include more than 145 National Academy members, six members of the National Inventors Hall of Fame, six National Medal of Technology winners, five National Medal of Science winners, and a Nobel Prize winner in Physics. With nearly 200 years of experience advancing scientific and technological knowledge, Rensselaer remains focused on addressing global challenges with a spirit of ingenuity and collaboration.

    With 7,900 students and more than 100,000 living alumni, Rensselaer is addressing the global challenges facing the 21st century—to change lives, to advance society, and to change the world.

    From renewable energy to cybersecurity, from biotechnology to materials science, from big data to nanotechnology, the world needs problem solvers—exactly the kind of talent Rensselaer produces—to address the urgent issues of today and the emerging issues of tomorrow.

     
  • richardmitnick 11:53 am on January 1, 2020 Permalink | Reply
    Tags: "Theta and the Future of Accelerator Programming at Argonne", , , , Supercomputing   

    From Argon ALCF via insideHPC: “Theta and the Future of Accelerator Programming at Argonne” 

    Argonne Lab
    News from Argonne National Laboratory

    From Argonne Leadership Computing Facility

    From insideHPC

    January 1, 2020
    Rich Brueckner


    In this video from the Argonne Training Program on Extreme-Scale Computing 2019, Scott Parker from Argonne presents: Theta and the Future of Accelerator Programming.

    ANL ALCF Theta Cray XC40 supercomputer

    Designed in collaboration with Intel and Cray, Theta is a 6.92-petaflops (Linpack) supercomputer based on the second-generation Intel Xeon Phi processor and Cray’s high-performance computing software stack. Capable of nearly 10 quadrillion calculations per second, Theta enables researchers to break new ground in scientific investigations that range from modeling the inner workings of the brain to developing new materials for renewable energy applications.

    “Theta’s unique architectural features represent a new and exciting era in simulation science capabilities,” said ALCF Director of Science Katherine Riley. “These same capabilities will also support data-driven and machine-learning problems, which are increasingly becoming significant drivers of large-scale scientific computing.”

    Scott Parker is the Lead for Performance Tools and Programming Models at the ALCF. He received his B.S. in Mechanical Engineering from Lehigh University, and a Ph.D. in Mechanical Engineering from the University of Illinois at Urbana-Champaign. Prior to joining Argonne, he worked at the National Center for Supercomputing Applications, where he focused on high-performance computing and scientific applications. At Argonne since 2008, he works on performance tools, performance optimization, and spectral element computational fluid dynamics solvers.

    See the full article here .

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Founded on December 28, 2006, insideHPC is a blog that distills news and events in the world of HPC and presents them in bite-sized nuggets of helpfulness as a resource for supercomputing professionals. As one reader said, we’re sifting through all the news so you don’t have to!

    If you would like to contact me with suggestions, comments, corrections, errors or new company announcements, please send me an email at rich@insidehpc.com. Or you can send me mail at:

    insideHPC
    2825 NW Upshur
    Suite G
    Portland, OR 97239

    Phone: (503) 877-5048

    Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation’s first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America’s scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science. For more visit http://www.anl.gov.

    About ALCF
    The Argonne Leadership Computing Facility’s (ALCF) mission is to accelerate major scientific discoveries and engineering breakthroughs for humanity by designing and providing world-leading computing facilities in partnership with the computational science community.

    We help researchers solve some of the world’s largest and most complex problems with our unique combination of supercomputing resources and expertise.

    ALCF projects cover many scientific disciplines, ranging from chemistry and biology to physics and materials science. Examples include modeling and simulation efforts to:

    Discover new materials for batteries
    Predict the impacts of global climate change
    Unravel the origins of the universe
    Develop renewable energy technologies

    Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science

    Argonne Lab Campus

     
  • richardmitnick 1:12 pm on December 20, 2019 Permalink | Reply
    Tags: "Using Satellites and Supercomputers to Track Arctic Volcanoes", , ArcticDEM project, , , NASA Terra MODIS, NASA Terra satellite, Supercomputing   

    From Eos: “Using Satellites and Supercomputers to Track Arctic Volcanoes” 

    From AGU
    Eos news bloc

    From Eos

    New data sets from the ArcticDEM project help scientists track elevation changes from natural hazards like volcanoes and landslides before, during, and long after the events.

    1
    The 2017 Okmok eruption resulted in a new volcanic cone, as well as consistent erosion of that cone’s flanks over subsequent years. Credit: NASA image courtesy of Jeff Schmaltz, MODIS Rapid Response Team, NASA-Goddard Space Flight Center

    NASA Terra MODIS schematic


    NASA Terra satellite

    Conical clues of volcanic activity speckle the Aleutian Islands, a chain that spans the meeting place of the Pacific Ring of Fire and the edge of the Arctic. (The chain also spans the U.S. state of Alaska and the Far Eastern Federal District of Russia.) Scientists are now turning to advanced satellite imagery and supercomputing to measure the scale of natural hazards like volcanic eruptions and landslides in the Aleutians and across the Arctic surface over time.

    When Mount Okmok, Alaska, unexpectedly erupted in July 2008, satellite images informed scientists that a new, 200-meter cone had grown beneath the ashy plume. But scientists suspected that topographic changes didn’t stop with the eruption and its immediate aftermath.

    For long-term monitoring of the eruption, Chunli Dai, a geoscientist and senior research associate at The Ohio State University, accessed an extensive collection of digital elevation models (DEMs) recently released by ArcticDEM, a joint initiative of the National Geospatial-Intelligence Agency and National Science Foundation. With ArcticDEM, satellite images from multiple angles are processed by the Blue Waters petascale supercomputer to provide elevation measures, producing high-resolution models of the Arctic surface.

    NCSA U Illinois Urbana-Champaign Blue Waters Cray Linux XE/XK hybrid machine supercomputer

    3
    In this map of ArcticDEM coverage, warmer colors indicate more overlapping data sets available for time series construction, and symbols indicate different natural events such as landslides (rectangles) and volcanoes (triangles). Credit: Chunli Dai

    Dai first utilized these models to measure variations in lava thickness and estimate the volume that erupted from Tolbachik volcano in Kamchatka, Russia, in work published in Geophysical Research Letters in 2017. The success of that research guided her current applications of ArcticDEM for terrain mapping.

    Monitoring long-term changes in a volcanic landscape is important, said Dai. “Ashes easily can flow away by water and by rain and then cause dramatic changes after the eruption,” she said. “Using this data, we can even see these changes…so that’s pretty new.”

    Creating time series algorithms with the ArcticDEM data set, Dai tracks elevation changes from natural events and demonstrates their potential for monitoring the Arctic region. Her work has already shown that erosion continues years after a volcanic event, providing first-of-their-kind measurements of posteruption changes to the landscape. Dai presented this research at AGU’s Fall Meeting.

    Elevating Measurement Methods

    “This is absolutely the best resolution DEM data we have,” said Hannah Dietterich, a research geophysicist at the U.S. Geological Survey’s Alaska Volcano Observatory not involved in the study. “Certainly, for volcanoes in Alaska, we are excited about this.”

    Volcanic events have traditionally been measured by aerial surveys or drones, which are expensive and time-consuming methods for long-term study. Once a hazardous event occurs, Dietterich explained, the “before” shots in before-and-after image sets are often missing. Now, ArcticDEM measurements spanning over a decade can be utilized to better understand and monitor changes to the Arctic surface shortly following such events, as well as years later.

    For example, the volcanic eruption at Okmok resulted in a sudden 200-meter elevation gain from the new cone’s formation but also showed continuing erosion rates along the cone flanks of up to 15 meters each year.

    Landslides and Climate

    For Dai, landslides provide an even more exciting application of ArcticDEM technology. Landslides are generally unmapped, she explained, whereas “we know the locations of volcanoes, so a lot of studies have been done.”

    Mass redistribution maps for both the Karrat Fjord landslide in Greenland in 2017 and the Taan Fiord landslide in Alaska in 2015 show significant mass wasting captured by DEMs before and after the events.

    “We’re hoping that our project with this new data program [will] provide a mass wasting inventory that’s really new to the community,” said Dai, “and people can use it, especially for seeing the connection to global warming.”

    Climate change is associated with many landslides studied by Dai and her team, who focus on mass wasting caused by thawing permafrost. ArcticDEM is not currently intended for predictive modeling, but as more data are collected over time, patterns may emerge that could help inform future permafrost loss or coastal retreat in the Arctic, according to Dietterich. “It is the best available archive of data for when crises happen.”

    Global climate trends indicate that Arctic environments will continue to change in the coming years. “If we can measure that, then we can get the linkage between global warming and its impact on the Arctic land,” said Dai.

    See the full article here.

    five-ways-keep-your-child-safe-school-shootings

    Please help promote STEM in your local schools.

    Stem Education Coalition

    Eos is the leading source for trustworthy news and perspectives about the Earth and space sciences and their impact. Its namesake is Eos, the Greek goddess of the dawn, who represents the light shed on understanding our planet and its environment in space by the Earth and space sciences.

     
  • richardmitnick 3:38 pm on December 19, 2019 Permalink | Reply
    Tags: , , , , , Simulations on Summit, , Supercomputing   

    From Oak Ridge National Laboratory: “With ADIOS, Summit processes celestial data at scale of massive future telescope” 

    i1

    From Oak Ridge National Laboratory

    December 19, 2019
    Scott S Jones
    jonesg@ornl.gov
    865.241.6491

    Researchers
    Scott A Klasky
    klasky@ornl.gov
    865.241.9980

    Ruonan Wang
    wangr1@ornl.gov
    865.574.8984

    Norbert Podhorszki
    pnb@ornl.gov
    865.574.7159

    For nearly three decades, scientists and engineers across the globe have worked on the Square Kilometre Array (SKA), a project focused on designing and building the world’s largest radio telescope.

    SKA Square Kilometer Array

    Although the SKA will collect enormous amounts of precise astronomical data in record time, scientific breakthroughs will only be possible with systems able to efficiently process that data.

    Because construction of the SKA is not scheduled to begin until 2021, researchers cannot collect enough observational data to practice analyzing the huge quantities experts anticipate the telescope will produce. Instead, a team from the International Centre for Radio Astronomy Research (ICRAR) in Australia, the Department of Energy’s (DOE’s) Oak Ridge National Laboratory (ORNL) in the United States, and the Shanghai Astronomical Observatory (SHAO) in China recently used Summit, the world’s most powerful supercomputer, to simulate the SKA’s expected output. Summit is located at the Oak Ridge Leadership Computing Facility, a DOE Office of Science User Facility at ORNL.

    ORNL IBM AC922 SUMMIT supercomputer, No.1 on the TOP500. Credit: Carlos Jones, Oak Ridge National Laboratory/U.S. Dept. of Energy

    3
    An artist rendering of the SKA’s low-frequency, cone-shaped antennas in Western Australia. Credit: SKA Project Office.

    “The Summit supercomputer provided a unique opportunity to test a simple SKA dataflow at the scale we are expecting from the telescope array,” said Andreas Wicenec, director of Data Intensive Astronomy at ICRAR.

    To process the simulated data, the team relied on the ORNL-developed Adaptable IO System (ADIOS), an open-source input/output (I/O) framework led by ORNL’s Scott Klasky, who also leads the laboratory’s scientific data group. ADIOS is designed to speed up simulations by increasing the efficiency of I/O operations and to facilitate data transfers between high-performance computing systems and other facilities, which would otherwise be a complex and time-consuming task.

    The SKA simulation on Summit marks the first time radio astronomy data have been processed at such a large scale and proves that scientists have the expertise, software tools, and computing resources that will be necessary to process and understand real data from the SKA.

    “The scientific data group is dedicated to researching next-generation technology that can be developed and deployed for the most scientifically demanding applications on the world’s fastest computers,” Klasky said. “I am proud of all the hard work the ADIOS team and the SKA scientists have done with ICRAR, ORNL, and SHAO.”

    Using two types of radio receivers, the telescope will detect radio light waves emanating from galaxies, the surroundings of black holes, and other objects of interest in outer space to help astronomers answer fundamental questions about the universe. Studying these weak, elusive waves requires an army of antennas.

    The first phase of the SKA will have more than 130,000 low-frequency, cone-shaped antennas located in Western Australia and about 200 higher frequency, dish-shaped antennas located in South Africa. The international project team will eventually manage close to a million antennas to conduct unprecedented studies of astronomical phenomena.

    To emulate the Western Australian portion of the SKA, the researchers ran two models on Summit—one of the antenna array and one of the early universe—through a software simulator designed by scientists from the University of Oxford that mimics the SKA’s data collection. The simulations generated 2.6 petabytes of data at 247 gigabytes per second.

    “Generating such a vast amount of data with the antenna array simulator requires a lot of power and thousands of graphics processing units to work properly,” said ORNL software engineer Ruonan Wang. “Summit is probably the only computer in the world that can do this.”

    Although the simulator typically runs on a single computer, the team used a specialized workflow management tool Wang helped ICRAR develop called the Data Activated Flow Graph Engine (DALiuGE) to efficiently scale the modeling capability up to 4,560 compute nodes on Summit. DALiuGE has built-in fault tolerance, ensuring that minor errors do not impede the workflow.

    “The problem with traditional resources is that one problem can make the entire job fall apart,” Wang said. Wang earned his doctorate degree at the University of Western Australia, which manages ICRAR along with Curtin University.

    The intense influx of data from the array simulations resulted in a performance bottleneck, which the team solved by reducing, processing, and storing the data using ADIOS. Researchers usually plug ADIOS straight into the I/O subsystem of a given application, but the simulator’s unusually complicated software meant the team had to customize a plug-in module to make the two resources compatible.

    “This was far more complex than a normal application,” Wang said.

    Wang began working on ADIOS1, the first iteration of the tool, 6 years ago during his time at ICRAR. Now, he serves as one of the main developers of the latest version, ADIOS2. His team aims to position ADIOS as a superior storage resource for the next generation of astronomy data and the default I/O solution for future telescopes beyond even the SKA’s gargantuan scope.

    “The faster we can process data, the better we can understand the universe,” he said.

    Funding for this work comes from DOE’s Office of Science.

    The International Centre for Radio Astronomy Research (ICRAR) is a joint venture between Curtin University and The University of Western Australia with support and funding from the State Government of Western Australia. ICRAR is helping to design and build the world’s largest radio telescope, the Square Kilometre Array.

    See the full article here .


    five-ways-keep-your-child-safe-school-shootings
    Please help promote STEM in your local schools.

    Stem Education Coalition

    ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science. DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.

    i2

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: