Tagged: Supercomputing Toggle Comment Threads | Keyboard Shortcuts

  • richardmitnick 8:34 am on July 23, 2016 Permalink | Reply
    Tags: , , PPPL and Princeton join high-performance software project, Supercomputing   

    From PPPL: “PPPL and Princeton join high-performance software project” 


    PPPL

    July 22, 2016
    John Greenwald

    1
    Co-principal investigators William Tang and Bei Wang. (Photo by Elle Starkman/Office of Communications)

    Princeton University and the U.S. Department of Energy’s Princeton Plasma Physics Laboratory (PPPL) are participating in the accelerated development of a modern high-performance computing code, or software package. Supporting this development is the Intel Parallel Computing Center (IPCC) Program, which provides funding to universities and laboratories to improve high-performance software capabilities for a wide range of disciplines.

    The project updates the GTC-Princeton (GTC-P) code, which was originally developed for fusion research applications at PPPL and has evolved into highly portable software that is deployed on supercomputers worldwide. The National Science Foundation (NSF) strongly supported advances in the code from 2011 through 2014 through the “G8” international extreme scale computing program, which represented the United States and seven other highly industrialized countries during that period.

    New activity

    Heading the new IPCC activity for the University’s Princeton Institute for Computational Science & Engineering (PICSciE) is William Tang, a PPPL physicist and PICSciE principal investigator (PI). Working with Tang is Co-PI Bei Wang, Associate Research Scholar at PICSciE, who leads this accelerated modernization effort. Joining them in the project are Co-PIs Carlos Rosales of the NSF’s Texas Advanced Computing Center at the University of Texas at Austin and Khaled Ibrahim of the Lawrence Berkeley National Laboratory.

    Dell Poweredge U Texas Austin Stampede Supercomputer. Texas Advanced Computer Center 9.6 PF
    Dell Poweredge U Texas Austin Stampede Supercomputer. Texas Advanced Computer Center 9.6 PF

    The current GTC-P code has advanced understanding of turbulence and confinement of the superhot plasma that fuels fusion reactions in doughnut-shaped facilities called tokamaks.

    PPPL NSTXII
    PPPL NSTX tokamak

    Understanding and controlling fusion fuel turbulence is a grand challenge of fusion science, and great progress has been made in recent years. It can determine how effectively a fusion reactor will contain energy generated by fusion reactions, and thus can strongly influence the eventual economic attractiveness of a fusion energy system. Further progress on the code will enable researchers to study conditions that arise as tokamaks increase in size to the enlarged dimensions of ITER — the flagship international fusion experiment under construction in France.

    ITER Tokamak
    ITER tokamak

    Access to Intel computer clusters

    Through the IPCC, Intel will provide access to systems for exploring the modernization of the code. Included will be clusters equipped with the most recent Intel “Knights Landing” (KNL) central processing chips.

    The upgrade will become part of the parent GTC code, which is led by Prof. Zhihong Lin of the University of California, Irvine, with Tang as co-PI. That code is also being modernized and will be proposed, together with GTC-P, to be included in the early science portfolio for the Aurora supercomputer.

    3
    Cray Aurora supercomputer to be built for ANL

    Aurora will begin operations at the Argonne Leadership Computing Facility, a DOE Office of Science User Facility at Argonne National Laboratory, in 2019. Powering Aurora will be Intel “Knights Hill” processing chips.

    Last year, the GTC and GTC-P codes were selected to be developed as an early science project designed for the Summit supercomputer that will be deployed at Oak Ridge Leadership Computing Facility, also a DOE Office of Science User Facility, at Oak Ridge National Laboratory in 2018.

    4
    IBM Summit supercomputer

    That modernization project differs from the one to be proposed for Aurora because Summit is being built around architecture powered by NVIDIA Volta graphical processing units and IBM Power 9 central processing chips.

    Moreover, the code planned for Summit will be designed to run on the Aurora platform as well.

    Boost U.S. computing power

    The two new machines will boost U.S. computing power far beyond Titan, the current leading U.S. supercomputer at Oak Ridge that can perform 27 quadrillion — or million billion — calculations per second. Summit and Aurora plan to perform some 200 quadrillion and 180 quadrillion calculations per second, respectively. Said Tang: “These new machines hold tremendous promise for helping to accelerate scientific discovery in many application domains, including fusion, that are of vital importance to the country.”

    PPPL, on Princeton University’s Forrestal Campus in Plainsboro, N.J., is devoted to creating new knowledge about the physics of plasmas — ultra-hot, charged gases — and to developing practical solutions for the creation of fusion energy. Results of PPPL research have ranged from a portable nuclear materials detector for anti-terrorist use to universally employed computer codes for analyzing and predicting the outcome of fusion experiments. The Laboratory is managed by the University for the U.S. Department of Energy’s Office of Science, which is the largest single supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov (link is external).

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition

    Princeton Plasma Physics Laboratory is a U.S. Department of Energy national laboratory managed by Princeton University. PPPL, on Princeton University’s Forrestal Campus in Plainsboro, N.J., is devoted to creating new knowledge about the physics of plasmas — ultra-hot, charged gases — and to developing practical solutions for the creation of fusion energy. Results of PPPL research have ranged from a portable nuclear materials detector for anti-terrorist use to universally employed computer codes for analyzing and predicting the outcome of fusion experiments. The Laboratory is managed by the University for the U.S. Department of Energy’s Office of Science, which is the largest single supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

     
  • richardmitnick 11:16 am on July 8, 2016 Permalink | Reply
    Tags: , , , Supercomputing   

    From Oak Ridge: “New 200-petaflop supercomputer to succeed Titan at ORNL” 

    i1

    Oak Ridge National Laboratory

    1
    Depiction of ORNL IBM Summit supercomputer

    A new 200-petaflop supercomputer will succeed Titan at Oak Ridge National Laboratory, and it could be available to scientists and researchers in 2018, a spokesperson said this week.

    The new IBM supercomputer, named Summit, could about double the computing power of what is now the world’s fastest machine, a Chinese system named Sunway TaihuLight, according to a seminannual list of the world’s top supercomputers released in June.

    Sunway TaihuLight is capable of 93 petaflops, according to the list, the TOP500 list. A petaflop is one quadrillion calculations per second. That’s 1,000 trillion calculations per second.

    Summit, which is expected to start operating at ORNL early in 2018, is one of three supercomputers that the U.S. Department of Energy expects to exceed 100 petaflops at three U.S. Department of Energy laboratories in 2018. The three planned systems are:

    the 200-petaflop Summit at ORNL, which is expected to be available to users in early 2018;

    a 150-petaflop machine known as Sierra at Lawrence Livermore National Laboratory near San Francisco in mid-2018;

    3
    IBM Sierra supercomputer depiction

    and
    a 180-petaflop supercomputer called Aurora at Argonne National Laboratory in Chicago in late 2018.

    4
    Cray Aurora supercomputer depiction

    “High performance computing remains an integral priority for the Department of Energy,” DOE Under Secretary Lynn Orr said. “Since 1993, our national supercomputing capabilities have grown exponentially by a factor of 300,000 to produce today’s machines like Titan at Oak Ridge National Lab. DOE has continually supported many of the world’s fastest, most powerful super-computers, and shared its facilities with universities and businesses ranging from auto manufacturers to pharmaceutical companies, enabling unimaginable economic benefits and leaps in science and technology, including the development of new materials for batteries and near zero-friction lubricants.”

    The supercomputers have also allowed the United States to maintain a safe, secure, and effective nuclear weapon stockpile, said Orr, DOE under secretary for science and energy.

    “DOE continues to lead in software and real world applications important to both science and industry,” he said. “Investments such as these continue to play a crucial role in U.S. economic competitiveness, scientific discovery, and national security.”

    At 200 petaflops, Summit would have at least five times as much power as ORNL’s 27-petaflop Titan. That system was the world’s fastest in November 2012 and recently achieved 17.59 petaflops on a test used by the TOP500 list that was released in June.

    Titan is used for research in areas such as materials research, nuclear energy, combustion, and climate science.

    “For several years, Titan has been the most scientifically productive in the world, allowing academic, government, and industry partners to do remarkable research in a variety of scientific fields,” ORNL spokesperson Morgan McCorkle said.

    Summit will be installed in a building close to Titan. Titan will continue operating while Summit is built and begins operating, McCorkle said.

    “That will ensure that scientific users have access to computing resources during the transition,” she said.

    Titan will then be decommissioned, McCorkle said.

    She said the total contract value for the new Summit supercomputer with all options and maintenance is $280 million. The U.S. Department of Energy is funding the project.

    McCorkle said the Oak Ridge Leadership Computing Facility at ORNL has been working with IBM, Nvidia, and Mellanox since 2014 to develop Summit.

    Like Titan, a Cray system, Summit will be part of the Oak Ridge Leadership Computing Facility, or OLCF. Researchers from around the world will be able to submit proposals to use the computer for a wide range of scientific applications, McCorkle said.

    She said the delivery of Summit will start at ORNL next year. Summit will be a hybrid computing system that uses traditional central processing units, or CPUs, and graphic processing units, or GPUs, which were first created for computer games.

    “We’re already scaling applications that will allow Summit to deliver an order of magnitude more science with at least 200 petaflops of compute power,” McCorkle said. “Early in 2018, users from around the world will have access to this resource.”

    Summit will have more than five times the computational power of Titan’s 18,688 nodes, using only about 3,400 nodes. Each Summit node will have IBM POWER9 CPUs and NVIDIA Volta GPUs connected with NVIDIA’s high-speed NVLinks and a huge amount of memory, according to the OLCF.

    Titan is also a hybrid system that combines CPUs with GPUs. That combination allowed the more powerful Titan to fit into the same space as Jaguar, an earlier supercomputer at ORNL, while using only slightly more electricity. That’s important because supercomputers can consume megawatts of power.

    China now has the top two supercomputers. Sunway TaihuLight was capable of 93 petaflops, and Tianhe-2, an Intel-based system ranked number two in the world, achieved 33.86 petaflops, according to the June version of the TOP500 list.

    But as planned, all three of the new DOE supercomputers would be more powerful than the top two Chinese systems.

    However, DOE officials said it’s not just about the hardware.

    “The strength of the U.S. program lies not just in hardware capability, but also in the ability to develop software that harnesses high-performance computing for real-world scientific and industrial applications,” DOE said. “American scientists have used DOE supercomputing capability to improve the performance of solar cells, to design new materials for batteries, to model the melting of ice sheets, to help optimize land use for biofuel crops, to model supernova explosions, to develop a near zero-fiction lubricant, and to improve laser radiation treatments for cancer, among countless other applications.

    Extensive work is already under way to prepare software and “real-world applications” to ensure that the new machines bring an immediate benefit to American science and industry, DOE said.

    “Investments such as these continue to play a crucial role in U.S. economic competitiveness, scientific discovery, and national security,” the department said.

    DOE said its supercomputers have more than 8,000 active users each year from universities, national laboratories, and industry.

    Among the supercomputer uses that DOE cited:

    Pratt and Whitney used the Argonne Leadership Computing Facility to improve the fuel efficiency of its Pure Power turbine engines.
    Boeing used the Oak Ridge Leadership Computing Facility to study the flow of debris to improve the safety of a thrust reverser for its new 787 Dreamliner.
    General Motors used the Oak Ridge Leadership Computing Facility to accelerate research on thermoelectric materials to help increase vehicle fuel efficiency.
    Proctor and Gamble used the Argonne Leadership Computing Facility to learn more about the molecular mechanisms of bubbles—important to the design of a wide range of consumer products.
    General Electric used the Oak Ridge Leadership Computing Facility to improve the efficiency of its world-leading turbines for electricity-generation.
    Navistar, NASA, the U.S. Air Force, and other industry leaders collaborated with scientists from Lawrence Livermore National Lab to develop technologies that increase semi-truck fuel efficiency by 17 percent.

    Though it was once the top supercomputer, Titan was bumped to number two behind Tianhe-2 in June 2013. It dropped to number three this June.

    As big as a basketball court, Titan is 10 times faster than Jaguar, the computer system it replaced. Jaguar, which was capable of about 2.5 petaflops, had ranked as the world’s fastest computer in November 2009 and June 2010.

    The new top supercomputer, Sunway TaihuLight, was developed by the National Research Center of Parallel Computer Engineering and Technology, or NRCPC, and installed at the National Supercomputing Center in Wuxi, China.

    Tianhe-2 was developed by China’s National University of Defense Technology.

    In the United States, DOE said its Office of Science and National Nuclear Security Administration are collaborating with other U.S. agencies, industry, and academia to pursue the goals of what is known as the National Strategic Computing Initiative:

    accelerating the delivery of “exascale” computing;
    increasing the coherence between the technology base used for modeling and simulation and that for data analytic computing;
    charting a path forward to a post-Moore’s Law era; and
    building the overall capacity and capability of an enduring national high-performance computing ecosystem.

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition

    ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science. DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.

    i2

     
  • richardmitnick 7:06 pm on July 7, 2016 Permalink | Reply
    Tags: , , OCLF, Supercomputing   

    From Oak Ridge: “One Billion Processor Hours Awarded to 22 Projects through ALCC” 

    i1

    Oak Ridge National Laboratory

    July 5, 2016
    Maleia Wood

    2016–17 ALCC projects allocated time on OLCF’s world-class resources

    ORNL OLCF

    1
    ALCC’s mission is to provide high-performance computing resources to projects that align with DOE’s broad energy mission, with an emphasis on high-risk, high-return simulations.

    The US Department of Energy (DOE) Office of Science has awarded nearly 1 billion processor hours to 22 projects at the Oak Ridge Leadership Computing Facility (OLCF)— a DOE Office of Science User Facility located at DOE’s Oak Ridge National Laboratory—through the DOE Office of Advanced Scientific Computing Research Leadership Computing Challenge (ALCC).

    ALCC’s mission is to provide high-performance computing resources to projects that align with DOE’s broad energy mission, with an emphasis on high-risk, high-return simulations. The ALCC program allocates up to 30 percent of the computational resources at the OLCF and the Argonne Leadership Computing Facility, as well as up to 10 percent at the National Energy Research Scientific Computing Center.

    “In addition to supporting the DOE mission, the program also seeks to broaden the field of researchers able to use some of the world’s fastest and most powerful supercomputers like Titan at the OLCF,” said Jack Wells, OLCF director of science.

    The ALCC grants 1-year awards and supports scientists from industry, academia, and national laboratories who are advancing scientific and technological research in energy-related fields. Past ALCC allocations contributed to scientific discovery in energy efficiency, computer science, climate modeling, materials science, bioenergy, and basic research.

    The 2016 projects will continue that reputation of discovery with topics that range from the biology of neurotransmitters to the search for affordable catalysts to the development of future clean energy technology. Scientific domains represented among the awards include biology, climate science, engineering, computer science, nuclear fusion, cosmology, materials science, nuclear engineering, and nuclear physics.

    ORNL Cray Titan Supercomputer
    ORNL Cray Titan Supercomputer

    Awards on Titan—ranging from 9 million to 167 million processor hours—went to projects that include the following:

    Materials Science. Catalysis plays a critical role both in the current energy landscape and by potentially enabling future clean energy technologies. A catalyst facilitates a chemical reaction and increases the reaction rate, but the substance isn’t consumed in the reaction and is, therefore, available to ease subsequent reactions. Breakthroughs in catalyst design paradigms could significantly increase energy efficiency. The search for effective and affordable catalysts is critical in both chemistry and materials science, as well as in large-scale industrial concerns.

    The computing resources on Titan will allow a team led by Efthimios Kaxiras from Harvard University to use high-throughput computation to generate datasets that will augment scientists’ abilities to predict useful catalysts. The team will focus on using nanoporous gold, which is an active, stable, and highly selective catalyst, in a particular reaction—anhydrous dehydrogenation of methanol to formaldehyde (a common chemical with multiple industrial uses).

    Catalysts traditionally have been developed using experimental trial and error or by testing known catalysts for similar reactions. The primary obstacles to designing new novel catalysts are twofold: the complexity of catalytic materials and the wide range of possible catalytic materials.

    The team is searching for an alloy catalyst that can produce formaldehyde from methanol without producing water, which involves energy-intensive separation steps. Because it is impossible to experimentally synthesize and test tens of thousands of possible bimetallic catalysts, the researchers will use Titan to computationally perform the screening.

    Biology. A team led by Cornell University’s Harel Weinstein seeks to determine the functional properties and energy-conserving mechanisms of cellular membrane proteins called neurotransmitter transporters. Specifically, the team hopes to uncover the biological machinery of neurotransmitter sodium symporters, a family of neurotransmitter transporters responsible for the release and reuptake of chemical signals between neurons.

    A major focus of the team is the dopamine transporter (DAT), the gatekeeper for the neurotransmitter dopamine that is associated with reward-motivated behavior. By simulating DAT, Weinstein and his collaborators hope not only to learn how cells harness energy to move molecules against a concentration gradient but also to uncover potential strategies for treating DAT-related disorders such as addiction and depression.

    Using molecular dynamics and high-performance computing, the team will be able to gain a clearer picture of how the transporter works at the molecular level—how energy is gained, stored, and used. Additionally, simulation of updated models could shed light on DAT mutations related to diseases such as autism, Parkinson’s disease, and attention deficit hyperactivity disorder, which have been shown to be affected by malfunctions of the neurotransmission process.

    Nuclear Physics. The accurate description of nuclear fission is relevant to a number of fields, including basic energy, nuclear waste disposal, national security, and nuclear forensics. The current theoretical description is based on limited models that rely on mathematical shortcuts and constraints and on a large collection of experimental data accumulated since 1939.

    Because many aspects of nuclear fission cannot be probed in the laboratory, devising a microscopic theory based on the fundamental properties of nuclear forces is a highly desirable goal. A team led by the University of Washington’s Aurel Bulgac will use Titan to study the sensitivity of fission fragment properties and fission dynamics using a novel theoretical approach that extends the widely used density functional theory (DFT) to superfluid nuclei. In particular, the team will focus on fission fragment excitation energies and total kinetic energy, which are difficult to extract using phenomenological models.

    In previous work involving Titan, the team developed a real-time DFT extension that explicitly includes the full dynamics of the crucial pairing correlations. Applying the method to a fissioning plutonium-240 nucleus, the team determined the final stages of fission last about 10 times longer than previously calculated. The code is one of the first in nuclear theory to take full advantage of GPU accelerators.

    Engineering. The High Performance Computing for Manufacturing (HPC4Mfg) program pairs US manufacturers with national labs’ world-class computing experts and advanced computing resources to address key challenges in US manufacturing. The solutions resulting from this collaboration will have broad industry and national impact.

    Three companies—Global Foundries, General Electric, and United Technologies Research Center—will use Titan as part of the HPC4Mfg program seeking to deliver solutions that can revolutionize the manufacturing industry through energy efficiency and increased innovation.

    Oak Ridge National Laboratory is supported by the US Department of Energy’s Office of Science. The single largest supporter of basic research in the physical sciences in the United States, the Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition

    ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science. DOE’s Office of Science is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time.

    i2

     
  • richardmitnick 5:32 pm on June 29, 2016 Permalink | Reply
    Tags: , , LLNL IBM Sierra supercomputer, Supercomputing   

    From LLNL: “Lawrence Livermore National Laboratory dedicates new supercomputer facility” 


    Lawrence Livermore National Laboratory

    Officials from the Department of Energy’s National Nuclear Security Administration (link is external) (NNSA) and government representatives today dedicated a new supercomputing facility at Lawrence Livermore National Laboratory (LLNL).

    The $9.8 million modular and sustainable facility provides the Laboratory flexibility to accommodate future advances in computer technology and meet a rapidly growing demand for unclassified high-performance computing (HPC). The facility houses supercomputing systems in support of NNSA’s Advanced Simulation and Computing (ASC) program. ASC is an essential and integral part of NNSA’s Stockpile Stewardship Program to ensure the safety, security and effectiveness of the nation’s nuclear deterrent without additional underground testing.

    “High performance computing is absolutely essential to the science and engineering that underpins our work in stockpile stewardship and national security. The unclassified computing capabilities at this facility will allow us to engage the young talent in academia on which NNSA’s future mission work will depend,” said NNSA Administrator Lt. Gen. Frank G. Klotz USAF (Ret.).

    Also in attendance at the dedication was Livermore Mayor John Marchand. Charles Verdon, LLNL principal associate director for Weapons and Complex Integration, presided over the ceremony.

    1
    Kim Cupps, Computing department head at Lawrence Livermore National Laboratory, gives a tour of the new computing facility.

    “The opening of this new facility underscores the vitality of Livermore’s world-class efforts to advance the state of the art in high performance computing,” said Bill Goldstein, LLNL director. “This facility provides the Laboratory the flexibility to accommodate future computing architectures and optimize their efficient use for applications to urgent national and global challenges.”

    Located on Lawrence Livermore’s east side, the new facility adjoins the Livermore Valley Open Campus. Located outside LLNL’s high-security perimeter, the open campus is home to LLNL’s High Performance Computing Innovation Center and facilitates collaboration with industry and academia to foster the innovation of new technologies.

    The new dual-level building consists of a 6,000-square-foot machine floor flanked by support space. The main computer structure is flexible in design to allow for expansion and the testing of future computer technology advances.

    The facility is now home to some of the systems acquired as part of the Commodity Technology Systems-1 (CTS-1) procurement announced in October. Delivery of those systems began in April. The Laboratory also intends to house in FY18 a powerful, but smaller, unclassified companion to the IBM “Sierra” system.

    IBM Sierra supercomputer
    LLNL IBM Sierra supercomputer

    It will support academic alliances, as well as other efforts of national importance, including the DOE-wide exascale computing project. The Sierra supercomputer will be delivered to Livermore starting in late 2017 under the tri-lab Collaboration of Oak Ridge, Argonne and Livermore (CORAL) multi-system procurement announced in November 2014. The Sierra system is expected to be capable of about 150 petaflops (quadrillion floating operations per second).

    In-house modeling and simulation expertise in energy-efficient building design was used in drawing up the specifications for the facility; heating, ventilation and air conditioning systems to meet federal sustainable design requirements to promote energy conservation. The flexible design will accommodate future liquid cooling solutions for HPC systems. The building is able to scale to 7.5 megawatts of electric power to support future platforms and was designed so that power and mechanical resources can be added as HPC technologies evolve.

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition
    LLNL Campus

    Operated by Lawrence Livermore National Security, LLC, for the Department of Energy’s National Nuclear Security
    Administration
    DOE Seal
    NNSA

     
  • richardmitnick 6:07 pm on June 20, 2016 Permalink | Reply
    Tags: , , , Supercomputing   

    Rutgers New Supercomputer Ranked #2 among Big Ten Universities, #8 among U.S. Academic Institutions by the Top500 List 

    The updated Top 500 ranking of world’s most powerful supercomputers issued today ranks Rutgers’ new academic supercomputer #2 among Big Ten universities, #8 among U.S. academic institutions, #49 among academic institutions globally, and #165 among all supercomputers worldwide.

    The Top 500 project provides a reliable basis for tracking and detecting trends in high-performance computing. Twice each year it assembles and releases a list of the sites operating the 500 most powerful computer systems in the world.

    Rutgers’ new supercomputer, which is named “Caliburn,” is the most powerful system in the state. It was built with a $10 million award to Rutgers from the New Jersey Higher Education Leasing Fund. The lead contractor is HighPoint Solutions of Bridgewater, N.J., which was chosen as the lead contractor after a competitive bidding process. The system manufacturer and integrator is Super Micro Computer Inc. of San Jose, Calif.

    Source: Rutgers New Supercomputer Ranked #2 among Big Ten Universities, #8 among U.S. Academic Institutions by the Top500 List

    Rutgersensis

     
  • richardmitnick 11:14 am on June 3, 2016 Permalink | Reply
    Tags: , , Supercomputing,   

    From ALCF: “3D simulations illuminate supernova explosions” 

    ANL Lab
    News from Argonne National Laboratory

    June 1, 2016
    Jim Collins

    1
    Top: This visualization is a volume rendering of a massive star’s radial velocity. In comparison to previous 1D simulations, none of the structure seen here would be present.

    2

    Bottom: Magnetohydrodynamic turbulence powered by neutrino-driven convection behind the stalled shock of a core-collapse supernova simulation. This simulation shows that the presence of rotation and weak magnetic fields dramatically impacts the development of the supernova mechanism as compared to non-rotating, non-magnetic stars. The nascent neutron star is just barely visible in the center below the turbulent convection.

    Credit:
    Sean M. Couch, Michigan State University

    Researchers from Michigan State University are using Mira to perform large-scale 3D simulations of the final moments of a supernova’s life cycle.

    MIRA IBM Blue Gene Q supercomputer at the Argonne Leadership Computing Facility
    MIRA IBM Blue Gene Q supercomputer at the Argonne Leadership Computing Facility

    While the 3D simulation approach is still in its infancy, early results indicate that the models are providing a clearer picture of the mechanisms that drive supernova explosions than ever before.

    In the landmark television series “Cosmos,” astronomer Carl Sagan famously proclaimed, “we are made of star stuff,” in reference to the ubiquitous impact of supernovas.

    At the end of their life cycles, these massive stars explode in spectacular fashion, scattering their guts—which consist of carbon, iron, and basically all other natural elements—across the cosmos. These elements go on to form new stars, solar systems, and everything else in the universe (including the building blocks for life on Earth).

    Despite this fundamental role in cosmology, the mechanisms that drive supernova explosions are still not well understood.

    “If we want to understand the chemical evolution of the entire universe and how the stuff that we’re made of was processed and distributed throughout the universe, we have to understand the supernova mechanism,” said Sean Couch, assistant professor of physics and astronomy at Michigan State University.

    To shed light on this complex phenomenon, Couch is leading an effort to use Mira, the Argonne Leadership Computing Facility’s (ALCF’s) 10-petaflops supercomputer, to carry out some of the largest and most detailed 3D simulations ever performed of core-collapse supernovas. The ALCF is a U.S. Department of Energy (DOE) Office of Science User Facility.

    After millions of years of burning ever-heavier elements, these super-giant stars (at least eight solar masses, or eight times the mass of the sun) eventually run out of nuclear fuel and develop an iron core. No longer able to support themselves against their own immense gravitational pull, they start to collapse. But a process, not yet fully understood, intervenes that reverses the collapse and causes the star to explode.

    “What theorists like me are trying to understand is that in-between step,” Couch said. “How do we go from this collapsing iron core to an explosion?”

    Through his work at the ALCF, Couch and his team are developing and demonstrating a high-fidelity 3D simulation approach that is providing a more realistic look at this “in-between step” than previous supernova simulations.

    While this 3D method is still in its infancy, Couch’s early results have been promising. In 2015, his team published a paper* in the Astrophysical Journal Letters, detailing their 3D simulations of the final three minutes of iron core growth in a 15 solar-mass star. They found that more accurate representations of the star’s structure and the motion generated by turbulent convection (measured at several hundred kilometers per second) play a substantial role at the point of collapse.

    “Not surprisingly, we’re showing that more realistic initial conditions have a significant impact on the results,” Couch said.

    Adding another dimension

    Despite the fact that stars rotate, have magnetic fields, and are not perfect spheres, most 1D and 2D supernova simulations to date have modeled non-rotating, non-magnetic, spherically symmetric stars. Scientists were forced to take this simplified approach because modeling supernovas is an extremely computationally demanding task. Such simulations involve highly complex multiphysics calculations and extreme timescales (the stars evolve over millions of years, yet the supernova mechanism occurs in a second).

    According to Couch, working with unrealistic initial conditions has led to difficulties in triggering robust and consistent explosions in simulations—a long-standing challenge in computational astrophysics.

    However, thanks to recent advances in computing hardware and software, Couch and his peers are making significant strides toward more accurate supernova simulations by employing the 3D approach.

    The emergence of petascale supercomputers like Mira has made it possible to include high-fidelity treatments of rotation, magnetic fields, and other complex physics processes that were not feasible in the past.

    “Generally when we’ve done these kinds of simulations in the past, we’ve ignored the fact that magnetic fields exist in the universe because when you add them into a calculation, it increases the complexity by about a factor of two,” Couch said. “But with our simulations on Mira, we’re finding that magnetic fields can add a little extra kick at just the right time to help push the supernova toward explosion.”

    Advances to the team’s open-source FLASH hydrodynamics code have also aided simulation efforts. Couch, a co-developer of FLASH, was involved in porting and optimizing the code for Mira as part of the ALCF’s Early Science Program in 2012. For his current project, Couch continues to collaborate with ALCF computational scientists to enhance the performance, scalability, and capabilities of FLASH to carry out certain tasks. For example, ALCF staff modified the code for writing Hierarchical Data Format (HDF5) files that sped up I/O performance by about a factor of 10.

    But even with today’s high-performance computing hardware and software, it is not yet feasible to include high-fidelity treatments of all the relevant physics in a single simulation; that would require a future exascale system, Couch said. For their ongoing simulations, Couch and his team have been forced to make a number of approximations, including a reduced nuclear network and simulating only one eighth of the full star.

    “Our simulations are only a first step toward truly realistic 3D simulations of supernova,” Couch said. “But they are already providing a proof-of-principle that the final minutes of a massive star evolution can and should be simulated in 3D.”

    The team’s results were published in Astrophysical Journal Letters in a 2015 paper titled “The Three-Dimensional Evolution to Core Collapse of a Massive Star.” The study also used computing resources at the Texas Advanced Computing Center at the University of Texas at Austin.

    Couch’s supernova research began at the ALCF with a Director’s Discretionary award and now continues with computing time awarded through DOE’s Innovative and Novel Computational Impact on Theory and Experiment (INCITE) program. This work is being funded by the DOE Office of Science and the National Science Foundation.

    See the full article here .

    Please help promote STEM in your local schools.
    STEM Icon
    Stem Education Coalition
    Argonne National Laboratory seeks solutions to pressing national problems in science and technology. The nation’s first national laboratory, Argonne conducts leading-edge basic and applied scientific research in virtually every scientific discipline. Argonne researchers work closely with researchers from hundreds of companies, universities, and federal, state and municipal agencies to help them solve their specific problems, advance America’s scientific leadership and prepare the nation for a better future. With employees from more than 60 nations, Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science. For more visit http://www.anl.gov.

    The Advanced Photon Source at Argonne National Laboratory is one of five national synchrotron radiation light sources supported by the U.S. Department of Energy’s Office of Science to carry out applied and basic research to understand, predict, and ultimately control matter and energy at the electronic, atomic, and molecular levels, provide the foundations for new energy technologies, and support DOE missions in energy, environment, and national security. To learn more about the Office of Science X-ray user facilities, visit http://science.energy.gov/user-facilities/basic-energy-sciences/.

    Argonne is managed by UChicago Argonne, LLC for the U.S. Department of Energy’s Office of Science

    Argonne Lab Campus

     
  • richardmitnick 10:14 am on April 5, 2016 Permalink | Reply
    Tags: , Cosmic Origins, , Supercomputing   

    From Science Node: “Toward a realistic cosmic evolution” 

    Science Node bloc
    Science Node

    1
    Courtesy Cosmology and Astroparticle Physics Group University of Geneva. Switzerland Supercomputing Center.

    23 Mar, 2016 [Just popped up]
    Simone Ulmer

    Scientists exploring the universe have at their disposal research facilities such as at Laser Interferometer Gravitational-Wave Observatory (LIGO) — which recently achieved the breakthrough detection of gravitational waves — as well as telescopes and space probes.

    MIT/Caltech Advanced aLIGO Hanford Washington USA installation
    MIT/Caltech Advanced aLIGO Hanford Washington USA installation

    ESO/VLT
    ESO/VLT

    Keck Observatory, Mauna Kea, Hawaii, USA
    Keck Observatory, Mauna Kea, Hawaii, USA

    NASA/ESA Hubble Telescope
    NASA/ESA Hubble Telescope

    NASA/Spitzer Telescope
    NASA/Spitzer Telescope

    Considering that the Big Bang does not lend itself to experimental re-enactment, researchers must use supercomputers like the Piz Daint of the Swiss National Supercomputing Center (CSCS) to simulate the evolution of cosmic structures.

    Piz Daint supercomputer of the Swiss National Supercomputing Center (CSCS)
    Piz Daint supercomputer of the Swiss National Supercomputing Center (CSCS)


    Access mp4 video here .
    The Piz Daint supercomputer calculated 40963 grid points and 67 billion particles to help scientists visualize these gravitational waves. Courtesy Cosmology and Astroparticle Physics Group University of Geneva and the Swiss National Supercomputing Center.

    This entails modeling a complex, dynamical system that acts at vastly different scales of magnitude and contains a gigantic number of particles. With the help of such simulations, researchers can determine the movement of those particles and hence their formation into structures under the influence of gravitational forces at cosmological scales.

    To date, simulations like these have been entirely based on Newton’s law of gravitation. Yet this is formulated for classical physics and mechanics. It operates within an absolute space-time, where the cosmic event horizon of the expanding universe does not exist. It is also of no use in describing gravitational waves, or the rotation of space-time known as ‘frame-dragging’. Yet in the real expanding universe, space-time is dynamical. And, according to the general theory of relativity, masses such as stars or planets can give it curvature.

    Consistent application of the general theory of relativity

    Led by postdoctoral researcher Julian Adamek and PhD student David Daverio under the supervision of Martin Kunz and Ruth Durrer, the researchers of the Cosmology and Astroparticle Physics Group at the University of Geneva tackled their objective of developing a realistic code. This meant the equations to be solved in the code should make consistent use of the general theory of relativity in cosmic structure evolution simulation, which entails calculating gravitational waves as well as frame-dragging.

    The research team presents the code and the results in the current issue of the journal Nature Physics.

    4
    An image of the flow field where moving masses cause space-time to be pulled along slightly (frame-dragging). The yellow-orange collections are regions of high particle density, corresponding to the clustered galaxies of the real universe. Courtesy Cosmology and Astroparticle Physics Group University of Geneva, Switzerland Supercomputing Center.

    To allow existing simulations to model cosmological structure formation, one needs to calculate approximately how fast the universe would be expanding at any given moment. That result can then be fed into the simulation.

    “The traditional methods work well for non-relativistic matter such as atomic building blocks and cold dark matter, as well as at a small scale where the cosmos can be considered homogeneous and isotropic,” says Kunz.

    But given that Newtonian physics knows no cosmic horizon, the method has only limited applicability at large scales or to neutrinos, gravitational waves, and similar relativistic matter. Since this is an approximation to a dynamical system, it may happen that a simulation of the creation of the cosmos shows neutrinos moving at faster-than-light speeds. Such simulations are therefore subject to uncertainty.
    Self-regulating calculations

    With the new method the system might now be said to regulate itself and exclude such errors, explains Kunz. In addition, the numerical code can be used for simulating various models that bring into play relativistic sources such as dynamical dark energy, relativistic particles and topological defects, all the way to core collapse supernovae (stellar explosions).

    There are two parts to the simulation code. David Daverio was instrumental in developing and refining the part named ‘LATfield2’ to make it perform highly parallel and efficient calculations on a supercomputer. This library manages the basic tools for field-based particle-mesh N-body codes, i.e. the grid spanning the simulation space, the particles and fields acting therein, and the fast Fourier transform necessary for solving the model’s constituent equations, developed largely by Julian Adamek.

    These equations resulted in the second part of the code, ‘gevolution,’ that ensures the calculations take into account the general theory of relativity. The equations describe interactions between matter, space, and time that describe gravitation in terms of curved four-dimensional space-time.

    “Key to the simulation are the metrics describing space-time curvature, and the stress-energy tensor describing distribution of matter,” says Kunz.

    The largest simulation conducted on Piz Daint consisted of a cube with 4,0963 grid points and 67 billion particles. The scientists simulated regions with weak gravitational fields and other weak relativistic effects using the new code. Thus, for the first time it was possible to fully calculate the gravitational waves and rotating space-time induced by structure formation.


    Access mp4 video here .
    Spin cycle. A visualization of the rotation of space-time. Courtesy Cosmology and Astroparticle Physics Group University of Geneva and the Swiss National Supercomputing Center.

    The scientists compared the results with those they computed using a conventional, Newtonian code, and found only minor differences. Accordingly, it appears that structure formation in the universe has little impact on its rate of expansion.

    “For the conventional standard model to work, however, dark energy has to be a cosmological constant and thus have no dynamics,” says Adamek. Based on current knowledge, this is by no means established. “Our method now facilitates the consistent simulation and study of alternative scenarios.”
    Elegant approach

    With the new method, the researchers have managed — without significantly complicating the computational effort — to consistently integrate the general theory of relativity, 100 years after its formulation by Albert Einstein, with the dynamical simulation of structure formation in the universe. The researchers say that their method of implementing the general theory of relativity is an elegant approach to calculating a realistic distribution of radiation or very high-velocity particles in a way that considers gravitational waves and the rotation of space-time.

    General relativity and cosmic structure formation

    Julian Adamek, David Daverio, Ruth Durrer & Martin Kunz

    Affiliations

    Département de Physique Théorique & Center for Astroparticle Physics, Université de Genève, 24 Quai E. Ansermet, 1211 Genève 4, Switzerland
    Julian Adamek, David Daverio, Ruth Durrer & Martin Kunz
    African Institute for Mathematical Sciences, 6 Melrose Road, Muizenberg 7945, South Africa
    Martin Kunz

    Contributions

    J.A. worked out the equations in our approximation scheme and implemented the cosmological code gevolution. He also produced the figures. D.D. developed and implemented the particle handler for the LATfield2 framework. R.D. contributed to the development of the approximation scheme and the derivation of the equations. M.K. proposed the original idea. All authors discussed the research and helped with writing the paper.

    See the full article here .

    Please help promote STEM in your local schools.
    STEM Icon

    Stem Education Coalition

    Science Node is an international weekly online publication that covers distributed computing and the research it enables.

    “We report on all aspects of distributed computing technology, such as grids and clouds. We also regularly feature articles on distributed computing-enabled research in a large variety of disciplines, including physics, biology, sociology, earth sciences, archaeology, medicine, disaster management, crime, and art. (Note that we do not cover stories that are purely about commercial technology.)

    In its current incarnation, Science Node is also an online destination where you can host a profile and blog, and find and disseminate announcements and information about events, deadlines, and jobs. In the near future it will also be a place where you can network with colleagues.

    You can read Science Node via our homepage, RSS, or email. For the complete iSGTW experience, sign up for an account or log in with OpenID and manage your email subscription from your account preferences. If you do not wish to access the website’s features, you can just subscribe to the weekly email.”

     
  • richardmitnick 2:45 pm on March 30, 2016 Permalink | Reply
    Tags: , , Supercomputing   

    From SN: “Simulating stars with less computing power” 

    Science Node bloc
    Science Node

    30 Mar, 2016
    David Lugmayer

    For the first time, a research team has harnessed the JUQUEEN supercomputer to find a way to simulate the fusion of heavier elements within stars.

    JUQUEEN - Jülich Blue Gene Q IBM
    JUQUEEN – Jülich Blue Gene Q IBM supercomputer

    An international collaboration of researchers has developed a new method to simulate the creation of elements inside stars. The process was developed by researchers from the University of Bonn, and University of Bochum, Germany, working with North Carolina State University, Mississippi State University, and the Jülich research center in Germany. The goal of their work was to devise a way to allow simulations of this type to be conducted with less computational power. With this method they were able to model a more complex process that was not previously possible.

    A large part of a star’s life is governed by the process of thermonuclear fusion, through which hydrogen atoms are converted into helium at the core of the star. But fusion also creates a host of other elements in the core of the star, produced by the fusion of the nuclei of helium atoms, which are also known as alpha particles.

    But when scientists want to observe these processes they come up against a problem: the conditions inside the core of a star (15 million degrees Celsius in the case of our sun) are not reproducible inside a laboratory. Thus, the only way to recreate the processes inside a star is to use ‘ab-initio’ computer simulations.

    To allow more effective ab-initio simulation, the team devised a new technique that involves simulating the nucleons (the subatomic particles that comprise the nucleus of atoms) on a virtual lattice grid, instead of in free space. This allowed for very efficient calculation by parallel processing from supercomputers, and significantly lowered the computational demand required for the simulation.

    Using this technique, with the help of the supercomputer JUQUEEN at the Jülich Supercomputing Center, a simulation of the scattering and deflection of two helium nuclei was carried out that involved a grand total of eight nucleons. This may not sound extraordinary, but it is in fact unprecedented, as up until now even the fastest supercomputers in the world could only simulate the very lightest of elements involving a maximum of five total nucleons.

    The problem comes when the number of nucleons simulated is increased. Each of these particles interacts with every other particle present, which must be simulated along with the quantum state of each particle. “All existing methods have an exponential scaling of computing resources with the number of particles,” explains Ulf Meißner from the University of Bonn.

    The difference this makes to the simulation process is astounding: this particular simulation required roughly two million core hours of computing using the new method, which on a supercomputer as powerful as JUQUEEN could take only a number of days to run. However, the same simulation run with older methods would take JUQUEEN several thousand years to complete.

    “This is a major step in nuclear theory” says Meißner. He explains that this new method makes more advanced ab-initio simulations of element generation in stars possible. The next step Meißner and his colleagues are working towards is the ab-initio calculation of the “holy grail of nuclear astrophysics”: the process through which oxygen is generated in stars.

    See the full article here .

    Please help promote STEM in your local schools.
    STEM Icon

    Stem Education Coalition

    Science Node is an international weekly online publication that covers distributed computing and the research it enables.

    “We report on all aspects of distributed computing technology, such as grids and clouds. We also regularly feature articles on distributed computing-enabled research in a large variety of disciplines, including physics, biology, sociology, earth sciences, archaeology, medicine, disaster management, crime, and art. (Note that we do not cover stories that are purely about commercial technology.)

    In its current incarnation, Science Node is also an online destination where you can host a profile and blog, and find and disseminate announcements and information about events, deadlines, and jobs. In the near future it will also be a place where you can network with colleagues.

    You can read Science Node via our homepage, RSS, or email. For the complete iSGTW experience, sign up for an account or log in with OpenID and manage your email subscription from your account preferences. If you do not wish to access the website’s features, you can just subscribe to the weekly email.”

     
  • richardmitnick 10:57 am on March 18, 2016 Permalink | Reply
    Tags: , , Supercomputing   

    From Node: “Why should I believe your HPC research?” 

    Science Node bloc
    Science Node

    16 Mar, 2016
    Lorena Barba

    The strategy for scientific computing today is roughly the same as it was at its historical beginnings. When do we have evidence that claims to knowledge originating from simulation are justified?

    Supercomputing propels science forward in fields like climate change, precision medicine, and astrophysics. It is considered a vital part in the scientific endeavor today.

    The strategy for scientific computing: Start with a trusted mathematical model, transform it into a computable form, and express the algorithm into computer code that is executed to produce a result. This result is inferred to give information about the original physical system.

    In this way, computer simulations originate new claims to scientific knowledge. But when do we have evidence that claims to knowledge originating from simulation are justified? Questions like these were raised by Eric Winsberg in his book Science in the Age of Computer Simulation.

    In many engineering applications of computer simulations, we are used to speaking about verification and validation (V&V). Verification means confirming the simulation results match the solutions to the mathematical model. Validation means confirming that the simulation represents the physical phenomenon well. In other words, V&V separates the issues of solving the equations right, versus solving the right equations.

    If a published computational research reports on completing a careful V&V study, we are likely to trust the results more. But is it enough? Does it really produce reliable results, which we trust to create knowledge?

    Texas Stampede Supercomputer Texas Advanced Computer Center
    Texas Stampede Supercomputer. Texas Advanced Computer Center

    Thirty years ago, the same issues were being raised about experiments: How do we come to rationally believe in an experimental result? Allan Franklin wrote about the strategies that experimental scientists use to provide grounds for rational belief in experimental results. For example: confidence in an instrument increases if we can use it to get results that are expected. Or we gain confidence in an experimental result if it can be replicated with a different instrument/apparatus.

    The question of whether we have evidence that claims to scientific knowledge stemming from simulation are justified is not so clear as V&V. When we compare results with other simulations, for example, simulations that used a different algorithm or a more refined model, this does not fit neatly into V&V.

    And our work is not done when a simulation completes. Data requires interpretation, visualization, and analysis — all crucial for reproducibility. We usually try to summarize qualitative features of the system under study, and generalize these features to a class of similar phenomena (i.e. managing uncertainties).

    The new field of uncertainty quantification (UQ) aims to give mathematical grounds for confidence in simulation results. It is a response to the complicated nature of justifying the use of simulation results to draw conclusions. UQ presupposes verification and informs validation.

    Verification deals with the errors that occur when converting a continuous mathematical model into a discrete one, and then to a computer code. There are known sources of errors —  truncation, round-off, partial iterative convergence  —  and unknown sources of errors  — coding mistakes, instabilities.

    Uncertainties stem from input data, modeling errors, genuine physical uncertainties, random processes  —  UQ is thus associated with the validation of a model. It follows that sufficient verification should be done first, before attempting validation. But is this always done in practice, and what is meant by ‘sufficient’?

    Verification provides evidence that the solver is fit for purpose, but this is subject to interpretation: the idea of accuracy is linked to judgments.

    Many articles discussing reproducibility in computational science place emphasis on the importance of code and data availability. But making code and data open and publicly available is not enough. To provide evidence that results from simulation are reliable requires solid V&V expertise and practice, reproducible-science methods, and carefully reporting our uncertainties and judgments.

    Supercomputing research should be executed using reproducible practices, taking good care of documentation and reporting standards, including appropriate use of statistics, and providing any research objects needed to facilitate follow-on studies.

    Even if the specialized computing system used in the research is not available to peers, conducting the research as if it will be reproduced increases trust and helps justify the new claims to knowledge.

    Computational experiments often involve deploying precise and complex software stacks, with several layers of dependencies. Multiple details must be taken care of during compilation, setting up the computational environment, and choosing runtime options. Thus, making available the source code (with a detailed mathematical description of the algorithm) is a minimum pre-requisite for reproducibility: necessary, but not sufficient.

    We also require detailed description and/or provision of:

    Dependencies
    Environment
    Automated build process
    Running scripts
    Post-processing scripts
    Secondary data generating published figures

    Not only does this practice facilitate follow-on studies, removing roadblocks for building on our work, it also enables getting at the root of discrepancies if and when another researcher attempts a full replication of our study.

    How far are we from achieving this practice as standard? A recent study surveyed a sample (admittedly small) of papers submitted to a supercomputing conference: only 30% of the papers provide a link to the source code, only 40% mention the compilation process, and only 30% mention the steps taken to analyze the data.

    We have a long way to go.

    See the full article here .

    Please help promote STEM in your local schools.
    STEM Icon

    Stem Education Coalition

    Science Node is an international weekly online publication that covers distributed computing and the research it enables.

    “We report on all aspects of distributed computing technology, such as grids and clouds. We also regularly feature articles on distributed computing-enabled research in a large variety of disciplines, including physics, biology, sociology, earth sciences, archaeology, medicine, disaster management, crime, and art. (Note that we do not cover stories that are purely about commercial technology.)

    In its current incarnation, Science Node is also an online destination where you can host a profile and blog, and find and disseminate announcements and information about events, deadlines, and jobs. In the near future it will also be a place where you can network with colleagues.

    You can read Science Node via our homepage, RSS, or email. For the complete iSGTW experience, sign up for an account or log in with OpenID and manage your email subscription from your account preferences. If you do not wish to access the website’s features, you can just subscribe to the weekly email.”

     
  • richardmitnick 1:07 pm on March 16, 2016 Permalink | Reply
    Tags: , , Supercomputing,   

    From TACC: “Wrangler Special Report” 

    TACC bloc

    Texas Advanced Computing Center

    Dell Wrangler Supercomputer Speeds through Big Data
    Data-intensive supercomputer brings new users to high performance computing for science

    TACC Wrangler
    TACC Wrangler

    Handling big data can sometimes feel like driving on an unpaved road for researchers with a need for speed and supercomputers.

    “When you’re in the world of data, there are rocks and bumps in the way, and a lot of things that you have to take care of,” said Niall Gaffney, a former Hubble Space Telescope scientist who now heads the Data Intensive Computing group at the Texas Advanced Computing Center (TACC).

    Gaffney led the effort to bring online a new kind of supercomputer, called Wrangler. Like the old Western cowboys who tamed wild horses, Wrangler tames beasts of big data, such as computing problems that involve analyzing thousands of files that need to be quickly opened, examined and cross-correlated.

    Wrangler fills a gap in the supercomputing resources of XSEDE, the Extreme Science and Engineering Discovery Environment, supported by the National Science Foundation (NSF). XSEDE is a collection of advanced digital resources that scientists can easily use to share and analyze the massive datasets being produced in nearly every field of research today. In 2013, NSF awarded TACC and its academic partners Indiana University and the University of Chicago $11.2 million to build and operate Wrangler, a supercomputer to handle data-intensive high performance computing.

    Wrangler was designed to work closely with the Stampede supercomputer, the 10th most powerful in the world according to the bi-annual Top500 list, and the flagship of TACC at The University of Texas at Austin (UT Austin).

    Texas Stampede Dell Supercomputer
    Dell Stampede

    Stampede has computed over six million jobs for open science since it came online in 2013.

    “We kept a lot of what was good with systems like Stampede,” said Gaffney, “but added new things to it like a very large flash storage system, a very large distributed spinning disc storage system, and high speed network access. This allows people who have data problems that weren’t being fulfilled by systems like Stampede and Lonestar to be able to do those in ways that they never could before.”

    Gaffney made the analogy that supercomputers like Stampede are like racing sports cars, with fantastic compute engines optimized for going fast on smooth, well-defined race-tracks. Wrangler, on the other hand, is built like a rally car to go fast on unpaved, bumpy roads with muddy gravel.

    “If you take a Ferrari off-road you may want to change the way that the suspension is done,” Gaffney said. “You want to change the way that the entire car is put together, even though it uses the same components, to build something suitable for people who have a different job.”

    At the heart of Wrangler lie 600 terabytes of flash memory shared via PCI interconnect across Wrangler’s over 3,000 Haswell compute cores. “All parts of the system can access the same storage,” Gaffney said. “They can work in parallel together on the data that are stored inside this high-speed storage system to get larger results they couldn’t get otherwise.”

    This massive amount of flash storage comes from DSSD, a startup co-founded by Andy Bechtolsheim of Sun Microsystems fame and acquired in May of 2015 by EMC. Bechtolsheim’s influence at TACC goes back to the ‘Magnum’ Infiniband network switch he led design on for the now-decommissioned Ranger supercomputer, the predecessor to Stampede.

    What’s new is that DSSD took a shortcut between the CPU and the data. “The connection from the brain of the computer goes directly to the storage system. There’s no translation in between,” Gaffney said. “It actually allows people to compute directly with some of the fastest storage that you can get your hands on, with no bottlenecks in between.”

    Speeding up the gene analysis pipeline

    Gaffney recalled the hang-up scientists had with code called OrthoMCL, which combs through DNA sequences to find common genetic ancestry in seemingly unrelated species. The problem was that OrthoMCL let loose databases wild as a bucking bronco.

    “It generates a very large database and then runs computational programs outside and has to interact with this database,” said biologist Rebecca Young of the Department of Integrative Biology and the Center for Computational Biology and Bioinformatics at UT Austin. She added, “That’s not what Lonestar and Stampede and some of the other TACC resources were set up for.”

    U Texas Lonestar supercomputer
    Lonestar

    Young recounted how at first, using OrthoMCL with online resources, she was only able to pull out 350 comparable genes across 10 species. “When I run OrthoMCL on Wrangler, I’m able to get almost 2,000 genes that are comparable across the species,” Young said. “This is an enormous improvement from what is already available. What we’re looking to do with OrthoMCL is to allow us to make an increasing number of comparisons across species when we’re looking at these very divergent, these very ancient species separated by 450 million years of evolution.”

    “We were able to go through all of these work cases in anywhere between 15 minutes and 6 hours,” Gaffney said. “This is a game changer.”

    Gaffney added that getting results quickly lets scientists explore new and deeper questions by working with larger collections of data and driving previously unattainable discoveries.

    Tuning energy efficiency in buildings

    Computer scientist Joshua New with the Oak Ridge National Laboratory (ORNL) hopes to take advantage of Wrangler’s ability to tame big data. New is the principal investigator of the Autotune project, which creates a software version of a building and calibrates the model with over 3,000 different data inputs from sources like utility bills to generate useful information such as what an optimal energy-efficient retrofit might be.

    “Wrangler has enough horsepower that we can run some very large studies and get meaningful results in a single run,” New said. He currently uses the Titan supercomputer of ORNL to run 500,000 simulations and write 45 TB of data to disk in 68 minutes.

    ORNL Titan Supercomputer
    ORNL’s Cray Titan supercomputer

    He said he wants to scale out his parametric studies to simulate all 125.1 million buildings in the U.S.

    “I think that Wrangler fills a specific niche for us in that we’re turning our analysis into an end-to-end workflow, where we define what parameters we want to vary,” New said. “It creates the sampling matrix. It creates the input files. It does the computationally challenging task of running all the simulations in parallel. It creates the output. Then we run our artificial intelligence and statistic techniques to analyze that data on the back end. Doing that from beginning to end as a solid workflow on Wrangler is something that we’re very excited about.”

    When Gaffney talks about storage on Wrangler, he’s talking about is a lot of data storage — a 10 petabyte Lustre-based file system hosted at TACC and replicated at Indiana University. “We want to preserve data,” Gaffney said. “The system for Wrangler has been set up for making data a first-class citizen amongst what people do for research, allowing one to hold onto data and curate, share, and work with people with it. Those are the founding tenants of what we wanted to do with Wrangler.”

    Shedding light on dark energy

    “Data is really the biggest challenge with our project,” said UT Austin astronomer Steve Finkelstein. His NSF-funded project is called HETDEX, the Hobby-Eberly Telescope Dark Energy Experiment.

    Hobby-Eberly Telescope
    U Texas Hobby-Eberly Telescope

    It’s the largest survey of galaxies ever attempted. Scientists expect HETDEX to map over a million galaxies in three dimensions, in the process discovering thousands of new galaxies. The main goal is to study dark energy, a mysterious force pushing galaxies apart.

    “Every single night that we observe — and we plan to observe more or less every single night for at least three years — we’re going to make 200 GB of data,” Finkelstein said. It’ll measure the spectra of 34,000 points of skylight every six minutes.

    “On Wrangler is our pipeline,” Finkelstein said. “It’s going to live there. As the data comes in, it’s going to have a little routine that basically looks for new data, and as it comes in every six minutes or so it will process it. By the end of the night it will actually be able to take all the data together to find new galaxies.”

    Human origins buried in fossil data

    Another example of a new HPC user Wrangler enables is an NSF-funded science initiative called PaleoCore. It hopes to take advantage of Wrangler’s swiftness with databases to build a repository for scientists to dig through geospatially-aware data on all fossils related to human origins. This would combine older digital collections in formats like Excel worksheets and SQL databases with newer ways of gathering data such as real-time fossil GPS information collected from iPhones or iPads.

    “We’re looking at big opportunities in linked open data,” PaleoCore principal investigator Denne Reed said. Reed is an associate professor in the Department of Anthropology at UT Austin.

    Linked open data allows for queries to get meaning from the relationships of seemingly disparate pieces of data. “Wrangler is the type of platform that enables that,” Reed said. “It enables us to store large amounts of data, both in terms of photo imagery, satellite imagery and related things that go along with geospatial data. Then also, it allows us to start looking at ways to effectively link those data with other data repositories in real time.”

    Data analytics for science

    Wrangler’s shared memory supports data analytics on the Hadoop and Apache Spark frameworks. “Hadoop is a big buzzword in all of data science at this point,” Gaffney said. “We have all of that and are able to configure the system to be able to essentially be like the Google Search engines are today in data centers. The big difference is that we are servicing a few people at a time, as opposed to Google.”

    Users bring data in and out of Wrangler in one of the fastest ways possible. Wrangler connects to Internet2, an optical network which provides 100 gigabytes per second worth of throughput to most of the other academic institutions around the country.

    What’s more, TACC has tools and techniques to transfer their data in parallel. “It’s sort of like being at the supermarket,” explained Gaffney. “If there’s only one lane open, it is just as fast as one person checking you out. But if you go in and have 15 lanes open, you can spread that traffic across and get more people through in less time.”

    A new user community for supercomputers

    Biologists, astronomers, energy efficiency experts, and paleontologists are just a small slice of the new user community Wrangler aims to attract.

    Wrangler is also more web-enabled than typically found in high performance computing. A web portal allows users to manage the system and gives the ability to use web interfaces such as VNC, RStudio, and Jupyter Notebooks to support more desktop-like user interactions with the system.

    “We need these bigger systems for science,” Gaffney said. “We need more kinds of systems. And we need more kinds of users. That’s where we’re pushing towards with these sort of portals. This is going to be the new face, I believe, for many of these systems that we’re moving forward with now. Much more web-driven, much more graphical, much less command line driven. ”

    “The NSF shares with TACC great pride in Wrangler’s continuing delivery of world-leading technical throughput performance as an operational resource available to the open science community in specific characteristics most responsive to advance data-focused research,” said Robert Chadduck, the program officer overseeing the NSF award.

    Wrangler is primed to lead the way in computing the bumpy world of data-intensive science research. “There are some great systems and great researchers out there who are doing groundbreaking and very important work on data, to change the way we live and to change the world,” Gaffney said. “Wrangler is pushing forth on the sharing of these results, so that everybody can see what’s going on.”

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition

    The Texas Advanced Computing Center (TACC) designs and operates some of the world’s most powerful computing resources. The center’s mission is to enable discoveries that advance science and society through the application of advanced computing technologies.

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
Follow

Get every new post delivered to your Inbox.

Join 584 other followers

%d bloggers like this: