Tagged: Fermilab Toggle Comment Threads | Keyboard Shortcuts

  • richardmitnick 7:46 pm on May 29, 2018 Permalink | Reply
    Tags: Adaptive Input/Output System (ADIOS) and the BigData Express (BDE), Fermilab, ITER Tokamak in Saint-Paul-lès-Durance which is in southern France, ,   

    From Fermilab and OLCF: “ADIOS and BigData Express offer new data streaming capabilities” 

    FNAL II photo

    FNAL Art Image
    FNAL Art Image by Angela Gonzales

    From Fermilab is an enduring source of strength for the US contribution to scientific research world wide.

    1

    Projects large enough to run on high-performance computing (HPC) resources pack data—and a lot of it. Transferring this data between computational and experimental facilities is a challenging but necessary part of projects that rely on experiments to validate computational models.

    Staff at two U.S. Department of Energy (DOE) Office of Science User Facilities — the Oak Ridge Leadership Computing Facility (OLCF) and Fermi National Accelerator Laboratory — facilitated this process by executing the integration of the Adaptive Input/Output System (ADIOS) and the BigData Express (BDE) high-speed data transfer service.

    Now ADIOS and BDE developers are changing the way researchers can transport and analyze data by incorporating a new methodology into the tool that allows for compressing and streaming of data coming out of simulations in real time. The methodology is being tested by OLCF user C. S. Chang, a plasma physics researcher at Princeton Plasma Physics Laboratory (PPPL) who studies the properties of the plasmas that exist in giant fusion devices called tokamaks.

    PPPL NSTX -U at Princeton Plasma Physics Lab, Princeton, NJ,USA

    Chang seeks an understanding of the power needed to run ITER and the heat load to the material wall that will surround its plasma, both of which are key to fusion’s viability.

    ITER Tokamak in Saint-Paul-lès-Durance, which is in southern France

    ITER is an international collaboration working to design, construct, and assemble a burning plasma experiment that can demonstrate the scientific and technological feasibility of fusion power for the commercial power grid. ITER, which counts DOE’s Oak Ridge National Laboratory (ORNL) among its partners, is currently under construction in southern France.

    “If users can separate out the most important pieces of data and move those to another processor that can recognize the intended prioritization and reduce the data, it can provide them with feedback that they may need to stop a simulation if necessary,” said Scott Klasky, leader of the ADIOS framework and group leader for ORNL’s Scientific Data Group.

    Wenji Wu, principal investigator of the BDE project and principal network research investigator of Fermilab’s Core Computing Division, added, “The new approach leverages the software-defining network [SDN] capabilities for resource scheduling and the high-performance data streaming capabilities of BDE.”

    SDN allows users to dynamically control network resources rather than manually request to connect.

    “This combination enables real-time data streaming with guaranteed quality of service, whether it be over short or long distances,” Wu said. “In addition, this approach yields small memory footprints.”

    Although the project is still in the development phase, preliminary tests allowed Chang and his team to successfully transfer fusion data between the OLCF — located at ORNL — and PPPL.

    “With this new methodology, users can stream data on the network without ever touching the file system and request network resources on the fly,” said ADIOS and BDE researcher Qing Liu, who has a joint appointment with the New Jersey Institute of Technology and ORNL.

    Without streaming capabilities, scientists can perform only after-the-fact analyses for many experiments, such as KSTAR, the Korean Superconducting Tokamak Advanced Research.

    KSTAR Korean Superconducting Tokamak Advanced Research

    But with simulations and experiments increasing in size, near–real-time monitoring and control are becoming necessary. The new ADIOS–BDE integration could also play a major role in large experimental projects, such as the fusion project Chang is leading and the Square Kilometer Array, an effort involving dozens of institutions to build the world’s largest radio telescope.

    SKA Square Kilometer Array

    The new streaming capabilities could more easily enable the capture of short-lived events such as pulsars — neutron stars that emit electromagnetic radiation — that the telescope aims to record.

    “KSTAR wants to transfer their data as the experiment is happening, to process their data during the experiment,” Klasky said. “These additions to ADIOS will enable both sides to quickly perform data analysis and visualization in real time.”

    Seo-Young Noh, director of the Global Science Experimental Data Hub Center at the Korea Institute of Science and Technology Information, leads a group that has contributed significantly to the BDE project.

    “Our work has made cross-Pacific, real-time data streaming possible,” Noh said.

    Klasky, Liu, and their collaborators will give a best paper plenary talk related to these new capabilities—titled “Understanding and Modeling Lossy Compression Schemes on HPC Scientific Data” — at the 32nd IEEE International Parallel and Distributed Processing Symposium. The team noted that the new ADIOS methodology will allow scientists to efficiently select the type of compression that will best fit their scientific and research needs, affording them the ability to analyze their data faster than ever before.

    Liang Zhang, the developer of BDE data streaming capabilities, is working with Liu to enhance and test the tool. They expect the tool’s new capabilities to be fully tested and deployed by late 2019. This work also involves ADIOS researcher Jason Wang and BDE researchers Nageswara Rao, Phil DeMar, Qiming Lu, Sajith Sasidharan, S. A. R. Shah, Jin Kim, and Huizhang Luo.

    ORNL is managed by UT-Battelle for the Department of Energy’s Office of Science, the single largest supporter of basic research in the physical sciences in the United States. DOE’s Office of Science is working to address some of the most pressing challenges of our time. For more information, please visit science.energy.gov.

    See the full article here .


    five-ways-keep-your-child-safe-school-shootings

    stem

    Stem Education Coalition

    FNAL Icon

    Fermi National Accelerator Laboratory (Fermilab), located just outside Batavia, Illinois, near Chicago, is a US Department of Energy national laboratory specializing in high-energy particle physics. Fermilab is America’s premier laboratory for particle physics and accelerator research, funded by the U.S. Department of Energy. Thousands of scientists from universities and laboratories around the world
    collaborate at Fermilab on experiments at the frontiers of discovery.


    FNAL/MINERvA

    FNAL DAMIC

    FNAL Muon g-2 studio

    FNAL Short-Baseline Near Detector under construction

    FNAL Mu2e solenoid

    Dark Energy Camera [DECam], built at FNAL

    FNAL DUNE Argon tank at SURF

    FNAL/MicrobooNE

    FNAL Don Lincoln

    FNAL/MINOS

    FNAL Cryomodule Testing Facility

    FNAL Minos Far Detector

    FNAL LBNF/DUNE from FNAL to SURF, Lead, South Dakota, USA

    FNAL/NOvA experiment map

    FNAL NOvA Near Detector

    FNAL ICARUS

    FNAL Holometer

     
  • richardmitnick 3:18 pm on April 29, 2018 Permalink | Reply
    Tags: , Fermilab, Length contraction: the real explanation,   

    From Don Lincoln at Fermilab: “Length contraction: the real explanation” Video 

    FNAL II photo

    FNAL Art Image
    FNAL Art Image by Angela Gonzales

    Fermilab is an enduring source of strength for the US contribution to scientific research world wide.

    FNAL’s Don Lincoln

    Relativity has many mind-bending consequences, but one of the weirdest is the idea that objects in motion get shorter. Bizarre or not, Fermilab’s Dr. Don Lincoln explains just how it works. You’ll be a believer.

    See the full article here .

    Please help promote STEM in your local schools.

    STEM Icon

    Stem Education Coalition

    FNAL Icon

    Fermi National Accelerator Laboratory (Fermilab), located just outside Batavia, Illinois, near Chicago, is a US Department of Energy national laboratory specializing in high-energy particle physics. Fermilab is America’s premier laboratory for particle physics and accelerator research, funded by the U.S. Department of Energy. Thousands of scientists from universities and laboratories around the world
    collaborate at Fermilab on experiments at the frontiers of discovery.


    FNAL/MINERvA

    FNAL DAMIC

    FNAL Muon g-2 studio

    FNAL Short-Baseline Near Detector under construction

    FNAL Mu2e solenoid

    Dark Energy Camera [DECam], built at FNAL

    FNAL DUNE Argon tank at SURF

    FNAL/MicrobooNE

    FNAL Don Lincoln

    FNAL/MINOS

    FNAL Cryomodule Testing Facility

    FNAL Minos Far Detector

    FNAL LBNF/DUNE from FNAL to SURF, Lead, South Dakota, USA

    FNAL/NOvA experiment map

    FNAL NOvA Near Detector

    FNAL ICARUS

    FNAL Holometer

     
c
Compose new post
j
Next post/Next comment
k
Previous post/Previous comment
r
Reply
e
Edit
o
Show/Hide comments
t
Go to top
l
Go to login
h
Show/Hide help
shift + esc
Cancel
%d bloggers like this: