From ASCRDiscovery: Women in STEM -“Thinking networks” Mariam Kiran

ASCRDiscovery
Advancing Science Through Computing


ESnet map

December 2017

ESnet’s DOE early-career awardee works to overcome roadblocks in computational networks.

1
ESnet’s Mariam Kiran. Image courtesy of ESnet.

2
The Atlas detector at CERN, in Switzerland. Users of it and other linked research facilities stand to benefit from ESnet’s efforts to reduce bottlenecks in intercontinental scientific data flow. Image courtesy of ESnet.

CERN/ATLAS detector

Like other complex systems, computer networks can break down and suffer bottlenecks. Keeping such systems running requires algorithms that can identify problems and find solutions on the fly so information moves quickly and on time.

Mariam Kiran – a network engineer for the Energy Sciences Network (ESnet), a DOE Office of Science user facility managed by Lawrence Berkeley National Laboratory – is using an early-career research award from DOE’s Office of Science to develop methods combining machine-learning algorithms with parallel computing to optimize such networks.

Kiran’s interest in science and mathematics was fueled by Doctor Who and other popular television shows she watched in her native United Kingdom. At 15 she got her first taste of computer programming, through a school project, using the BASIC programming language to create an airline database system. “I added a lot of graphics so that if you entered a wrong password, two airplanes would come across (the screen) and crash,” she says. It felt great to use a computer to create something out of nothing.

Kiran’s economist father and botanist mother encouraged her interests and before long she was studying computer science at the University of Sheffield. Pop culture also influenced her interests there, at a time when many students dressed in long black coats like those seen in the blockbuster movie The Matrix. The core computer science concept from that film – using computer simulations to test complex theories – was appealing.

She started coding such simulations, but along the way discovered another interest: developing ways around computer science roadblocks in those experiments. With simulations “you have potentially too much data to be processed, so you need a very fast and good system on the back end to make sure that the simulation goes as fast as it can,” she says. That challenge got her interested in computing and network infrastructure such as high-performance computing systems and cloud computing. She wanted to understand the problems and find strategies that help software run correctly and smoothly.

Kiran’s interest led her to join the software engineering and testing group at the University of Sheffield, where she also completed her master’s degree and Ph.D. She was part of a team that assembled a simulation platform for coding interacting components of a complex system – or agent-based modeling, used widely in Europe to calculate problems in economics or biology. Each agent could represent a government, a person, an individual organism, or a cell. “You code everything up as an agent and then let them interact with other agents, randomly or by following certain rules, and see how the system reacts overall.”

In 2014, she joined the UK’s University of Bradford as an associate professor and taught software engineering and machine learning. However, her research interests in performance optimization of computing and networks led her to investigate new projects that examined similar problems in applications that run over distributed compute and network resources. As a result, in 2016 she joined ESnet, which supports international science research computing networks and has produced a variety of innovations such as TCP and high-speed connections.

With her early career grant, Kiran has five years of support to pursue software innovations that can manage the efficiency of today’s computer networks and take them to the next level. Machine learning algorithms – such as deep neural networks used for image recognition and analysis – can be exploited to understand user behavior and data-movement patterns across the network. A computer networks is a complex distributed system. How one heals itself or performs corrective measures at the edge while operating optimally overall is an interesting challenge to understand and solve, Kiran says.

Managing information across networks is like transporting cargo on a highway system, she says. “You’re moving data from one building to the next building, and you have to find the shortest possible route.” The fastest path might depend on the time of day and traffic patterns.

Some science applications, however, are deadline-driven and require data to arrive by certain times to succeed. Short routes might become overly congested, whereas slightly longer paths may be under-used.

In the end, it’s a dynamic, multi-objective problem – finding the best possible route for data, one that is fast and less congested.

“Throughout the day, the state of the network changes depending on the users and applications interacting on it,” Kiran notes. “Understanding these complex relationships is a challenge. I’m interested in seeing whether machine learning can help us understand these more and allow networks to automate corrective measures in near-real time to prevent outages and application failures.”

She’s now identifying main problems along autonomous networks and applying those lessons to analogous computational and network problems. For example, she’s examining how engineers deal with outage-triggering bottlenecks and how bandwidth is controlled across links. Being at ESnet, which has led networking research for years, provides immense experience and capabilities to learn and apply solutions to a high-speed network that is built to think, she says.

Better-functioning networks could speed computational research on a range of topics, including climate, weather and nuclear energy. High performance computing boosts these calculations by rapidly distributing them across multiple computers and processors, sometimes across the world. It also allows international scientists to collaborate quickly. Researchers at diverse locations from Berkeley Lab to Switzerland’s CERN to labs in South America can interact with data quickly and seamlessly and develop new theories and findings.

This type of science and the problems it can address can make a real impact, Kiran says. “That’s what excites me about research – that we can improve or provide solutions to real-world problems.”

Now in its eighth year, the DOE Office of Science’s Early Career Research Program for researchers in universities and DOE national laboratories supports the development of individual research programs of outstanding scientists early in their careers and stimulates research careers in the disciplines supported by the Office of Science. The Office of Science is the single largest supporter of basic research in the physical sciences in the United States and is working to address some of the most pressing challenges of our time. For more information, please visit htp://science.energy.gov.

See the full article here.

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

ASCRDiscovery is a publication of The U.S. Department of Energy

#a-dynamic-multi-objective-problem-finding-the-best-possible-route-for-data-one-that-is-fast-and-less-congested, #applied-research-technology, #ascrdiscovery, #basic-research, #early-careerresearch-award-from-does-office-of-science-to-develop-methods-combining-machine-learning-algorithms-with-parallel-computing-to-optimize-such-networks, #esnet, #mariam-kiran, #this-type-of-science-and-the-problems-it-can-address-can-make-a-real-impact, #women-in-stem

From ESnet: “The CONNECT interview: Joe Metzger”

ESnet map


ESnet

2017-12-18

Joe Metzger, Senior Network Engineer in Lawrence Berkeley National Laboratory’s Scientific Networking Division and member of the ESnet network engineering team, has spent the past year working at CERN. He recently chatted with GÉANT’s community blog CONNECT about his experiences supporting science at home and abroad.

The CONNECT interview: Joe Metzger

1

Read the entire interview

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

#accelerator-science, #basic-research, #esnet, #the-connect-interview-joe-metzger

From ESnet: “ESnet’s Petascale DTN Project Speeds up Data Transfers between Leading HPC Centers”

ESnet map

ESnet

2017-12-11

1
Operations staff monitor the network in the ESnet/NERSC control room. (Photo by Marilyn Chung, Berkeley Lab)

The Department of Energy’s (DOE) Office of Science operates three of the world’s leading supercomputing centers, where massive data sets are routinely imported, analyzed, used to create simulations and exported to other sites. Fortunately, DOE also runs a networking facility, ESnet (short for Energy Sciences Network), the world’s fastest network for science, which is managed by Lawrence Berkeley National Laboratory.

Over the past two years, ESnet engineers have been working with staff at DOE labs to fine tune the specially configured systems called data transfer nodes (DTNs) that move data in and out of the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory and the leadership computing facilities at Argonne National Laboratory in Illinois and Oak Ridge National Laboratory in Tennessee. All three of the computing centers and ESnet are DOE Office of Science User Facilities used by thousands of researchers across the country.

NERSC Cray XC40 Cori II supercomputer

LBL NERSC Cray XC30 Edison supercomputer


The Genepool system is a cluster dedicated to the DOE Joint Genome Institute’s computing needs. Denovo is a smaller test system for Genepool that is primarily used by NERSC staff to test new system configurations and software.

NERSC PDSF


PDSF is a networked distributed computing cluster designed primarily to meet the detector simulation and data analysis requirements of physics, astrophysics and nuclear science collaborations.

ANL ALCF Cetus IBM supercomputer

ANL ALCF Theta Cray supercomputer

ANL ALCF Cray Aurora supercomputer

ANL ALCF MIRA IBM Blue Gene Q supercomputer at the Argonne Leadership Computing Facility

Currently at ORNL OCLF
3
Titan Cray XK7 at OCLF

Soon to come at ORNL OCLF

ORNL IBM Summit supercomputer depiction

The collaboration, named the Petascale DTN project, also includes the National Center for Supercomputing Applications (NCSA) at the University of Illinois in Urbana-Champaign, a leading center funded by the National Science Foundation (NSF). Together, the collaboration aims to achieve regular disk-to-disk, end-to-end transfer rates of one petabyte per week between major facilities, which translates to achievable throughput rates of about 15 Gbps on real world science data sets.

4
Blue Waters IBM supercomputer at the University of Illinois in Urbana-Champaign,

5
Performance data from March 2016 showing transfer rates between facilities. (Image credit: Eli Dart, ESnet)

Research projects such as cosmology and climate have very large (multi-petabyte) datasets and scientists typically compute at multiple HPC centers, moving data between facilities in order to take full advantage of the computing and storage allocations available at different sites.

Since data transfers traverse multiple networks, the slowest link determines the overall speed. Tuning the data transfer nodes and the border router where a center’s internal network connects to ESnet can smooth out virtual speedbumps. Because transfers over the wide area network have high latency between sender and receiver, getting the highest speed requires careful configuration of all the devices along the data path, not just the core network.

In the past few weeks, the project has shown sustained data transfers at well over the target rate of 1 petabyte per week. The number of sites with this base capability is also expanding, with Brookhaven National Laboratory in New York now testing its transfer capabilities with encouraging results. Future plans including bringing the NSF-funded San Diego Supercomputer Center and other big data sites into the mix.

SDSC Triton HP supercomputer

SDSC Gordon-Simons supercomputer

SDSC Dell Comet supercomputer

“This increase in data transfer capability benefits projects across the DOE mission science portfolio” said Eli Dart, an ESnet network engineer and leader of the project. “HPC facilities are central to many collaborations, and they are becoming more important to more scientists as data rates and volumes increase. The ability to move data in and out of HPC facilities at scale is critical to the success of an ever-growing set of projects.”

When it comes to moving data, there are many factors to consider, including the number of transfer nodes and their speeds, their utilization, the file systems connected to these transfer nodes on both sides, and the network path between them, according to Daniel Pelfrey, a high performance computing network administrator at the Oak Ridge Leadership Computing Facility.

The actual improvements being made range from updating software on the DTNs to changing the configuration of existing DTNs to adding new nodes at the centers.

6
Performance measurements from November 2017 at the end of the Petascale DTN project. All of the sites met or exceed project goals. (Image Credit: Eli Dart, ESnet)

“Transfer node operating systems and applications need to be configured to allow for WAN transfer,” Pelfrey said. “The connection is only going to be as fast as the slowest point in the path allows. A heavily utilized server, or a misconfigured server, or a heavily utilized network, or heavily utilized file system can degrade the transfer and make it take much longer.”

At NERSC, the DTN project resulted in adding eight more nodes, tripling the number, in order achieve enough internal bandwidth to meet the project’s goals. “It’s a fairly complicated thing to do,” said Damian Hazen, head of NERSC’s Storage Systems Group. “It involves adding infrastructure and tuning as we connected our border routers to internal routers to the switches connected to the DTNs. Then we needed to install the software, get rid of some bugs and tune the entire system for optimal performance.”

The work spanned two months and involved NERSC’s Storage Systems, Networking, and Data and Analytics Services groups, as well as ESnet, all working together, Hazen said.

At the Argonne Leadership Computing Facility, the DTNs were already in place and with minor tuning, transfer speeds were increased to the 15 Gbps.

“One of our users, Katrin Heitmann, had a ton of cosmology data to move and she saw a tremendous benefit from the project,” said Bill Allcock, who was director of operations at the ALCF during the project. “The project improved the overall end-to-end transfer rates, which is especially important for our users who are either moving their data to a community archive outside the center or are using data archived elsewhere and need to pull it in to compute with it at the ALCF.”

As a result of the Petascale DTN project, the OLCF now has 28 transfer nodes in production on 40-Gigabit Ethernet. The nodes are deployed under a new model—a diskless boot—which makes it easy for OLCF staff to move resources around, reallocating as needed to respond to users’ needs.

“The Petascale DTN project basically helped us increase the ‘horsepower under the hood’ of network services we provide and make them more resilient,” said Jason Anderson, an HPC UNIX/storage systems administrator at OLCF. “For example, we recently moved 12TB of science data from OLCF to NCSA in less than 30 minutes. That’s fast!”

Anderson recalled that a user at the May 2017 OLCF user meeting said that she was very pleased with how quickly and easily she was able to move her data to take advantage of the breadth of the Department of Energy’s computing resources.

“When the initiative started we were in the process of implementing a Science DMZ and upgrading our network,” Pelfrey said. “At the time, we could move a petabyte internally in 6-18 hours, but moving a petabyte externally would have taken just a bit over a week. With our latest upgrades, we have the ability to move a petabyte externally in about 48 hours.”

The fourth site in the project is the NSF-funded NCSA in Illinois, where senior network engineer Matt Kollross said it’s important for NCSA, the only non DOE participant, to collaborate with other DOE HPC sites to develop common practices and speed up adoption of new technologies.

“The participation in this project helped confirm that the design and investments in network and storage that we made when building Blue Waters five years ago were solid investments and will help in the design of future systems here and at other centers,” Kollross said. “It’s important that real-world benchmarks which test many aspects of an HPC system, such as storage, file systems and networking, be considered in evaluating overall performance of an HPC compute system and help set reasonable expectations for scientists and researchers.”

Origins of the project

The project grew out of a Cross-Connects Workshop on “Improving Data Mobility & Management for International Cosmology,” held at Berkeley Lab in February 2015 and co-sponsored by ESnet and Internet2.

Salman Habib, who leads the Computational Cosmology Group at Argonne National Laboratory, gave a talk at the workshop, noting that large-scale simulations are critical for understanding observational data and that the size and scale of simulation datasets far exceed those of observational data. “To be able to observe accurately, we need to create accurate simulations,” he said.

During the workshop, Habib and other attendees spoke about the need to routinely move these large data sets between computing centers and agreed that it would be important to be able to move at least a terabyte a week. As the Argonne lead for DOE’s High Energy Physics Center for Computational Excellence project, Habib had been working with ESnet and other labs on data transfer issues.

To get the project moving, Katrin Heitmann, who works in cosmology at Argonne, created a data package of small and medium files totaling about 4.4 terabytes. The data would then be used to test network links between the leadership computing facilities at Argonne and Oak Ridge national labs, the National Energy Research Scientific Computing Center (NERSC) at Lawrence Berkeley National Laboratory, and the National Center for Supercomputing Applications (NCSA) at the University of Illinois in Urbana-Champaign, a leading center funded by the National Science Foundation.

“The idea was to use the data as a test, to send it over and over and over between the centers,” Habib said. “We wanted to establish a performance baseline, then see if we could improve the performance by eliminating any choke points.”

Habib admitted that moving a petabyte in a week would only use a fraction of ESnet’s total bandwidth, but the goal was to automate the transfers using Globus Online, a primary tool for researchers accessing high performance networks like ESnet for rapidly sharing data or to use remote computing facilities.

“For our research, it’s very important that we have the ability to transfer large amounts of data,” Habib said. “For example, we may run a simulation at one of the large DOE computing centers, but often where we run the simulation is not where we want to do the analysis. Each center has different capabilities and we have various accounts at the centers, so the data gets moved around to take advantage of this. It happens all the time.”

Although the project’s roots are in cosmology, the Petascale DTN project will help all DOE scientists who have a need to transfer data to, from, or between the DOE computing facilities to take advantage of rapidly advancing data analytics techniques. In addition, the increase in data transfer capability at the HPC facilities will improve the performance of data portals, such as the Research Data Archive at the National Center for Atmospheric Research, that use Globus to transfer data from their storage systems.

“As the scientists deal with data deluge and more research disciplines depend on high-performance computing, data movement between computing centers needs to be a no-brainer for scientists so they can take advantage of the compute cycles at all DOE Office of Science user facilities and the extreme heterogeneity of systems in the future” said ESnet Director Inder Monga.

This work was supported by the HEP Center for Computational Excellence. ESnet is funded by DOE’s Office of Science.

Not included in this Center:

Ohio Super Computer Center

Ohio Oakley HP supercommputer

Ohio Ruby HP supercomputer

Ohio Dell Owens supercompter

TACC Maverick HP NVIDIA supercomputer

TACC Lonestar Cray XC40 supercomputer

Dell Poweredge U Texas Austin Stampede Supercomputer. Texas Advanced Computer Center 9.6 PF

TACC HPE Apollo 8000 Hikari supercomputer

TACC Maverick HP NVIDIA supercomputer

TACC DELL EMC Stampede2 supercomputer


See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

#esnet, #hpc-centers, #supercomputing

From ESnet: “SLAC, AIC and Zetta Move Petabyte Datasets at Unprecedented Speed via ESnet”

1

ESnet

2017-07-26
ESNETWORK

Twice a year, ESnet staff meet with managers and researchers associated with each of the DOE Office of Science program offices to look toward the future of networking requirements and then take the planning steps to keep networking capabilities out in front of those demands.

Network engineers and researchers at DOE national labs take a similar forward-looking approach. Earlier this year, DOE’s SLAC National Accelerator Laboratory (SLAC) teamed up with AIC and Zettar and tapped into ESnet’s 100G backbone network to repeatedly transfer 1-petabyte files in 1.4 days over a 5,000-mile portion of ESnet’s production network. Even with the transfer bandwidth capped at 80Gbps, the milestone demo resulted in transfer rates five times faster than other technologies. The demo data accounted for a third of all ESnet traffic during the tests. Les Cottrel from SLAC presented the results at the ESnet Site Coordinators meeting (ESCC) held at Lawrence Berkeley National Laboratory in May 2017.

1
No image caption or credit

Read the AICCI/Zettar news release.

Read the story in insideHPC.

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

#d-o-e, #esnet

From ESnet: “North Atlantic Network Collaboration Building Foundation for Global Network Architecture” This is a Big Deal

1

ESnet

2017-04-19
No writer credit

1
ESnet’s four links comprising 340 Gbps connectivity to Europe represents 45 percent of the North Alantic Network Collaboration’s combined 740 Gbps of bandwidth.

Transatlantic R&E bandwidth now at record-breaking 740 Gbps – ESnet contributes 340 Gbps

Scientists around the world are increasingly collaborating to address global issues such as clean energy, medicine and protecting the environment. Their ability to share and analyse data is essential for advancing research, and as the size of those datasets grows, the need for high-speed global network connectivity becomes ever more critical.

Collaborating Across the Atlantic

That is why research and education (R&E) networks in Europe and North America have joined forces to find new ways to help facilitate and enable scientific collaboration. Between them, the R&E networks on the two continents have now deployed links providing a total bandwidth of 740 gigabits per second (Gbps).

This record-breaking connectivity and resilience is the work of the Advanced North Atlantic (ANA) Collaboration. Started in 2013, ANA consists of six leading R&E networks: CANARIE (Canada), ESnet (USA), GÉANT (Europe), Internet2 (USA), NORDUnet (European Nordics), and SURFnet (The Netherlands).

“We’ve seen a tremendous growth in transatlantic connectivity since we have set up the first 100 Gbps R&E transatlantic link at TNC 2013,” said Erwin Bleumink, CEO of SURFnet. ”I am very pleased with the success of this international collaboration, in which SURFnet has been involved from the beginning.”

“Collaborations between research and education networks are unique and enable us as a community to address the exponentially growing data needs of science collaborations worldwide,” said Inder Monga, director of the US Department of Energy’s ESnet, which deployed four trans-Atlantic links comprising 340 Gbps in December 2014. “The combined capability offered to the research and education community far exceeds what any single organization can provide and moves us many steps forward towards accomplishing our vision of ‘scientific progress being completely unconstrained’.”

Adding Robust Resiliency to Keep Research Data Moving

Over the past three years, this partnership has continued to add bandwidth and resilience across the North Atlantic. The most recent addition by the NEAAR Project, coordinated by Indiana University (USA) and funded by the US National Science Foundation (NSF), adds a 100 Gbps connection between exchange points in New York City and London.

“While NEAAR brings an additional 100 Gbit/s to the mix, we are excited to not only add to the total bandwidth but also to the increased resilience, as the NEAAR 100 Gbit/s is implemented on a cable system that was not in use yet by the R&E networks,” said Jennifer Schopf, the NEAAR Project PI at Indiana University. “We are delighted to be part of this North Atlantic collaboration, where trust and sharing of resources is key, and will now be able to put more emphasis on reaching out to African R&E networks, as part of the mission of NEAAR.”

With the addition of NEAAR’s 100 Gbps link, the total amount of general purpose R&E connectivity across the North Atlantic Ocean now is at 400 Gbps. Additionally, ESnet operates 340 Gbps of bandwidth via four distinct circuits across the North Atlantic Ocean. To provide robust resilience to science, research and education traffic, the partners also act as each other’s back-ups in case of major outages or lengthy fibre cuts, which are two common concerns with managing transoceanic high-speed cable systems.

Securing future connectivity—The Global Network Architecture

While the transatlantic high-speed links develop and expand, a group of network specialists from R&E networking organizations and Exchange Points operators from around the world are also collaborating to ensure researchers see the end-to-end performance results their science requires not just locally but globally.

To achieve this, senior R&E network architects from around the world, including organizations involved in the North Atlantic collaboration, have developed a set of global principles and technical guidelines for collaboration, as well as sharing costs and aligning investments.

This work—under the umbrella of the Global Network Architecture (GNA) initiative—is defining a reference architecture and creating a roadmap for both national and regional research & education networks to more seamlessly support research on an end-to-end basis. Ultimately, this effort will establish a more capable, predictable and resilient next-generation global interconnect for research and education.

ANA declares itself GNA compliant

In January 2017, the first public version of the GNA Reference Architecture was released. The ANA Collaboration is very excited to announce its compliance with this architecture.

“We are pleased to see that the first piece of intercontinental R&E network infrastructure has been assessed as compliant with the GNA Reference Architecture,” said NORDUnet CEO René Buch. “We are confident that organisations and collaborations elsewhere soon will be able to declare their GNA compliancy. This will enable us to further develop a GNA compliant global research and education interconnect.”

Dave Lambert, President and CEO, Internet2: “The vision to establish a more robust set of architecture principles upon which advanced international research networks could be built was launched a couple of years ago, and I am happy to see that we have agreed to initial standards and moved to implementation so quickly. What this creates is a much more solid foundation upon which global research and science projects can advance their outcomes.”

Steve Cotter, GÉANT CEO: “This work is a critical first step in showing how diverse networking organisations supported by multiple government funding bodies can come together in a collaborative way to support those tackling society’s challenges. Together, we are testing new operational models and finding cost-efficiencies that can become the model for a global research and education infrastructure.”

Jim Ghadbane, President and CEO, CANARIE: “We’re very happy to work with our global partners to design and deliver the infrastructure that researchers need to advance science in their communities and around the world. The work of the ANA is a great example of how global collaboration of resources and expertise can serve the needs of researchers at the local level.”

View a timeline of the planned growth of the Global Network Architecture.

For more information on the Global Network Architecture initiative: https://gna-re.net/

For an in-depth look into the world of R&E networks, please go the international blog In The Field, showcasing stories from around the world about people and projects making a difference and connected by research and education networks.

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

#advanced-north-atlantic-ana-collaboration, #anarie-canada, #cross-atlantic-bandwidth, #esnet, #esnet-usa, #geant-europe, #internet2-usa, #nordunet-european-nordics, #north-atlantic-network-collaboration-building-foundation-for-global-network-architecture, #surfnet-the-netherlands, #transatlantic-re-bandwidth-now-at-record-breaking-740-gbps-esnet-contributes-340-gbps

From SLAC: SLAC, Berkeley Lab Researchers Prepare for Scientific Computing on the Exascale”


SLAC Lab

November 3, 2016

1
NERSC CRAY Cori supercomputer
Development and testing of future exascale computing tools for X-ray laser data analysis and the simulation of plasma wakefield accelerators will be done on the Cori supercomputer at NERSC, the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory. (NERSC)

Researchers at the Department of Energy’s SLAC National Accelerator Laboratory are playing key roles in two recently funded computing projects with the goal of developing cutting-edge scientific applications for future exascale supercomputers that can perform at least a billion billion computing operations per second – 50 to 100 times more than the most powerful supercomputers in the world today.

The first project, led by SLAC, will develop computational tools to quickly sift through enormous piles of data produced by powerful X-ray lasers. The second project, led by DOE’s Lawrence Berkeley National Laboratory (Berkeley Lab), will reengineer simulation software for a potentially transformational new particle accelerator technology, called plasma wakefield acceleration.

The projects, which will each receive $10 million over four years, are among 15 fully-funded application development proposals and seven proposals selected for seed funding by the DOE’s Exascale Computing Project (ECP). The ECP is part of President Obama’s National Strategic Computing Initiative and intends to maximize the benefits of high-performance computing for U.S. economic competiveness, national security and scientific discovery.

“Many of our modern experiments generate enormous quantities of data,” says Alex Aiken, professor of computer science at Stanford University and director of the newly formed SLAC Computer Science division, who is involved in the X-ray laser project. “Exascale computing will create the capabilities to handle unprecedented data volumes and, at the same time, will allow us to solve new, more complex simulation problems.”

Analyzing ‘Big Data’ from X-ray Lasers in Real Time

X-ray lasers, such as SLAC’s Linac Coherent Light Source (LCLS) have been proven to be extremely powerful “microscopes” that are capable of glimpsing some of nature’s fastest and most fundamental processes on the atomic level.

SLAC/LCLS
SLAC/LCLS

Researchers use LCLS, a DOE Office of Science User Facility, to create molecular movies, watch chemical bonds form and break, follow the path of electrons in materials and take 3-D snapshots of biological molecules that support the development of new drugs.

At the same time X-ray lasers also generate giant amounts of data. A typical experiment at LCLS, which fires 120 flashes per second, fills up hundreds of thousands of gigabytes of disk space. Analyzing such a data volume in a short amount of time is already very challenging. And this situation is set to become dramatically harder: The next-generation LCLS-II X-ray laser will deliver 8,000 times more X-ray pulses per second, resulting in a similar increase in data volumes and data rates.

SLAC/LCLS II schematic
SLAC/LCLS II schematic

Estimates are that the data flow will greatly exceed a trillion data ‘bits’ per second, and require hundreds of petabytes of online disk storage.

As a result of the data flood even at today’s levels, researchers collecting data at X-ray lasers such as LCLS presently receive only very limited feedback regarding the quality of their data.

“This is a real problem because you might only find out days or weeks after your experiment that you should have made certain changes,” says Berkeley Lab’s Peter Zwart, one of the collaborators on the exascale project, who will develop computer algorithms for X-ray imaging of single particles. “If we were able to look at our data on the fly, we could often do much better experiments.”

Amedeo Perazzo, director of the LCLS Controls & Data Systems Division and principal investigator for this “ExaFEL” project, says, “We want to provide our users at LCLS, and in the future LCLS-II, with very fast feedback on their data so that can make important experimental decisions in almost real time. The idea is to send the data from LCLS via DOE’s broadband science network ESnet to NERSC, the National Energy Research Scientific Computing Center, where supercomputers will analyze the data and send the results back to us – all of that within just a few minutes.” NERSC and ESnet are DOE Office of Science User Facilities at Berkeley Lab.

LBL NERSC Cray XC30 Edison supercomputer
LBL NERSC Cray XC30 Edison supercomputer

lcls-ii-image
LCLS II

X-ray data processing and analysis is quite an unusual task for supercomputers. “Traditionally these high-performance machines have mostly been used for complex simulations, such as climate modeling, rather than processing real-time data” Perazzo says. “So we’re breaking completely new ground with our project, and foresee a number of important future applications of the data processing techniques being developed.”

This project is enabled by the investments underway at SLAC to prepare for LCLS-II, with the installation of new infrastructure capable of handling these enormous amounts of data.

A number of partners will make additional crucial contributions.

“At Berkeley Lab, we’ll be heavily involved in developing algorithms for specific use cases,” says James Sethian, a professor of mathematics at the University of California, Berkeley, and head of Berkeley Lab’s Mathematics Group and the Center for Advanced Mathematics for Energy Research Applications (CAMERA). “This includes work on two different sets of algorithms. The first set, developed by a team led by Nick Sauter, consists of well-established analysis programs that we’ll reconfigure for exascale computer architectures, whose larger computer power will allow us to do better, more complex physics. The other set is brand new software for emerging technologies such as single-particle imaging, which is being designed to allow scientists to study the atomic structure of single bacteria or viruses in their living state.”

The “ExaFEL” project led by Perazzo will take advantage of Aiken’s newly formed Stanford/SLAC team, and will collaborate with researchers at Los Alamos National Laboratory to develop systems software that operates in a manner that optimizes its use of the architecture of the new exascale computers.

“Supercomputers are very complicated, with millions of processors running in parallel,” Aiken says. “It’s a real computer science challenge to figure out how to use these new architectures most efficiently.”

Finally, ESnet will provide the necessary networking capabilities to transfer data between the LCLS and supercomputing resources. Until exascale systems become available in the mid-2020s, the project will use NERSC’s Cori supercomputer for its developments and tests.

esnet-map
ESnet

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

SLAC Campus
SLAC is a multi-program laboratory exploring frontier questions in photon science, astrophysics, particle physics and accelerator research. Located in Menlo Park, California, SLAC is operated by Stanford University for the DOE’s Office of Science.
i1

#applied-research-technology, #basic-research, #esnet, #slac-lcls, #slac-lcls-ii, #x-ray-technology

From esnet: “National Science Foundation & Department of Energy’s ESnet Launch Innovative Program for Women Engineers”

1

ESnet

2016-10-26

Women in Networking @SC (WINS) Kicks off this week in Salt Lake City!

1
(Left to Right) Julia Locke (LANL), Debbie Fligor (SC15 WINS returning participant, University of Illinois at Urbana-Champaign), Jessica Schaffer (Georgia Tech), Indira Kassymkhanova (LBNL), Denise Grayson (Sandia), Kali McLennan (Univ. of Oklahoma), Angie Asmus (CSU). Not in photo: Amber Rasche (N. Dakota State) and Julie Staats (CENIC).

The University of Corporation for Atmospheric Research (UCAR) and The Keystone Initiative for Network Based Education and Research (KINBER) together with the Department of Energy’s (DOE) Energy Science Network (ESnet) today announce the official launch of an Networking at SC (WINS) program.

Funded through a grant from the National Science Foundation (NSF) and directly from ESnet, the program funds eight early to mid-career women in the research and education (R&E) network community to participate in the 2016 setup, build out and live operation of SCinet, the Supercomputing Conference’s (SC) ultra high performance network. SCinet supports large-scale computing demonstrations at SC, the premier international conference on high performance computing, networking, data storage and data analysis and is attended by over 10,000 of the leading minds in these fields.

The SC16 WINS program kicked off this week as the selected participants from across the U.S., headed to Salt Lake City, the site of the 2016 conference to begin laying the groundwork for SCinet inside the Salt Palace Convention Center. The WINS participants join over 250 volunteers that make up the SCinet engineering team and will work side by side with the team and their mentors to put the network into full production service when the conference begins on November 12. The women will return to Salt Lake City a week before the conference to complete the installation of the network.

“We are estimating that SCinet will be outfitted with a massive 3.5 Terabits per second (Tbps) of bandwidth for the conference and will be built from the ground up with leading edge network equipment and services (even pre-commercial in some instances) and will be considered the fastest network in the world during its operation,” said Corby Schmitz, SC16 SCinet Chair.”

The WINS participants will support a wide range of technical areas that comprise SCinet’s incredible operation, including wide area networking, network security, wireless networking, routing, network architecture and other specialties.

2
Several WINS participants hard at work with their mentors configuring routers & switches

“While demand for jobs in IT continues to increase, the number of women joining the IT workforce has been on the decline for many years,” said Marla Meehl, Network Director from UCAR and co-PI of the NSF grant. “WINS aims to help close this gap and help to build and diversify the IT workforce giving women professionals a truly unique opportunity to gain hands-on expertise in a variety of networking roles while also developing mentoring relationships with recognized technical leaders.”

Funds are being provided by the NSF through a $135,000 grant and via direct funding from ESnet supported by Advanced Scientific Computing Research (ASCR) in DOE Office of Science. Funding covers all travel expenses related to participating in the setup and operation of SCinet and will also provide travel funds for the participants to share their experiences at events like The Quilt Member Meetings, Regional Networking Member meetings, and the DOE National Lab Information Technology Annual Meeting.

“Not only is WINS providing hands-on engineering training to the participants but also the opportunity to present their experiences with the broader networking community throughout the year. This experience helps to expand important leadership and presentations skills and grow their professional connections with peers and executives alike,” said Wendy Huntoon, president and CEO of KINBER and co-PI of the NSF grant.”

The program also represents a unique cross-agency collaboration between the NSF and DOE. Both agencies recognize that the pursuit of knowledge and science discovery that these funding organizations support depends on bringing the best ideas from people of various backgrounds to the table.

“Bringing together diverse voices and perspectives to any team in any field has been proven to lead to more creative solutions to achieve a common goal,” says Lauren Rotman, Science Engagement Group Lead, ESnet. “It is vital to our future that we bring every expert voice, every new idea to bear if our community is to tackle some of our society’s grandest challenges from understanding climate change to revolutionizing cancer treatment.”

2016 WINS Participants are:

Denise Grayson, Sandia National Labs (Network Security Team), DOE-funded
Julia Locke, Los Alamos National Lab (Fiber and Edge Network Teams), DOE-funded
Angie Asmus, Colorado State (Edge Network Team), NSF-funded
Kali McLennan, University of Oklahoma (WAN Transport Team), NSF-funded
Amber Rasche, North Dakota State University (Communications Team), NSF-funded
Jessica Shaffer, Georgia Institute of Tech (Routing Team), NSF-funded
Julia Staats, CENIC (DevOps Team), NSF-funded
Indira Kassymkhanova, Lawrence Berkeley National Lab (DevOps and Routing Teams), DOE-funded

The WINS Supporting Organizations:
The University Corporation for Atmospheric Research (UCAR)
http://www2.ucar.edu/

The Keystone Initiative for Network Based Education and Research (KINBER)
http:www.kinber.org

THe Department of Energy’s Energy Sciences Network (ESnet)
http://www.es.net

See the full article here .

Please help promote STEM in your local schools.

STEM Icon

Stem Education Coalition

Created in 1986, the U.S. Department of Energy’s (DOE’s) Energy Sciences Network (ESnet) is a high-performance network built to support unclassified science research. ESnet connects more than 40 DOE research sites—including the entire National Laboratory system, supercomputing facilities and major scientific instruments—as well as hundreds of other science networks around the world and the Internet.

#applied-research-technology, #esnet, #nsf, #women-in-stem