From Berkeley Lab Computing Sciences
At
The DOE’s Lawrence Berkeley National Laboratory
5.22.24
Carol Pott
New AI Method Aids Complex Data Interpretation
PLOS
Within the intricate workings of the brain, groups of neurons coordinate, enabling our ability to hear sounds, perform intricate movements of our limbs, and delve into complex thoughts and plans. Understanding if and how things in the outside world, like the pitch of a melody or the angle of an arm movement, influence the collective actions of these neuron clusters is fundamental to understanding how the brain works and unraveling its mysteries. Complex brain activity can be simplified into discernible patterns that showcase how groups of neurons are activating over time.
However, current analysis techniques often fail to reveal the direct connections between these patterns and external factors like sounds or movements. This failure has sparked a lively debate about our perception and comprehension of the brain: is it a dynamical system orchestrating perception, cognition, and action, or a sophisticated computing engine directly manipulating sensory, cognitive, and behavioral data?
New research published in PLOS Computational Biology by a team in Lawrence Berkeley National Laboratory’s (Berkeley Lab’s) Scientific Data (SciData) Division and the Biological Systems and Engineering Division reveals a fascinating truth: the brain operates as both. The brain orchestrates perception and action through the synchronized dynamics of populations of neurons, each meticulously organized to mirror the world around us. This detailed organization enables sophisticated computations on these representations, supporting advanced mental functions related to perception and action.
The team achieved this finding by developing a novel, interpretable, unsupervised AI model and applying it to brain activity data. Called the orthogonal stochastic linear mixing model (OSLMM), a type of Gaussian process, the tool allows researchers to understand how groups of neurons work together over time. A Gaussian process is a powerful tool that models intricate observations by breaking them down into simpler functions. It not only measures the uncertainty surrounding these functions but also empowers predictions and inference, making it a versatile asset in data analysis. While Gaussian processes are commonly used for prediction and uncertainty quantification, the team used OSLMM to inspect those simple functions to gain insight into the unknown processes that generate the observed brain data. OSLMM is a Bayesian regression framework and utilizes Markov chain Monte Carlo inference procedures, which are statistical techniques used to estimate model parameters.
The team applied OSLMM on real brain data from animals: the brain activity of rats listening to pure tones (collected in the Neural Systems and Data Science Lab at Berkeley Lab) and previously collected data from monkeys performing complex arm movements (from Sabes Lab, University of California, San Francisco). OSLMM was able to accurately predict how the neurons would act and how the animals would move. OSLMM also revealed that the underlying patterns of brain activity directly relate to the sounds and movements the animals were experiencing. Together, these results suggest that OSLMM will be useful for learning interpretable and predictive AI models for the analysis of complex time-series datasets.
“Understanding how things work in biology, like brain functions, is tough because often it is the coordination of many individual components (e.g., neurons, molecules, microbes, etc.) that produce an interesting phenomenon. There are a lot of data observations, and they are noisy. We are also unable to observe all relevant components simultaneously,” said Kris Bouchard, leader of SciData’s Computational Biosciences Group and PI of the Neural Systems and Data Science Lab in the Biological Systems and Engineering Division. “OSLMM discovers hidden patterns better and is more scalable than other methods. While we created OSLMM to address problems in neuroscience, the method is completely general and can be applied to other complex time-series data sets.”
Breaking Down the Methodologies of Research
There are many ways to understand complex time-series data. In neuroscience, one popular method is called Gaussian Process Factor Analysis (GPFA), which looks at how neurons work together over time. GPFA assumes that the ‘interactions’ between neurons stay the same over time, which isn’t always true in real brain data. Other methods, like neural networks, try to capture how the brain changes over time in more flexible ways and can maximize metrics like predictive accuracy but generally do not provide interpretable insights into the processes that generated the data. Interpretable insights are a critical need for scientific ML/AI that differs from the needs of industrial ML/AI.
Compared to GPFA, OSLMM showed better performance in recovering hidden patterns in experiments and predicting outcomes in real data. Importantly, OSLMM revealed meaningful patterns in brain recordings. For example, in rat brain recordings, OSLMM showed patterns related to sound properties, and in monkey brain recordings, it showed patterns linked to arm movements that were undetected by GPFA. These findings suggest that how the brain works may be more complex than we thought, and OSLMM helps us understand these processes better. Overall, OSLMM helped uncover hidden structures in brain data that provided insights into how the brain functions.
The average hidden brain activity patterns across trials for different sound frequencies, using OSLMM (top) and GPFA (bottom). The patterns discovered by OSLMM were directly organized by the sound frequency (blue-to-green-to-orange-to-red), while the patterns discovered by GPFA were not.
Simplifying Findings
OSLMM outperformed GPFA, showing an improved ability to capture meaningful patterns and predict outcomes related to external factors. OSLMM achieved these advancements through two mathematical innovations: a flexible model that allowed the interactions of neurons to change over time and imposed an orthogonality constraint on how underlying patterns relate. An orthogonality constraint refers to a restriction that ensures certain aspects or components are unrelated to each other. In this context, it means enforcing a condition where specific patterns or factors are kept separate or orthogonal from one another without influencing or overlapping with each other. The team demonstrated that this constraint significantly improves scalability, both theoretically and in practical tests across various real-world datasets. These broad modeling capabilities, along with the robust results in fields beyond neuroscience, indicate that OSLMM is a promising tool for understanding the complex systems that generate observed data.
“OSLMM has been shown to be effective for unsupervised analysis of complex data with many hidden patterns over time. It performs well in predicting outcomes, but more importantly, it revealed that the hidden patterns were organized around external factors,” said Bouchard. “This is helpful not just in neuroscience but also in other areas of biology, where we collect a lot of data from various sensors over time but are not able to measure all relevant information. The new mathematics underlying OSLMM’s ability to learn predictive and interpretable hidden patterns make it valuable for understanding complex processes and discovering new insights from large datasets.”
This research was funded by the Department of Energy Office of Science Advanced Scientific Computing Research (ASCR) AI-codesign for Complex Data program, the National Institute of Health (NINDS), and Berkeley Lab’s Laboratory Directed Research and Development.
See the full article here.
Comments are invited and will be appreciated, especially if the reader finds any errors which I can correct.
five-ways-keep-your-child-safe-school-shootings
Please help promote STEM in your local schools.
The Wang Hall computational theory and research facility houses Berkeley Lab’s Computing Sciences and serves as a hub for innovation and collaboration for scientific computing.
About Berkeley Lab Computing Sciences
Scientists today rely on advances in computer science, mathematics, and computational science, as well as high-performance computing and networking facilities, to increase our understanding of ourselves, our planet, and our universe.
The mission of Computing Sciences at Berkeley Lab is to achieve transformational, breakthrough impacts in scientific domains through the discovery and use of advanced computational methods and systems and to make those instruments accessible to the broad scientific community.
The Computing Sciences organization advances computational science throughout the U.S. Department of Energy’s Office of Science research programs. Our greatest strength is our experienced staff. They specialize in solving scientific problems using their wide-ranging expertise in computer hardware and software, computational science, applied mathematics, networking, and middleware services.
Computing Sciences is made up of four divisions: Applied Mathematics & Computational Research, Scientific Data, National Energy Research Scientific Computing (NERSC) Center, and the Scientific Networking Division, which manages the Energy Sciences Network (ESnet) and by conducting applied research and development in computer science, computational science, and mathematics, the three essential elements of computational modeling and simulation.
Combining computer and network operations with research and development provides tangible benefits to both the scientists who use our systems and their research programs. Just as our R&D programs benefit science, the needs of scientists influence the direction of our research. These two sides of Computing Sciences’ work — systems and services, and research and development — are more than complimentary. Each enhances the quality of the other.
The Computing Sciences organization was created in 1996 and currently has about 500 employees. Most of the computational research and resources provided by Computing Sciences are funded by the Advanced Scientific Computing Research (ASCR) program in the DOE Office of Science.
Computing Sciences co-sponsors Berkeley CSE, a multidisciplinary research and education program in computational science and engineering. We also sponsor the Luis W. Alvarez Postdoctoral Fellowship in Computational Science and the Admiral Grace Murray Hopper Postdoctoral Fellowship in Computing Sciences. We also participate in the DOE Computational Science Graduate Fellowship (CSGF) program. Our Summer Student Program gives university students an opportunity to gain relevant research experience while pursuing their degrees. Our Diversity Task Force works to recruit women and minorities and encourages K–12 students to study and pursue careers in math and science.
Bringing Science Solutions to the World
In the world of science, Lawrence Berkeley National Laboratory (Berkeley Lab) is synonymous with “excellence.” Nobel prizes are associated with Berkeley Lab. Seventy Lab scientists are members of the The National Academy of Sciences , one of the highest honors for a scientist in the United States. Thirteen of our scientists have won the National Medal of Science, our nation’s highest award for lifetime achievement in fields of scientific research. Eighteen of our engineers have been elected to the The National Academy of Engineering, and three of our scientists have been elected into the Institute of Medicine. In addition, Berkeley Lab has trained thousands of university science and engineering students who are advancing technological innovations across the nation and around the world.
Berkeley Lab is a member of the national laboratory system supported by the U.S. Department of Energy through its Office of Science. It is managed by the University of California and is charged with conducting unclassified research across a wide range of scientific disciplines. Located on a 202-acre site in the hills above the University of California- Berkeley campus that offers spectacular views of the San Francisco Bay, Berkeley Lab employs over 3,000 scientists, engineers and support staff. The Lab’s total annual costs are over $800 million. The Laboratory’s overall economic impact through direct, indirect and induced spending on the nine counties that make up the San Francisco Bay Area are over $900 million annually. The Lab is also responsible for creating 6,000 jobs locally and 14,000 nationally. The overall economic impact on the national economy is estimated at $2 billion a year. Technologies developed at Berkeley Lab have generated billions of dollars in revenues, and thousands of jobs. Savings as a result of Berkeley Lab developments in lighting and windows, and other energy-efficient technologies, have also been in the billions of dollars.
Berkeley Lab was founded in 1931 by Ernest Orlando Lawrence, a University of California-Berkeley physicist who won the 1939 Nobel Prize in physics for his invention of the cyclotron, a circular particle accelerator that opened the door to high-energy physics. It was Lawrence’s belief that scientific research is best done through teams of individuals with different fields of expertise, working together. His teamwork concept is a Berkeley Lab legacy that continues today.
History
1931–1941
The laboratory was founded on August 26, 1931, by Ernest Lawrence, as the Radiation Laboratory of the University of California, Berkeley, associated with the Physics Department. It centered physics research around his new instrument, the cyclotron, a type of particle accelerator for which he was awarded the Nobel Prize in Physics in 1939.
LBNL 88 inch cyclotron.
Throughout the 1930s, Lawrence pushed to create larger and larger machines for physics research, courting private philanthropists for funding. He was the first to develop a large team to build big projects to make discoveries in basic research. Eventually these machines grew too large to be held on the university grounds, and in 1940 the lab moved to its current site atop the hill above campus. Part of the team put together during this period includes two other young scientists who went on to establish large laboratories; J. Robert Oppenheimer founded the DOE’s Los Alamos Laboratory, and Robert Wilson founded the DOE’s Fermi National Accelerator Laboratory.
1942–1950
Leslie Groves visited Lawrence’s Radiation Laboratory in late 1942 as he was organizing the Manhattan Project, meeting J. Robert Oppenheimer for the first time. Oppenheimer was tasked with organizing the nuclear bomb development effort and founded today’s Los Alamos National Laboratory to help keep the work secret. At the RadLab, Lawrence and his colleagues developed the technique of electromagnetic enrichment of uranium using their experience with cyclotrons. The “calutrons” (named after the University) became the basic unit of the massive Y-12 facility in Oak Ridge, Tennessee. Lawrence’s lab helped contribute to what have been judged to be the three most valuable technology developments of the war (the atomic bomb, proximity fuse, and radar). The cyclotron, whose construction was stalled during the war, was finished in November 1946. The Manhattan Project shut down two months later.
1951–2018
After the war, the Radiation Laboratory became one of the first laboratories to be incorporated into the Atomic Energy Commission (AEC) (now Department of Energy. The most highly classified work remained at Los Alamos, but the RadLab remained involved. Edward Teller suggested setting up a second lab similar to Los Alamos to compete with their designs. This led to the creation of an offshoot of the RadLab (now the DOE’s Lawrence Livermore National Laboratory) in 1952. Some of the RadLab’s work was transferred to the new lab, but some classified research continued at Berkeley Lab until the 1970s, when it became a laboratory dedicated only to unclassified scientific research.
Shortly after the death of Lawrence in August 1958, the UC Radiation Laboratory (both branches) was renamed the Lawrence Radiation Laboratory. The Berkeley location became the Lawrence Berkeley Laboratory in 1971, although many continued to call it the RadLab. Gradually, another shortened form came into common usage, LBNL. Its formal name was amended to Ernest Orlando Lawrence Berkeley National Laboratory in 1995, when “National” was added to the names of all DOE labs. “Ernest Orlando” was later dropped to shorten the name. Today, the lab is commonly referred to as “Berkeley Lab”.
The Alvarez Physics Memos are a set of informal working papers of the large group of physicists, engineers, computer programmers, and technicians led by Louis W. Alvarez from the early 1950s until his death in 1988. [Louis and his son Walter developed the “Alvarez hypothesis” which proposes that the extinction event that wiped out the non-avian dinosaurs was the result of an asteroid impact. They discovered world covering belt of iridium from the asteroid impact. It can still be seen today.] Over 1700 memos are available on-line, hosted by the Laboratory.
The lab remains owned by the Department of Energy, with management from the University of California. Companies such as Intel were funding the lab’s research into computing chips.
Science mission
From the 1950s through the present, Berkeley Lab has maintained its status as a major international center for physics research, and has also diversified its research program into almost every realm of scientific investigation. Its mission is to solve the most pressing and profound scientific problems facing humanity, conduct basic research for a secure energy future, understand living systems to improve the environment, health, and energy supply, understand matter and energy in the universe, build and safely operate leading scientific facilities for the nation, and train the next generation of scientists and engineers.
The Laboratory’s 20 scientific divisions are organized within six areas of research: Computing Sciences; Physical Sciences; Earth and Environmental Sciences; Biosciences; Energy Sciences; and Energy Technologies. Berkeley Lab has six main science thrusts: advancing integrated fundamental energy science; integrative biological and environmental system science; advanced computing for science impact; discovering the fundamental properties of matter and energy; accelerators for the future; and developing energy technology innovations for a sustainable future. It was Lawrence’s belief that scientific research is best done through teams of individuals with different fields of expertise, working together. His teamwork concept is a Berkeley Lab tradition that continues today.
Berkeley Lab operates five major National User Facilities for the DOE Office of Science:
The Advanced Light Source (ALS) is a synchrotron light source with 41 beam lines providing ultraviolet, soft x-ray, and hard x-ray light to scientific experiments.
LBNL/ALS
The ALS is one of the world’s brightest sources of soft x-rays, which are used to characterize the electronic structure of matter and to reveal microscopic structures with elemental and chemical specificity. About 2,500 scientist-users carry out research at ALS every year. Berkeley Lab is proposing an upgrade of ALS which would increase the coherent flux of soft x-rays by two-three orders of magnitude.
The DOE Joint Genome Institute supports genomic research in support of the DOE missions in alternative energy, global carbon cycling, and environmental management. The JGI’s partner laboratories are Berkeley Lab, DOE’s Lawrence Livermore National Laboratory, DOE’s Oak Ridge National Laboratory, DOE’s Pacific Northwest National Laboratory , and the HudsonAlpha Institute for Biotechnology. The JGI’s central role is the development of a diversity of large-scale experimental and computational capabilities to link sequence to biological insights relevant to energy and environmental research. Approximately 1,200 scientist-users take advantage of JGI’s capabilities for their research every year.
The LBNL Molecular Foundry [above] is a multidisciplinary nanoscience research facility. Its seven research facilities focus on Imaging and Manipulation of Nanostructures; Nanofabrication; Theory of Nanostructured Materials; Inorganic Nanostructures; Biological Nanostructures; Organic and Macromolecular Synthesis; and Electron Microscopy. Approximately 700 scientist-users make use of these facilities in their research every year.
The DOE’s NERSC National Energy Research Scientific Computing Center is the scientific computing facility that provides large-scale computing for the DOE’s unclassified research programs. Its current systems provide over 3 billion computational hours annually. NERSC supports 6,000 scientific users from universities, national laboratories, and industry.
DOE’s NERSC National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory.
NERSC is a DOE Office of Science User Facility.
The DOE’s Energy Science Network is a high-speed network infrastructure optimized for very large scientific data flows. ESNet provides connectivity for all major DOE sites and facilities, and the network transports roughly 35 petabytes of traffic each month.
Berkeley Lab is the lead partner in the DOE’s Joint Bioenergy Institute , located in Emeryville, California. Other partners are the DOE’s Sandia National Laboratory, the University of California campuses of Berkeley and Davis, the Carnegie Institution for Science, and DOE’s Lawrence Livermore National Laboratory. JBEI’s primary scientific mission is to advance the development of the next generation of biofuels – liquid fuels derived from the solar energy stored in plant biomass. JBEI is one of three new U.S. Department of Energy (DOE) Bioenergy Research Centers (BRCs).
Berkeley Lab has a major role in two DOE Energy Innovation Hubs. The mission of the Joint Center for Artificial Photosynthesis (JCAP) is to find a cost-effective method to produce fuels using only sunlight, water, and carbon dioxide. The lead institution for JCAP is the California Institute of Technology and Berkeley Lab is the second institutional center. The mission of the Joint Center for Energy Storage Research (JCESR) is to create next-generation battery technologies that will transform transportation and the electricity grid. DOE’s Argonne National Laboratory leads JCESR and Berkeley Lab is a major partner.
sciencesprings 