He calls it his “chocolate and peanut butter moment.”
A neuroimaging researcher at the University of Buffalo has developed a computer model of the human brain that more realistically simulates real patterns of brain damage than existing methods. The novel breakthrough represents the union of two established approaches to create a digital simulation environment that could help stroke victims and patients with other brain injuries by serving as a testing ground for specific neurological damage hypotheses.
This model is precisely linked to the functional connectivity of the brain and can demonstrate realistic patterns of cognitive impairment. Since the model reflects how the brain is connected, we can manipulate it in a way that provides information, for example, about areas of a patient’s brain that could be damaged.
This recent work does not prove that we have a digital facsimile of the human brain, but the results indicate that the model is functioning in a manner consistent with the functioning of the brain, and that it at least suggests that the model is taking properties that move in the direction of possibly one day creating a facsimile. “
Christopher McNorgan, assistant professor of psychology at the Faculty of Arts and Sciences of the UB
The findings provide a powerful means to identify and understand brain networks and how they work, which could lead to what were once undiscovered possibilities of discovery and understanding.
Details about the model and the results of its tests appear in the journal. NeuroImage.
Explaining the McNorgan model begins with a look at the two fundamental components of its design: functional connectivity and multivariate pattern analysis (MVPA).
For many years, traditional brain-based models have been based on a general linear approach. This method analyzes each point of the brain and how those areas respond to stimuli. This approach is used in traditional studies of functional connectivity, which rely on functional magnetic resonance imaging (fMRI) to explore how the brain is connected. A linear model assumes a direct relationship between two things, such as the visual region of the brain that becomes more or less active when a light is turned on or off.
While linear models are excellent for identifying which areas are active under certain conditions, they often cannot detect complicated relationships that potentially exist between multiple areas. That is the domain of the most recent advances, such as MVPA, a “teachable” machine learning technique that operates at a more holistic level to assess how activity is modeled in brain regions.
MVPA is nonlinear. Suppose, for example, that there is a set of neurons dedicated to recognizing the meaning of a stop signal. These neurons are not active when we see something red or something octagonal because there is no one-to-one linear mapping between being red and being a stop sign (an apple is not a stop sign), nor between being octagonal and being a signal of stop (a boardroom table is not a stop sign).
“A non-linear response ensures that they light up when we see an object that is red and octagonal,” McNorgan explains. “For this reason, nonlinear methods such as MVPA have been at the center of the so-called ‘Deep Learning’ approaches behind technologies, such as the computer vision software required for driverless cars.”
But MVPA uses brute force machine learning techniques. The process is opportunistic, sometimes confusing coincidence with correlation. Even ideal models require researchers to provide evidence that activity in the theoretical model would also be present in the same conditions in the brain.
On their own, both traditional functional connectivity and MVPA approaches have limitations, and the integration of the results generated by each of these approaches requires considerable effort and experience for brain researchers to discover the evidence.
However, when combined, the limitations are mutually limited, and McNorgan is the first researcher to successfully integrate functional connectivity and MVPA to develop a machine learning model that is explicitly based on real-world functional connections between brain regions. In other words, the mutually limited results are a self-assembly puzzle.
“It was my chocolate and peanut butter moment,” says McNorgan, an expert in neuroimaging and computational modeling.
“I have had a particular professional career that has allowed me to work extensively with different theoretical models. That experience gave me a particular set of experiences that made the combination seem obvious in retrospect.”
To build their models, McNorgan begins by gathering brain data that will teach them the patterns of brain activity associated with each of the three categories, in this case, tools, musical instruments and fruits. These data come from 11 participants who imagined the appearance and sound of examples of familiar categories, such as hammers, guitars and apples, while undergoing an MRI. These scans indicate which areas are more or less active according to blood oxygen levels.
“There are certain activity patterns in the brain that are consistent with thinking in one category versus another,” says McNorgan. “We could think of this as a neuronal fingerprint.”
These MRI patterns were digitized and used to train a series of computer models to recognize which activity patterns were associated with each category.
“After training, the models receive activity patterns never seen,” he explains. “A classification accuracy significantly higher at random indicates that the models have learned a generalizable relationship between specific patterns of brain activity and think of a specific category.”
To test whether the digital brain models produced by this new method were more realistic, McNorgan gave them “virtual injuries” by interrupting activations in regions known to be important for each of the categories.
He found that mutually restricted models showed classification errors consistent with the location of the lesion. For example, injuries in areas that are considered important for representing tools altered the accuracy of tool patterns, but not the other two categories. In comparison, other versions of untrained models using the new method did not show this behavior.
“The model now suggests how areas of the brain that may not seem important to encode information when considered individually, may be important when it functions as part of a larger configuration or network,” he says. “Knowing these areas can help us understand why someone who suffered a stroke or other injury has trouble making these distinctions.”
McNorgan, C., et al. (2020) Integrating functional connectivity and MVPA through a multi-constraint network analysis. Neurolmage. doi.org/10.1016/j.neuroimage.2019.116412.