The investigational BCI, developed at the university of California, Davis, aims to provide faster and more natural conversation capabilities for individuals with neurological conditions that impair speech. ALS, also known as Lou gehrig’s disease, is a progressive neurodegenerative disease that affects nerve cells in the brain and spinal cord, leading to muscle weakness and eventual paralysis.

“Our voice is part of what makes us who we are. Losing the ability to speak is devastating for peopel living with neurological conditions,” said David Brandman, co-director of the UC Davis Neuroprosthetics Lab. He added, “The results of this research provide hope for people who want to talk but can’t. we showed how a paralyzed man was empowered to speak with a synthesized version of his voice. This kind of technology could be transformative for people living with paralysis.”

Real-Time Voice Synthesis Breakthrough

Current speech neuroprostheses often suffer from slow conversion speeds, with delays of several seconds between brain signals and audible speech, which impedes natural conversation flow.

“This new real-time voice synthesis is more like a voice call. With instantaneous voice synthesis, neuroprosthesis users will be able to be more included in a conversation. For example, they can interrupt, and people are less likely to interrupt them accidentally,” said sergey Stavisky, senior author and an assistant professor in the UC Davis Department of Neurological Surgery.

“Our voice is part of what makes us who we are. Losing the ability to speak is devastating for people living with neurological conditions.”

The new device uses surgically implanted microelectrode arrays in the brain’s speech-producing region to decode brain signals with high precision. the 256 electrodes capture the activity of hundreds of neurons, transmitting these signals to computers that interpret and reconstruct the voice.

Reduced Delay Enables Spontaneous communication

As part of the BrainGate2 clinical trial at UC Davis Health,researchers worked with a 45-year-old man. During the training phase, the participant was shown sentences on a screen and asked to attempt to speak them aloud while his brain activity was recorded.

The BCI swiftly translated the participant’s brain signals into audible speech with a delay of only one-fortieth of a second, closely mirroring the natural experience of hearing one’s own voice. This minimal delay allows for genuine, spontaneous conversation. The participant could even modulate the pitch of his voice to sing simple melodies.

According to the study, the BCI-synthesized voice achieved a 60% word recognition rate among listeners. Advanced artificial intelligence (AI) algorithms played a crucial role in enabling real-time speech generation by correlating the participant’s neural firing patterns with his intended speech sounds.

“The main barrier to synthesizing voice in real-time was not knowing exactly when and how the person with speech loss is trying to speak,” said Maitreyee Wairagkar, first author and project scientist. Wairagkar added, “Our algorithms map neural activity to intended sounds at each moment of time. This makes it possible to synthesize nuances in speech and give the participant control over the cadence of his BCI-voice.”

The researchers emphasize that brain-to-voice neuroprostheses are still in the early stages of development. Future research will focus on replicating these results with a larger and more diverse group of participants, including individuals with speech loss resulting from conditions such as stroke.

The complete study was published in the journal Nature.