For the first time, a bilingual person who struggles with articulating words to communicate has been able to speak two languages thanks to this AI-powered brain implant! The artificial intelligence (AI) system integrated into the brain implant interprets what the patient is attempting to say in either Spanish or English in real-time. 

The findings, published in Nature Biomedical Engineering, offer insights into how language is processed by our brains and could one day result in durable devices that can help those who are unable to communicate verbally. 

Experts say the study is a significant contribution to the emerging field of speech restoration neuroprostheses. Researchers say this strategy, when paired with other recent advances, is expected to work with higher accuracy in the future. 

The speech-restoring implant 

The heart of the study, Pancho, was only 20 when he suffered a stroke that left more than half of his body paralyzed. He can grunt and groan but is unable to speak correctly. 

To find out how the stroke affected his brain long-term, Pancho, in his thirties, teamed up with the neurologist Edward Chang of the University of California, San Francisco. 

Chang’s team implanted electrodes on Pancho’s cortex to capture neural activity, which was converted into words on a screen, in an innovative research study published in 2021. 

“My family is outside,” was Pancho’s first statement, which was translated into English. However, Pancho speaks Spanish natively and only picked up English after his stroke. Spanish still evokes in him feelings of familiarity and belonging. 

According to Chang, “a person’s identity is very linked to the languages they speak.” Thus, “restoring connection for people has always been our long-term goal, not just replacing words”, he said. 

The team created an AI system to interpret Pancho’s bilingual speech to accomplish this goal. Under the guidance of Alexander Silva, Chang’s Ph.D. student, Pancho attempted to speak almost 200 words while the system was being trained, under the guidance of Alexander Silva, Chang’s PhD student. His attempts to compose each word resulted in a unique neural pattern that the electrodes recorded. 

The authors then applied their AI system, which has a Spanish and English module, to phrases as Pancho tried to say them aloud. The Spanish module chooses the Spanish word that best fits the neural pattern as the first word in a phrase. 

The English component chooses from the English vocabulary instead of doing the same thing. For instance, if the English module selects “she” as the most likely first word in a phrase, it has a 70% chance of being correct, whereas the Spanish module might select “estar” (to be), with a 40% chance of being accurate. 

Word for word 

Both modules then try to construct a phrase. Based on the neural pattern match and the possibility that the second word will come after the first, they each select the second one. As a result, “I am” would have a larger probability than “I not.” 

The end result produces two lines, one in Spanish and one in English, but the display screen that Pancho faces shows only the version that has the highest total probability score. 

The modules decoded the proper text with a 75% accuracy rate and identified between English and Spanish based on the first word with 88% accuracy. Eventually, Pancho and the research team could have had candid talks. 

One brain area, two languages 

The results exhibited unanticipated facets of the brain’s language processing. Several earlier noninvasive instrument trials suggested that different languages stimulate different brain regions. However, after closely examining the signals captured in the cortex, the researchers discovered that “a lot of the activity for both Spanish and English was actually from the same area,” according to Silva. 

Furthermore, contrary to the findings of other studies, Pancho learned English when he was in his thirties, but his neural reactions didn’t seem to be all that different from those of kids who grew up bilingual. Taken as a whole, these results imply to Silva that there may be neural similarities between other languages and that these similarities could apply to different individuals.  

Source: Nature

Image: Unsplash