Mind-reading implant can decode what your ears are hearing

A device that can recognise what words people are hearing and reproduce them in a robot voice could take us further down the road to reading the minds of people who can’t speak.

The technique used temporary electrodes placed in the brain to monitor people before surgery, but the aim is to make a permanent implant.

So far the approach has only been able to decode simple words that people were listening to. But the researchers hope that with further development it will understand speech that people are thinking but not voicing.

If successful it could help people who can’t speak because they are paralysed after a stroke, for instance, or have motor neuron disease.

At the moment such people can generate computer speech by using a headset that places electrodes on the outside of their head. These can detect simple “flashes” of brain activity that lets users select letters on a screen in front of them, but communicating this way is slow.

“Speech is much faster than we type,” says Nima Mesgarani at Columbia University’s Zuckerman Institute in New York. “We want to let people talk to their families again.”

Mesgarani’s team is trying to use electrodes that connect directly with the brain. It’s potentially risky so they exploited the fact that people who need surgery for epilepsy often have electrodes put into or on the surface of their brain temporarily to find out where their seizures are coming from.

They asked five people in hospital who had either of these kinds of electrodes in place for a few days to listen to recordings of sentences. Their brain activity was used to train the group’s artificial intelligence speech recognition software.

Then the patients heard 40 numbers spoken, from zero to nine; the AI tried to decode what they had heard and reproduce the words. Its success was gauged by a separate group of healthy volunteers – they correctly guessed what the AI was saying three-quarters of the time.

Other groups are developing ways to read people’s minds while they lie inside an fMRI brain scanner. So far they can decode simple concepts such as the objects or celebrities someone is looking at, but this would be an impractical communication tool as the equipment takes up a small room.

Journal Reference: Scientific reports