Brain2Qwerty decodes typed sentences from brain activity recorded outside the skull

What happened

Thirty-five healthy volunteers typed briefly memorised sentences on a QWERTY keyboard while their brain activity was recorded with either magnetoencephalography (MEG) or electroencephalography (EEG). A new deep-learning architecture, Brain2Qwerty, was trained to decode the sentences from those recordings.

With MEG the model reached an average character error rate of 29%, far better than the 65% it achieved with EEG. For the best participants the error rate was 18%, and the model could perfectly decode a variety of sentences that were not in its training set.

Why it matters

The most accurate communication neuroprostheses require electrodes implanted in the brain, with the risks of neurosurgery. The authors present this as narrowing the gap between invasive and non-invasive decoding, a step towards safer interfaces for people who cannot communicate.

Caveats

The participants were healthy volunteers who were physically typing, so the recordings include activity related to real hand movements. The method was not tested in people who cannot move or speak. An average of 29% of characters wrong is still far from the accuracy of implanted systems. The project was funded by Meta, which employs the corresponding author and several co-authors. This entry is based on the abstract.