High-performance brain-to-text communication via handwriting
By Francis R. Willett, Donald T. Avansino, Leigh R. Hochberg, Jaimie M. Henderson, Krishna V. Shenoy. Nature, 2021
871 citations recorded by the source corpus.
Brain–computer interfaces (BCIs) can restore communication to people who have lost the ability to move or speak. So far, a major focus of BCI research has been on restoring gross motor skills, such as reaching and grasping1–5 or point-and-click typing with a computer cursor6,7. However, rapid sequences of highly dexterous behaviours, such as handwriting or touch typing, might enable faster rates of communication. Here we developed an intracortical BCI that decodes attempted handwriting movements from neural activity in the motor cortex and translates it to text in real time, using a recurrent neural network decoding approach. With this BCI, our study participant, whose hand was paralysed from spinal cord injury, achieved typing speeds of 90 characters per minute with 94.1% raw accuracy online, and greater than 99% accuracy offline with a general-purpose autocorrect. To our knowledge, these typing speeds exceed those reported for any other BCI, and are comparable to typical smartphone typing speeds of individuals in the age group of our participant (115 characters per minute)8. Finally, theoretical considerations explain why temporally complex movements, such as handwriting, may be fundamentally easier to decode than point-to-point movements. Our results open a new approach for BCIs and demonstrate the feasibility of accurately decoding rapid, dexterous movements years after paralysis. A brain–computer interface enables rapid communication through neural decoding of attempted handwriting movements in a person with paralysis.
Subfields: Cellular and Molecular Neuroscience, Cognitive Neuroscience, Electrical and Electronic Engineering
View Paper: https://www.semanticscholar.org/paper/bebaab40b42e63cd7cef7b334e5f6bacdcccfc79
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Cognitive Neuroscience
Citations in the corpus
- Semantic reconstruction of continuous language from non-invasive brain recordings (2022, methodology): “No statistical methods were used to predetermine sample sizes, but our sample sizes are similar to those reported in previous publications 1,3,4,18,19 .”
- Scalable Bayesian GPFA with automatic relevance determination and discrete noise models (2021, neutral): “to brain-computer interfaces in human subjects [21, 40, 56].”
- AI in health and medicine (2022, background): “Moreover, AI’s ability to directly transform brain waves to speech or text has remarkable potential value for patients with aphasia or locked-in syndrome who have had strokes 62 .”
- Spinal cord injury: molecular mechanisms and therapeutic interventions (2023, background): “Another clinical trial showed that the neural activities of the motor cortex can be decoded into handwritten actions through BCI, significantly improving the writing speed and accuracy of paralyzed hands in patients with SCI (NCT00912041).(383)”
- Insular intracranial activity identifies multiple facial expressions via diverse, intermixed temporal patterns at the single-contact level (2026, unknown): “The importance of temporal features in discriminating between facial expressions was evaluated by a PCA-based procedure 42,43 .”
- Single-Neuron Recordings Research in Children: Ethical Considerations, Feasibility, Technical Aspects, and Scientific Opportunities (2025, unknown): “1), leading to, among many others, advances in our understanding of human language [2], memory [3, 4], sensorimotor function [5–7], psychiatric disease [8, 9], executive control [10–14], and brain-machine interfaces [15–18].”