Breakthrough Neural Interface Reads Handwriting From Paralyzed Man's Brain
A groundbreaking brain-computer interface has allowed a paralyzed man to "type" at 90 characters per minute, with new research suggesting an incredible leap ahead in communication is possible for those using a "brain to text" system. Rather than attempt to make a virtual keyboard usable by reading the brain's neural activity, the team responsible for the breakthrough focused on tracking imaginary handwriting.
Existing systems that track the brain's activity and map that to a computer have typically relied upon the thoughts associated with arm movements. By tracking those, even if the arm itself can't move, they can be mapped to highlighting keys on a virtual keyboard or another kind of interface.
While it works, it's speed-limited. Current systems allow for around 40 characters per minute via brain computer interface (BCI), according to Krishna Shenoy, a Howard Hughes Medical Institute Investigator at Stanford University who, with Stanford neurosurgeon Jaimie Henderson, authored this new study. Instead of arm movements, they looked at brain activity when people imagine handwriting.
Turns out, according to HHMI research specialist and neuroscientist Frank Willett, who worked on the project, imagining letters that would be handwritten results in highly distinctive activity patterns. An algorithm trained to recognize those, then, can be much faster than the existing BCI system.
The study subject was a 65 year old man who, after a spinal cord injury, had been left paralyzed from the neck down. He had two sensors implanted into the part of the brain that typically controls the hand and arm. Linked to a computer, when he imagined writing letters as if with a pen on paper, the algorithms could convert those impulses into digital text.
"With this system, the man could copy sentences and answer questions at a rate similar to that of someone his age typing on a smartphone," the researchers say. In fact, he was able to produce text at 90 characters per minute, almost twice the rate of existing BCI systems.
The aim is to include mental handwriting as one input option, alongside rather than replacing existing point-and-click navigation. The team responsible for the new BCI has also worked on speech decoding, and they envisage a unified system that would support a number of different input modalities that collectively taps the speed and accuracy advantages of each.
Next, the group plans to work with another participant who is unable to speak, as the abilities of the system in development are rounded out. While it's too early for a production version of the BCI, the goal is to eventually allow paralyzed users the ability to communicate in real-time ways rather than force them to pick through more time-consuming interfaces.