Someday, instead of typing your text message on a cramped iPhone keyboard, neuroscientist Michael Linderman says you'll be scrawling your thoughts in the air.
Linderman and colleagues have figured out how to translate electrical impulses from muscles in the forearm and hand into written language. Using pattern-recognition algorithms and a technique called electromyography, the researchers can recognize and reproduce a person's unique handwriting pattern from the movement of their hands. Eventually, they hope to create a fingerless glove equipped with electrode sensors that can automatically translate hand motions into digital or handwritten text.
"You can write in the air, can write on any surface, can write using a pen or pencil or even without," said Linderman, "and this will be converted into text on the display of a cellphone or computer."
For the first phase of their research project, published Wednesday in PLoS ONE, the scientists asked six volunteers to don a prototype glove that recorded the electrical activity of eight muscles in the hand and forearm. Each subject wrote the numerals zero to nine 50 times, and a computer recorded both the input from their digital pen and the electrical activity of their hand muscles as they completed the exercise.
Then, using pattern recognition software, the researchers taught the computer to convert the electrical-impulse tracings into written language. "We wanted the system to be able to recognize those bursts of electricity as particular patterns associated with this type of mechanical activity," Linderman said.
The results of the experiment can be seen in the figure What participants actually wrote is shown in blue, while their handwriting reconstructed from the electrical recordings is in red. The reproduction isn't perfect, but since submitting the paper, Linderman says the team has significantly refined its process.
"That was using a very simplistic algorithm, because we had a very short time to work on this project," he said. "We do not have the pictures yet, but we have certainly improved the algorithms."
Once they could accurately reproduce handwriting from electrical tracings, the team tested how well their computer program could recognize characters based on muscle movements using a technique called discriminate analysis. After five repetitions of each character, the computer could recognize 63 percent of the numerals; with 35 training repetitions, the computer achieved 97 percent accuracy.
Once they could accurately reproduce handwriting from electrical tracings, the team tested how well their computer program could recognize characters based on muscle movements using a technique called discriminate analysis. After five repetitions of each character, the computer could recognize 63 percent of the numerals; with 35 training repetitions, the computer achieved 97 percent accuracy.
"I think it's a very solid piece of work," said electrophysiology expert Andrew Fuglevand of the University of Arizona, who was not involved in the research but has consulted with Linderman in the past. "It's something that they should in the future be able to use to as a way to electronically extract somebody's handwriting based on recording the patterns of their muscle activity." However, Fuglevand says he has a hard time envisioning what sort of practical functions the technology might serve.
Linderman, on the other hand, has no trouble imagining myriad uses for his device. He says digital hand technology isn't just for teenagers who want to send a faster text message: Because many neuromuscular disorders, including Parkinson's and Alzheimer's, often start with mild hand tremors, he thinks the glove could be used as a screening device to catch diseases earlier. It might also be useful for creating prosthetic writing devices, he said, or helping patients with a hand tremor learn to write again.
Linderman founded the company Norconnect Inc. to market his device, and last February, the researchers received a NSF grant of more than $450,000 for the second phase of their research.
But medical indications aside, there's at least one major drawback to this technology: If people look silly now while talking on a bluetooth, just think how ridiculous they'll look waving their hands in the air to send text messages.
