Texting by gesture

You may think the writing's on the wall - and you may also think that that writing spells the future. But the future is in your hands. That is, in the movement of your hands - because very soon, you could be writing all your mobile texts and emails, with nothing more than hand gestures.

The man behind Airwriting - as the technology is called - is Christoph Amma, a computer scientist and PhD student Germany's Karlsruhe Institute of Technology (KIT).

In his office, Amma picks up a worn climbing glove from where it's lying on a paper-strewn desk.

Fitted out with a motion sensor, a Bluetooth wireless transmitter and a microcontroller, the glove isn't for climbing though - it's a prototype that allows words and even sentences written in the air to be captured and turned into text on a computer.

Today, Amma's going to let me take the glove for a test run.

As he slips the glove onto my hand, he tells me the rules: I am only to write in English, I have to use capital letters and I shouldn't wriggle my wrist around too much.

Amma then connects the circuit and the laptop on his desk springs into life.

I try to imagine I'm holding a piece of chalk and writing on an invisible blackboard, and I spell out the word "Spectrum" (the name of DW's Science and Technology radio program) in midair.

Unfortunately, the word fails to appear on the computer screen - it turns out I've picked a rare word that isn't in the system yet.

"There's a fixed vocabulary of about 8,200 words," says Amma, "taken from a list of the most frequent English words."

I have another couple of false starts. I keep confusing the system because I'm staring at the computer screen and hesitating instead of continuing to write.

Then I get the knack and the sentences "this is hard" and "this is easy," which I trace in the air, appear as if by magic on the laptop screen.

Pattern recognition algorithms

It isn't magic, though.

It's the result of five years of hard work, developing pattern recognition algorithms capable of interpreting gestures correctly.

In order to create statistical models of every letter of the alphabet, Amma recorded airwriting data from 23 volunteers.

"They had to write characters, words and sentences in the air and we then tested the system on the data we recorded from them," Amma says.

The airwriting system has an average error rate of 11 percent, which means that just over every tenth word is wrong. When adapted for the individual user, though, this drops to around three percent.

"That's because everyone has their own block writing style even if they all write in block capitals. So if you adapt it for the individual user, it gets a lot better," says Amma, quickly adding that decreasing the error rate is something he's working on.

Amma slips on the glove himself, showing me how it still recognizes words even when he traces out tiny letters into his hand. "That way, it's not so distracting," he says.

The airwriting system has been designed so it doesn't have to be turned on when the user wants to write.

It's capable of recognizing alphabet-like patterns from among the millions of arm movements we make in a normal day.

I test it again by waving my arms around and rummaging in my bag. The system takes no notice of my gestures, while "no handwriting" blinks on the screen. As soon as I start scribbling the word "science" in midair, it switches back to handwriting mode and identifies the word.

Usability

Much of the research into gesture interfaces is focused on the recognition of single gestures assigned to a certain command - such as opening an application on a smartphone by gesturing from right to left in front of the screen. But there's a limit to how many gestures people can or want to remember.

The strength of an alphabet-based gesture interface is that people already know their ABCs.

But the question now is how to incorporate the gestures so they can control a phone.

KIT professor, Tanya Schultz, is part of the airwriting research team and Amma's supervisor. She says she can imagine airwriting being integrated with devices such Google glasses.

The integrated screen in the glasses "would give you an idea of what the phone is doing and whether the text you type is what you actually want to say," says Schultz.

In that case, a handheld device wouldn't even be necessary.

But there's work to be done yet.

Schulz and Amma won a 2013 Google Research Award worth 81,000 dollars to further the development of their airwriting system.