Researchers at the Georgia Institute of Technology have created speech-to-text software for Google Glass that helps hard-of-hearing users with everyday conversations. The setting involves hard-of-hearing individuals wearing Glass while a second person speaks directly into a smartphone. The speech is converted to text, sent to Glass and displayed on its heads-up display.

School of Interactive Computing Professor Jim Foley is one of those having trouble hearing, and he’s impressed with the software. “If hard-of-hearing people understand the speech, the conversation can continue immediately without waiting for the caption,” he said. “However, if I miss a word, I can glance at the transcription, get the word or two I need and get back into the conversation.”

If captioning errors are sent to Glass, the smartphone software also allows the speaker to edit the mistakes.Foley’s colleague, Professor Thad Starner, leads the Contextual Computing Group working on the project. He says using a smartphone with Glass has several benefits as compared to using Glass by itself. For one thing the microphone that Google Glass has is primarily designed for the wearer, whereas the mobile phone puts a microphone directly next to the speaker’s mouth, reducing background noise and helping to eliminate errors.

Starner added that the phone-to-Glass system would also help speakers construct their sentences more clearly, avoiding “uhs” and “ums.” However, if captioning errors are sent to Glass, the smartphone software also allows the speaker to edit the mistakes, sending the changes to the person wearing the device.

“The smartphone uses the Android transcription API to convert the audio to text,” said Jay Zuerndorfer, the Georgia Tech Computer Science graduate student who developed the software. “The text is then streamed to Glass in real time.”

Captioning on Glass is now available to install from MyGlass. It’s not the only such application Foley and the students are working on; another project, Translation on Glass, uses the same process to capture sentences spoken into the smartphone, translate them to another language and send them to Glass. The group is working to get Translation on Glass ready for the public.