In 1972, Julia Child’s PBS show The French Chef became the first program to offer captions for the hearing impaired. It was a watershed moment for the deaf and hard of hearing in the U.S., and television became increasingly inclusive thereafter.
Almost 50 years later, captioning has made its way to smart devices from Amazon, Google, Microsoft and Apple. And while technologies have radically changed consumer behavior, captioning remains the primary accessibility option offered by big tech for the deaf and hard of hearing. Generally, when it comes to voice-enabled devices without screens, the hearing impaired are out of luck.
But now startups are emerging with voice technology of their own to try to bridge the gap with the hearing world.
Computer programmer Abhishek Singh, for example, is using machine learning methods like deep learning to make a screenless Echo respond to sign language. A camera interprets queries in sign language, which then are converted to text and speech. Alexa’s response is then transcribed on a screen.
According to accessibility startup Ava, captioning, subtitles and sign language add a visual layer to increase accessibility, particularly in group conversations where reading lips is harder and hearing aids are not sufficient.
Ava has an app to support in-person conversations by converting group conversations into text for the deaf and for the hearing impaired. The company did not respond to requests for comment, but according to Google Play, Ava has more than 50,000 downloads.
“Interest from the deaf and hard of hearing [in Ava] was very, very big. Many people were interested. There were tens of thousands of downloads, but only fraction kept using the app,” said Jari Hazelebach, co-founder of the startup SpeakSee and former business developer at Ava—and the son of deaf parents.
After conducting interviews with Ava users, Hazelebach said they pointed to barriers, such as all participants in a given conversation having to download the app, but also the stigma associated with hearing loss, which sometimes makes it hard to ask others to download an app just to have a conversation.
“Many people still associate hearing loss with oldness and don’t want to admit they are not able to hear what’s being said,” Hazelebach said. “It can be very exhausting having to be so proactive and trying to listen and lip-read.”
That’s in part what inspired Hazelebach to develop hardware a deaf person could hand out to start conversing right away. He said he posed the idea to Ava, but it’s a very big leap from software to hardware—hardware is generally expensive—and it didn’t work out.
Instead, Hazelebach spent 18 months working on a prototype with technical co-founder Marcel van der Ven. The two met on Founder2Be, which Hazelbach described as being like a dating site for entrepreneurs.
SpeakSee opened preorders in June for a portable kit of smart microphones that transcribe speech to text in real time. Sets include a charging dock and wearable microphones that capture the voice of the speaker, isolate it from background noise and transcribe the conversation to a smartphone. That means users can distribute wearable microphones prior to a conversation and follow along on their phones.
“It all started three years ago when my father, at the dinner table, was talking about ways to be included in conversations at work because he was not able to follow along in business meetings and missed out on a lot of relevant information to do his work successfully,” Hazelebach said. “Then we thought of speech recognition—technology to convert the spoken word to text instantly in real time, which is like a utopia for someone who is deaf—it’s easy to follow without having to do lip-reading or needing an interpreter, so that’s when we started thinking about speech recognition.”
Hazelebach understands speech technology isn’t a replacement for sign language, but rather a supplement for group conversations in which existing speech recognition options from big tech are not able to identify individual speakers, making confusing to participants.
Rumors have swirled for nearly a year that Google, Facebook and Apple are working on screened devices, which could theoretically enable enhanced accessibility, but they would not comment on their plans.
In an email, Amazon noted, “We don’t have much to share at this point, but we do hear from customers every day that Alexa has changed their lives for the better and we’re both humbled and inspired by that feedback.”
An Indiegogo campaign has raised nearly $90,000 from more than 250 backers for SpeakSee, which is offering an early-bird deal for one dock and three mics for $349. Other configurations are available and Hazelebach said about 300 kits have been ordered so far. The first trial product run will be available in December and regular sales will start in February.
“My parents have done some initial experiments … and are very excited about the technology and would really like to use it in daily life—especially my father,” Hazelebach said.
