As mentioned in the previous post, Android apps from Transcense and MotionSavvy are poised to enable two-way communication between deaf and hearing people. I wrote to both companies to know more about their apps.
MotionSavvy
MotionSavvy UNI for Indian English would be available in 2017. For now, if you have friends in the US who might be interested, let them know.
Transcense
Thibault, a founder of Transcense, had news which was exciting: Transcense plans to offer Indian English for testing around March, 2015. They are keen in making the app affordable and fast enough.
Interestingly, Thibault worked at IIT-Kharagpur in 2013. While in India, he also created a video documentary interviewing 25 leaders and founders of social businesses. Thibault pointed me to Mirakle Couriers, whose entire staff is deaf - do go through their site.
If you are interested in participating in community testing of the Transcense app, please follow this blog and I'll share the details in March.
The video below is about the Transcense solution - it was also linked in the previous post.
How do such apps help
To get an idea of how things could improve with the help of new solutions, look at the number of deaf people in India - it is 6-15 million, according to DeafWay.org. If you include their families, the number of people who can directly benefit when deaf people's lives improve is mindboggling.
Finding teachers who know sign language to teach deaf children must be a challenge - if speech-to-text technology for Indian English is available, hearing people who do not know sign language may be able to teach deaf children.
Speech to text would be more effective if deaf children are taught to read and write English well via sign language. Once that is done, the kind of technology we have seen will help with further learning. (The ultimate solution may be to translate English to sign language since that is their native language - this may take some more time though, and is not easy since sign language is supported by facial expression, body language and the fact that signs could vary in their speed, angle, completeness, range of movement etc.)
Education
Here are a few important points on the education of deaf children from DeafWay.org and other sites. I have extended their content based on my understanding:
Children learn their first language by imitating and interacting with family and those around.
Deaf children of deaf parents learn the sign language at home and are proficient in it when they join school (just like hearing children and their mother tongue). Only 10% of deaf children have deaf parents.
Deaf children who have hearing parents i.e. the other 90% of deaf children, do not learn sign language at home. When they join school, they know neither sign language nor a spoken language. They learn sign language mostly from other children at school who know it already.
A child must know one language in order to learn a second language like English. This emphasises the importance of learning sign language early in childhood.
Sign language is a visual language and has a different grammar from spoken languages.
Indian Sign Language has the same grammar all over India, with a variation of around 8-12% in terms of vocabulary. So, people of one region can very well understand people from another region of India.
It is somewhat more difficult for a deaf person to learn English for the following reasons:
They do not have the aid of sound while learning it
People do not speak English with the expressiveness present in sign language
They cannot practise by speaking it.
However, if deaf children learn English well at a young age, it opens up a whole new world for them, allowing them to learn anything they want.
Great, you stayed till the end. If I have got something wrong, please let me know.
At work, teams may have people who are deaf. Jay, my teammate, is deaf and cannot speak.
While Jay uses the Indian sign language to talk to those who understand it, he uses an instant messenger to talk to the rest of us. This led me to explore options for better interaction between us.
(I was not sure which term to use and turned to the internet. The term "deaf" is not considered derogatory and is not to be grouped with other offensive terms that denote impairment. The term "hearing impaired" is not specific to people who cannot hear at all - they do not consider it appropriate. But if you find the usage offensive, I apologise sincerely and request you to leave a comment).
Google Translate and ListNote
Most of my colleagues use Android, so I didn't look at apps for iOS.
The Google Translate app can be set to translate from English to English. It understands Indian accent, but not that well. Translate sends the voice over a data/wi-fi connection to Google's servers, where it is converted to text and sent back to Jay. Translate stops listening to speech when it senses a pause, making it an inconvenient option. Translate may have been just intended to translate phrases.
ListNote for Android does not stop listening when it senses a pause. But just like Google Translate, it wasn't that good with Indian English.
Things are looking up
Apps like Transcense and MotionSavvy are changing the landscape - these are being developed especially for people who cannot hear or speak.
Both companies are run mainly by qualified and motivated people who are deaf themselves or have family members who are deaf. Both are raising funds from the public on the indiegogo web site (crowdfunding).
Transcense
Transcense is a two-way communication tool for Jay. Its Android app is in beta testing. See the video below (Turn on captions in the video. I have also described it below with additional information).
In this beta stage, Jay can buy one of the "perks" i.e. plans listed on the indiegogo site. Usually, these are cheaper in the beta period and cost more when the product is in generally released later.
Jay could opt for the US$60 perk to get the Transcense app and become a beta tester (others can buy it as a gift for Jay too). After Jay installs the app, he can use it to invite people to join a conversation. The app sends a request to the mobile devices of his colleagues. They can just tap a button to accept and join the conversation.
When other people speak, their speech is transmitted via their mobile's mic to Jay's Transcense app. The app converts it to text and displays it on Jay's mobile (if you would like to know the technical details, the speech is actually sent over an internet connection to a server. The server converts speech to text and sends it back; With a good connection, all this happens in a second).
Now, not only can Jay clearly follow who says what, he can also type his response in Transcense, which converts text to speech. As far as I know, there is no other tool that can help Jay to participate in a group discussion like this.
Transcense is working on Indian English as well. They can help Jay and his team of deaf and hearing people to get a good success rate in translating Indian English to text (and text to speech).
The Infinity plan (unlimited data) would cost US$29/month - other limited data plans might come cheaper, though I could not see any right now. Beta testers will get a discount on this monthly rate.
MotionSavvy's UNI
MotionSavvy's UNI also allows effective two-way communication. The difference from Trancense is that instead of typing a reply, Jay can use American Sign Language . UNI converts sign language to speech. See this video for a clear explanation (it's a silent one with captions).
UNI is a bundle of software and hardware. It consists of:
A regular Dell tablet
A special case for the tablet that contains a motion sensor to detect sign language (sensor is also called a controller)
The UNI app
The UNI app does two things. When Jay's sits in front of it and uses sign language, UNI translates it to speech. Secondly, when the presenter replies, it converts his speech to text.
Signs can also be added to a dictionary. The dictionary can then be shared among users. This implies that support for Indo-Pak Sign Language can also be added by users themselves. It is not clear how difficult or easy this is.
UNI does not need a data connection and so can be used anywhere. UNI is likely to be available for both Windows and Android platforms in 2015 and cost US$19.99/month. Unfortunately, it may support Indian English only by 2017.
Dragon Naturally Speaking
Dragon Naturally Speaking (DNS) is Windows software that can be used on the desktop or laptop.
DNS understands Indian English. The presenter's speech is captured and converted to text and displayed in a Windows application like Word. It is amazing to see the high success rate on YouTube video, recorded by individuals (video-1, video-2).
Speech-to-text software work best with good quality audio, so one of the following is recommended:
Buy a good quality bluetooth headset and ask the speaker to wear it. Get a hands-free headset so that the presenter can move about freely - for example, to write on the board. This is the best approach for most situations.
If the presenter is seated and will not move around, Jay can sit next to him so that his laptop mic can pick up the speech well.
If your mobile can connect to your office wi-fi network, install Dragon's free iOS or Android app on it. This will connect your mobile to the Dragon Windows app. Now you can talk on your mobile and the audio will be passed on to Dragon, which will convert it to text.
There is another video from the maker of Dragon, Nuance, on improving accuracy.
The limitation in DNS is that there is no way for Jay to reply - he has to fall back on existing options or try other text to speech software. I could not find any reference to a text to speech feature in DNS.
If you cannot use DNS on the laptop, DNS has an app for iOS. It needs an internet connection to work. The system requirements list OS 4.0 - this might mean iOS 4.0. India and Indian English are not listed in the product support page - wonder what this implies.
Conclusion
Unfortunately, I left Jay's organization by the time this was written. Hopefully Jay and his colleagues can dig more into the subject. Unless other solutions come to notice, my suggestion would be this:
DNS can be used till a two-way solution becomes available for Indian English. That solution must be affordable in the Indian context too.
DNS needs high quality audio and hence may only be suitable for situations where one person speaks at a time (single speaker or speakers take turns presenting etc.).
Transcense's solution might be the best where there are many speakers.
It supports Indian English - test and help to improve its accuracy.
Check how good the round trip time is.
UNI may be a good alternative for some as it provides another way of two-way communication.
Check if UNI works when multiple speakers are involved, as in a meeting.
Keep track of it to know when they support Indian English.
Find out how easy it is add support for Indian Sign Language.
