Opening the world of books to 32 million deaf children, StorySign truly embodies Huawei’s brand promise to ‘push the limits of what is humanly possible’. The first global literacy platform for the deaf, StorySign was created by an alliance of international partners, including 12 deaf charities, Academy Award-winning animators, and the world’s leading publisher. This revolutionary app shows how AI can make a positive difference, by helping to solve a human problem. And nothing is more human than a child being able to enjoy a bedtime story for the first time.
Background
Huawei are the world’s second largest smartphone manufacturer. Their brand promise is to ‘push the limits of what is humanly possible’, and they challenged us to bring it to life in way that made a positive difference in society. So we identified a human problem their technology could help solve.
There are 32 million deaf children in the world, and most of them struggle to read. Unlike hearing children, deaf children can’t learn phonetically, so they struggle to match words with sounds. This has a hugely negative effect on their futures, reducing their opportunities, impacting on their employability, future earnings, even their health. In fact, research shows that it takes 21 years for deaf children to catch up with their hearing peers.
We agreed three objectives:
• Raise awareness of deaf literacy issues
• Use innovative mobile technology to open the world of books to deaf children
• Improve Huawei brand perception
Describe the idea
We created the first global literacy platform for the deaf – StorySign. A revolutionary free app, StorySign scans the words in children’s books and instantly translates them into sign language, using the most advanced signing avatar ever. For the first time, the world’s 32 million deaf children and their parents can learn to read and sign, together.
StorySign uses the power of Huawei’s Kirin AI chip to boost performance. AI means the app is faster and more accurate, reads text at up to 45° with seamless image recognition, and even works in low light conditions for bedtime stories.
Available across Android and IOS in 13 national sign languages so far, through StorySign Huawei has shown how a brand promise, for them ‘push the limits of what is humanly possible’, can deliver real change, in this case rewriting the story of deaf illiteracy for a generation of deaf children.
What were the key dates in the development process?
June to December 2017 – Concept Development
After identifying the human problem of deaf illiteracy as a way to showcase the positive power of AI, we worked with experts from many fields to validate our innovative solution - StorySign. These included the British Deaf Association and academics at UCL and Cambridge University. All of them saw StorySign’s potential as a gamechanger for literacy in the deaf community.
January to June 2018 – Testing and UX development
Once the concept of StorySign had been validated, we wanted to make sure it was designed with the everyday needs of our audience, deaf children and their families, in mind. StorySign’s UX is unique, because it was designed to be used by a parent and deaf child, in conjunction with a physical book. We ran multiple user testing workshops, both at home and within deaf schools, to allow children and parents to play with the app prototype, and evolve it based on their feedback.
June to November 2018 – App development
With the feedback from our core audience in mind, we worked with Huawei and the Academy Award-winning team at Aardman to develop the full StorySign platform ready for launch. This included creating the most advanced signing avatar ever, building in the AI app benefits, and crafting the design to reinforce the link between words and signs. We worked with Penguin Random House to ensure the launch books on StorySign included titles every deaf child would love, like Spot the Dog and Peter Rabbit. We used state-of-the-art motion capture facilities at Pinewood Studios to capture the sign language performances to an unprecedented level of detail, including fingertips, facial expressions, even the tongue!
3 December 2018 – StorySign launches
In the run-up to Christmas, StorySign was released on Android in 11 countries, in 11 different sign languages, fully supported by 11 international deaf charities including The European Union of the Deaf.
December 2018 to present – Impact and expansion
The reaction to StorySign, both within the deaf community and from a broad global audience, exceeded all our expectations. Over 1000 pieces of international media coverage were devoted to the app, and the feedback from deaf children and their parents was hugely positive. The impact was so great that Huawei and all the StorySign partners have continued to expand the platform since it was launched, including releasing an IOS version in Sept 2019, adding 52 more books so far, and supporting American Sign Language (used in US, Canada, Africa, Asia and Central America). The World Federation of the Deaf have even become official StorySign charity partners, a sign of how much they believe in its potential.
Describe the innovation / technology
Available on Android and IOS, executing StorySign took an alliance of world-class partners over a year to create.
Working with 12 international deaf charities ensure StorySign has the needs of deaf children at its heart.
Testing with schools and families resulted in a unique user experience. Hold your smartphone over the words in children’s books, and StorySign translates them into sign language. Words are highlighted in sign language order to reinforce that crucial link between words and signs.
Huawei’s Kirin AI chip means StorySign is faster and more accurate, can read text at up to 45 degrees, and even works in low light, perfect for bedtime stories.
Oscar-winning Aardman Animations created ‘Star’, the most advanced signing avatar ever. Every detail appeals to deaf children, from her age (because children learn better from siblings), to her hearing aid (to destigmatise deafness).
Penguin Random House, the world’s leading publisher, are adding their huge library of books, so deaf children can enjoy the same stories as their hearing friends.
StorySign brings together expertise from many different fields to improve the literacy of deaf children across the world.
Describe the expectations / outcome
“This technology can change the future of an entire generation.” - Mark Wheatley, European Union of the Deaf.
StorySign has made such an impact in the deaf community and on Huawei’s brand that they are keeping the story going, with a $500K donation to deaf literacy projects and future expansion confirmed.
Results have exceeded every objective.
Raise awareness of deaf literacy issues:
• 1.5bn+ global reach
• 128m+ app film views
• 1000+ pieces of international coverage
Use innovative mobile technology to open the world of books to deaf children:
• Available on Android and IOS
• Launched in 11 countries
• Supported by 12 international deaf charities
• 69 books so far
• American Sign Language now available, which is used across the US, Canada, Africa, Asia and Central America
Improve Huawei brand perception:
• +19.9% within 2 months
Selected media reaction:
• Forbes: “A portal to a whole new world.”
• Wired: “AI translates kids’ books into sign language.”