The sign above says "Hello"
Hey!
Duo and Ann are working on the case study, but feel free to check our progress!
MY ROLE
Interaction design, research, user interviews, user flows, rapid prototyping and usability testing and iterations
TIMELINE
- 3 weeks.
- by Anusha Babu, Alejandro Zaldua and Divya Kanchanbaras, Ashin Joseph
TOOLS
Figma, Trello, Zoom, Slack, Illustrator, Maze
What is SignTalk?
SignTalk is a digital experience for learning sign language.
The idea of SignTalk came from an interaction I and Divya had with a person from the Deaf and Mute community, who was actually Divya's Aunt. She was born deaf and mute, a well educated and works for the empowerment of deaf and mute. Her life as a deaf and mute trainer inspired us to create SignTalk.
Our team diagnosed!
The deaf and mute community and the general population struggles to establish an effective communication with each other because of language limitations. There is a lack of awareness about sign language and its importance, thus hindering the two groups to communicate.
This is what we prescribed!
Our team introduced a character "Simon-The Signman" like any other character in the Duolingo application to learn the sign language along with the language preferred by the user to learn. Making sure the language is not force fed.
Let's see how Simon makes a difference.
Impact
Through our research and testing, we were able to bridge the communication gap between the Deaf and Mute community and the general population by integrating sign language learning into an engaging, gamified experience.
Here’s what we achieved:
1. Increased Awareness
88% of our surveyed participants acknowledged the importance of sign language after interacting with SignTalk.
3. Higher Willingness to Learn
Initially, users were uninterested in learning sign language, but after interacting with Simon, 72% expressed a desire to learn more.
2. Enhanced Engagement
Our testing showed that 4 out of 5 users found Simon’s integration engaging and enjoyable.
4. Seamless Learning Experience
By aligning with Duolingo’s existing gamification system, users found learning sign language as intuitive as learning any other language.
Let me tell you the story of our interview with Tulsi, who is a Deaf and Mute and a trainer in Sai Swayam institute in Delhi, India.
A GIF of the interview with Tulsi who is deaf and mute
Were you able to interpret her signs?
Neither did I at first, but we will come to it. This interview changed our perspective about their world, and it will change your's too.
![Wavy_Bus-17_Single-09 [Converted]-01.png](https://static.wixstatic.com/media/f792cc_1352859a006c44b1a4055af6e10e652a~mv2.png/v1/fill/w_358,h_358,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/Wavy_Bus-17_Single-09%20%5BConverted%5D-01.png)
Tulsi's silent beginnings
Tulsi Sathuja's story is a tale of incredible challenges and unwavering determination.
Born deaf and non-verbal in Delhi, her condition went unnoticed until later in her childhood. Upon diagnosis, doctors recommended hearing aids and speech therapy, marking the beginning of her challenging path.
Tough times in school for Tulsi
Tulsi's education in a regular school posed significant difficulties as she lacked fundamental language skills. Seeking assistance from her teachers often yielded empty promises due to their busy schedules. Exams were a constant struggle, with teachers providing answers to copy, leading to numerous errors. Making friends proved challenging as she couldn't follow conversations.
This is what she said about her education, "My longest time in school was wasted like this."
A Fresh Start with Sign Language at 16
At 16, she encountered the head of an inclusive school who introduced her to sign language, She persevered, completing her 12th grade at the inclusive school.
Despite her dream of pursuing a career in IT, others discouraged her, assuming her disability would be a hindrance. Nevertheless, Tulsi remained resolute, joining "Sai Swayam," an institution for the deaf and mute. Here, she acquired various skills and ultimately became a sign language trainer, empowering individuals like herself to find their paths in life.
-01.png)
"My parents understood that I am deaf, when they saw that I wasn't responding when they clapped or called out my name."
And this is what Tulsi said at the start.
A GIF of the interview with Tulsi who is deaf and mute
Despite knowing English, Hindi, and sign language, Tulsi still faces challenges when trying to communicate with people who don't know sign language. She says
"I wish they knew at least the basics of sign language, so I could talk to them better."
How did we come to this conclusion?
To understand whether there is a lack of awareness about sign language and it's importance and whether there is a struggle to have an effective communication, I and Divya conducted interviews with individuals from the deaf and mute community, assisted by interpreters and trainers and interviewed members of the general population. We also conducted a survey with 50 people from general population.
A GIF of the interview with Deaf and Mute interpreter
A GIF of the interview with Deaf and Mute (Tulsi)
A GIF of the interview with Deaf and Mute (Sameer)
Here are the insights we gathered.
88% of general population acknowledged the importance of communication between the deaf, mute community, and the broader society.
88%
90%
90% of general population believe that learning and utilizing sign language can bridge the communication gap between the two groups
Finally, 72% of general population suggested the broader society should play a more active role in enhancing communication with the deaf and mute community.
72%
What interviewees said and signed!
"I am 100% sure that now the society is open we should be able to include everyone and be inclusive"
"Sign language is very important just like speech is"
“I would like to learn French because my parents speak it and I find it a very useful and fancy language”
“if I go to a new country I can’t communicate with the locals because I don't know the language and not because I am disabled! It is just a language challenge.”
“For deaf their eyes are their ears”
" Sign language is a fully-fledged language with its own grammar, vocabulary, and syntax."
Research and Revelations
We obtained important research findings through interviews with individuals who are deaf and mute, as well as from members of the general public. Additionally, we collected data through an online survey that involved 50 participants. We've created two empathy maps and two user personas for both the deaf and mute community and the general population.
User Persona- Deaf and Mute
Tia is a 26-year-old deaf and mute artist who lives in Toronto. Tia strongly lobbies for and works towards creating awareness about Sign Language.
​
The biggest challenge according to Tia is the misconception of disability, the lack of drive to learn sign language by the broader population, and most importantly a quick way to teach or learn sign language.
User Persona- General Population
Lily.N.Aware, a 28-year-old insurance broker, married and living in Toronto. A workaholic by choice Lily wants to learn French, as she finds it very attractive and appealing and useful.
​Lily doesn't feel the need to learn Sign language as she has no family or friends who are deaf and hard-of-hearing people.
Perceptions and Challenges
Both the deaf and mute community and the general population noted challenges related to the status and awareness of sign language in society, as well as a need for better inclusivity measures. the below given are the perceptions and the challenges faced by both the communities.
Deaf and Mute
There is a lack of awareness in society about the importance of sign language
​There's a strong demand for quick and accessible ways to learn sign language ​
Learning sign language is often perceived differently from learning other languages like French or Korean
General Population
Sign language is undervalued due to a perceived lack of interest
Some express a desire to learn sign language for a deaf relative but are unsure of how to start
There are misconceptions about sign language, and it's not prioritized.
After analyzing the data, our team focused on three important insights that are crucial for both the deaf and mute community and the general population using a priority matrix for our solution.
Find a spot to expose Sign Language to broader society
Present sign language in a fun and attractive way
Create an optional opportunity to propel users to try out Sign Language
Learning from mistakes: sign language exposure
We were unsure about how to introduce sign language to the broader society. My initial idea was to incorporate it into popular platforms like Amazon and Uber Eats, given their large user bases. I considered offering coupons for answering sign language questions while using Amazon. However, from a business perspective, it didn't seem to benefit Amazon much beyond inclusivity.
We tried introducing sign language on Uber Eats, but users didn't respond positively. The long wait times for food orders made it inconvenient, and users weren't interested in learning sign language during that time. Similar results were found when we attempted to introduce sign language in the Uber ride-hailing app. Only one out of five users expressed an interest in learning sign language while using the app in a cab.
"We tried to desperately lump things together in any possible way, that we can"
Sign language and Language Learning Platform?
I got the idea of Language learning platform from the interview Me and Divya did with the deaf and hard of hearing community. They said,
" Sign language is a fully-fledged language with its own grammar, vocabulary, and syntax."
If sign language is included in a language learning platform that teaches various languages, their syntax, and vocabulary, users can study sign language just like any other language.
However, when it comes to people taking the initiative to learn sign language on these platforms, our initial findings from interviews and surveys were resoundingly negative—most individuals expressed little interest in doing so.
​
​
We recognized the need to consider several key factors before moving forward:
-
No compulsion or forceful approach.
2. Ensuring engagement.
3. Maintaining user-friendliness.
4. Providing a fun and enjoyable learning experience.
Why Duolingo?
Our choice to integrate the sign language component, in Duolingo is because Duolingo stands as a frontrunner among language learning apps, boasting widespread popularity and recognition. Its innovative use of gamification is seamlessly aligned with our goal of making sign language learning an enjoyable and engaging experience.
"Join a life changing mission that brings people together"
- Duolingo
Competitor analysis of popular language learning platforms
Presenting Signman "SIMON"
Lets learn Signs with fun!
01
Simon’ is a gamification integration for language learners to easily learn signs online
02
Foster awareness among Duolingo app users about the world of Sign Languages.
03
Engage and interact with Simon and learn the sign language
04
Play with Simon by Guessing the Sign, Earning Gems & Redeeming Hearts.
Integration of "SIMON" the signman
Simon will make its debut as a soft launch within the Duolingo app, positioned strategically at a crucial juncture – when users learning any language of their choice lose a heart. Simon is our solution here, presenting users with an opportunity to earn 'gems' by learning sign language. These gems can be exchanged to regain lost hearts or lives, effectively allowing users to continue their language learning journey.
The user starts to learn the language of their preference
If they make a mistake and loose a heart(lives) out of five
Simon comes in at this point. The user have an option to redeem the lost heart by learning sign language with Simon or they can continue without Simon
simple userflow of simon feature integrated in Duolingo app
userflow of simon feature integrated in Duolingo app
Making of Simon: Wireframes
It's important to respect users' choices and preferences when it comes to learning sign language or any other skill. Soft launching Simon, a sign language learning tool, allows users to opt-in and decide if they want to learn sign language alongside Simon or continue without him.
1. An add-on to onboarding
"We incorporated a fresh icon alongside our existing ones in the splash screen"
-This is the onboarding screen of the existing Duo Lingo app and the red square that you see is our little add-on. We are letting the users know prior to the lessons that they will get to meet Simon during your lessons.
2. Quizzes and Levels
User clicks the start button
The audio lesson starts
The correct words are selected from the options below
User clicks the "check" button to check whether the answer is right or wrong.
"Meeting Simon at this crucial jucture"
-In Duolingo everytime the user makes a mistake will cost them a heart Simon is introduced as a featured pop-up to regain these hearts through learning sign language.
It shows incorrect if the answer is wrong and the user ends up loosing one out of five hearts
3. Learning signs with Simon
An intro to signman simon.
The user begins to learn sign language by clicking "Let's start"
The sign questions and answers are formulated similar to that of the Duolingo patterns so that the users don't feel that they are in an entirely different platform.
The users checks the answers , if they are right they click the continue button and move forward to earn the gems which can earn you more hearts. Or if they are wrong they can learn the same again then earn the gems . Or they have the choice to stop learning with simon and continue with the language they were learning.
"In a nutshell, what we are offering are the interactions with simon that will let you learn the basics of sign language by smoothly integrating with duolingo’s existing learning process.It was thought of as a way to create awareness about sign language which can also be developed to an entire independend course"
Testing and Iteration
We conducted user testing asking participants to learn Spanish. Gathering feedback through multiple rounds, we iteratively improved the wireframes to achieve a seamless user experience.
![2lgq_tjfd_211014 [Converted]-02.png](https://static.wixstatic.com/media/f792cc_bca0915d2e5c47539af2fe23dbb6ef47~mv2.png/v1/fill/w_143,h_52,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/2lgq_tjfd_211014%20%5BConverted%5D-02.png)
01
During the onboarding process we hinted the users that they will get to meet Simon.
-But 3 out of 5 users couldn't relate the gift icon to Simon. Thus we replaced the icon with Simon’s avatar to create familiarity
02
4 /5 users did not understand the purpose of this pop-up which appears when they make a mistake. -So we added the Simon icon to the pop-up and a text “play sign with Simon and earn more hearts” now they know what to expect.
03
4 /5 users did not understand what they could do with the earned gems.
![2lgq_tjfd_211014 [Converted]-02.png](https://static.wixstatic.com/media/f792cc_bca0915d2e5c47539af2fe23dbb6ef47~mv2.png/v1/fill/w_133,h_42,al_c,q_85,usm_0.66_1.00_0.01,enc_avif,quality_auto/2lgq_tjfd_211014%20%5BConverted%5D-02.png){kind=small}
We created an extra interface to let the users know , how they can use the hearts and added a "gain more hearts" button to play with simon again to gain more hearts and thereby learning sign language along side.
Immediate Future Developments
-
Transition of visuals from images to animations.
2. Illustrate hearts redemption process.
3. Align Simon character with Duolingo's branding
4. Present the project to Duolingo team
Key Takeaways
By embedding sign language learning into a familiar and engaging platform, we shifted perspectives from "learning sign language is difficult and unnecessary" to "this is fun and worth exploring." This project laid the foundation for sign language education in a way that encourages inclusivity without being forceful.