I began the research process by developing research questions and goals for the study. These consisted of both business and user goals which helped decide potential tasks for use in the study.
I had the goal of discovering more about learning and education in Duolingo, so I also devised research questions to help guide my research.
There were other features in Duolingo that I considered, like friends and leaderboards. I decided to leave these out as I felt they were not as interesting to study since they are a secondary feature. I consider the main feature of Duolingo to be the language learning, so that is what I focused my tasks on.
One challenge I faced was in coming up with pretest questions. I wanted to ask questions that gave me information on the users background, but also didn’t want to prime them in any way. I decided to ask them about their learning, as I wanted to be able to compare their learning style to their results with Duolingo. I also asked about their prior experience with Duolingo and language learning, since Duolingo is supposed to match with a users skill level.
I had the goal of discovering more about learning and education in Duolingo, so I also devised research questions to help guide my research.
There were other features in Duolingo that I considered, like friends and leaderboards. I decided to leave these out as I felt they were not as interesting to study since they are a secondary feature. I consider the main feature of Duolingo to be the language learning, so that is what I focused my tasks on.
One challenge I faced was in coming up with pretest questions. I wanted to ask questions that gave me information on the users background, but also didn’t want to prime them in any way. I decided to ask them about their learning, as I wanted to be able to compare their learning style to their results with Duolingo. I also asked about their prior experience with Duolingo and language learning, since Duolingo is supposed to match with a users skill level.
Photo from one of my in person tests.
I found that I had a better set up for my usability testing compared to the pilot. A few of my usability tests were in person, and I was able to use a phone mount to better record the reactions of my participants, while in the study, I only looked at audio. I also made an effort to have users find elements of the interface themselves, rather than direction them.
Photo from one of my in person tests.
I found that I had a better set up for my usability testing compared to the pilot. A few of my usability tests were in person, and I was able to use a phone mount to better record the reactions of my participants, while in the study, I only looked at audio. I also made an effort to have users find elements of the interface themselves, rather than direction them.
Photo from one of my virtual tests.
For my virtual tests, I found it helpful to have users join from their phone and then share their screen, which was very helpful in seeing what they clicked. I didn’t get to see as much of their facial expressions or observe them tap the screen, but I think the questions I asked help make up for this.
Photo from one of my virtual tests.
For my virtual tests, I found it helpful to have users join from their phone and then share their screen, which was very helpful in seeing what they clicked. I didn’t get to see as much of their facial expressions or observe them tap the screen, but I think the questions I asked help make up for this.
Graph of Time on Task
I measured Time on Task as the time it took a user to find the button on the interface to start the task. I measured this because there were a lot of extraneous variables for the actual time on task and I consider the time it took to find the button the indicator of how discoverable the feature is.
Graph of Time on Task
I measured Time on Task as the time it took a user to find the button on the interface to start the task. I measured this because there were a lot of extraneous variables for the actual time on task and I consider the time it took to find the button the indicator of how discoverable the feature is.
Class: Library Information Sciences 613 User Experience 3 at UW Madison
Role: UX Researcher
Tools Used: Zoom
Timeline: June - August 2024
Class: Library Information Sciences 613 User Experience 3 at UW Madison
Role: UX Researcher
Tools Used: Zoom
Timeline: June - August 2024
Screenshot of my notes from the pilot study.
To test out my study design, I conducted a pilot study for two participants, who I recruited in person.
In my study, I had users complete three tasks and asked them questions about their experience.
The three tasks were:
Complete a lesson in Duolingo
Go through A Practice to Earn Hearts Practice Session (Go through a Super Duolingo Additional skills practice was a substitute for Super Duolingo Users)
Review a unit guidebook.
One challenge I faced during the study was in my second task, where I had participants select an additional practice mode. However, this practice mode was kind of hard to find, so I had to guide users on where to find it.
I think in hindsight, I should have waited a bit longer and let them explore the app more to find it in order to make the study more authentic.
Both of my participants had full hearts for that task, so I had to figure out a way to make them lose a heart. The solution was for them to go to a different lesson, get an answer wrong, and then go back. I don’t think this impacted the study that much, but it was an interruption that could have impacted the users thought process.
Screenshot of my notes from the pilot study.
To test out my study design, I conducted a pilot study for two participants, who I recruited in person.
In my study, I had users complete three tasks and asked them questions about their experience.
The three tasks were:
Complete a lesson in Duolingo
Go through A Practice to Earn Hearts Practice Session (Go through a Super Duolingo Additional skills practice was a substitute for Super Duolingo Users)
Review a unit guidebook.
One challenge I faced during the study was in my second task, where I had participants select an additional practice mode. However, this practice mode was kind of hard to find, so I had to guide users on where to find it.
I think in hindsight, I should have waited a bit longer and let them explore the app more to find it in order to make the study more authentic.
Both of my participants had full hearts for that task, so I had to figure out a way to make them lose a heart. The solution was for them to go to a different lesson, get an answer wrong, and then go back. I don’t think this impacted the study that much, but it was an interruption that could have impacted the users thought process.
Usability Testing
Results
I compiled the results from my usability study into a report and a presentation.
Results
I compiled the results from my usability study into a report and a presentation.
Takeaways
I found that writing some of the report was tricky, since I had a mix of usability issues and content issues when it came to the app. Because it was a learning app, my users would comment on the types of tasks they found, such as writing or speaking.
Some of the metrics were harder to decide. I discussed it with some of my peers, and opted to use numerical metrics for UI usability, and use the comments I received from my users to speak more on the learning aspect.
I wanted to know more about the long term learning that users might experience in Duolingo, but was not able to achieve that result since this was a short term study.
Takeaways
I found that writing some of the report was tricky, since I had a mix of usability issues and content issues when it came to the app. Because it was a learning app, my users would comment on the types of tasks they found, such as writing or speaking.
Some of the metrics were harder to decide. I discussed it with some of my peers, and opted to use numerical metrics for UI usability, and use the comments I received from my users to speak more on the learning aspect.
I wanted to know more about the long term learning that users might experience in Duolingo, but was not able to achieve that result since this was a short term study.
Graph of SEQ score.
For each task, I asked users to rate how difficult they found the task. This is pictured in the graph above. Task 2 was found to be the most difficult as users encountered modes of learning that were different from the usual daily lesson such as typing or pictures.
Graph of SEQ score.
For each task, I asked users to rate how difficult they found the task. This is pictured in the graph above. Task 2 was found to be the most difficult as users encountered modes of learning that were different from the usual daily lesson such as typing or pictures.
Objectives
Overview
What I did
As part of my LIS 613 User Experience 3 class at UW Madison, I conducted usability testing on Duolingo, a gamified language learning application where users complete lessons on a chosen language.
Class: Library Information Sciences 613 User Experience 3 at UW Madison
Role: UX Researcher
Tools Used: Zoom
Timeline: June - August 2024
As part of my LIS 613 User Experience 3 class at UW Madison, I conducted usability testing on Duolingo, a gamified language learning application where users complete lessons on a chosen language.
Class: Library Information Sciences 613 User Experience 3 at UW Madison
Role: UX Researcher
Tools Used: Zoom
Timeline: June - August 2024
Develop research and business goals
I compiled the usability key issues in a table, noting their severity and incidence rate.
I compiled the usability key issues in a table, noting their severity and incidence rate.
I compiled the usability key issues in a table, noting their severity and incidence rate.
User Experience Research
Duolingo Usability Study
As part of my LIS 613 User Experience 3 class at UW Madison, I conducted usability testing on Duolingo, a gamified language learning application where users complete lessons on a chosen language.
As part of my LIS 613 User Experience 3 class at UW Madison, I conducted usability testing on Duolingo, a gamified language learning application where users complete lessons on a chosen language.
I began the research process by developing research questions and goals for the study. These consisted of both business and user goals which helped decide potential tasks for use in the study.
I had the goal of discovering more about learning and education in Duolingo, so I also devised research questions to help guide my research.
There were other features in Duolingo that I considered, like friends and leaderboards. I decided to leave these out as I felt they were not as interesting to study since they are a secondary feature. I consider the main feature of Duolingo to be the language learning, so that is what I focused my tasks on.
One challenge I faced was in coming up with pretest questions. I wanted to ask questions that gave me information on the users background, but also didn’t want to prime them in any way. I decided to ask them about their learning, as I wanted to be able to compare their learning style to their results with Duolingo. I also asked about their prior experience with Duolingo and language learning, since Duolingo is supposed to match with a users skill level.
Takeaways
I found that writing some of the report was tricky, since I had a mix of usability issues and content issues when it came to the app. Because it was a learning app, my users would comment on the types of tasks they found, such as writing or speaking.
Some of the metrics were harder to decide. I discussed it with some of my peers, and opted to use numerical metrics for UI usability, and use the comments I received from my users to speak more on the learning aspect.
I wanted to know more about the long term learning that users might experience in Duolingo, but was not able to achieve that result since this was a short term study.
Graph of SEQ score.
For each task, I asked users to rate how difficult they found the task. This is pictured in the graph above. Task 2 was found to be the most difficult as users encountered modes of learning that were different from the usual daily lesson such as typing or pictures.
Results
I compiled the results from my usability study into a report and a presentation.
Screenshot of my notes from the pilot study.
To test out my study design, I conducted a pilot study for two participants, who I recruited in person.
In my study, I had users complete three tasks and asked them questions about their experience.
The three tasks were:
Complete a lesson in Duolingo
Go through A Practice to Earn Hearts Practice Session (Go through a Super Duolingo Additional skills practice was a substitute for Super Duolingo Users)
Review a unit guidebook.
One challenge I faced during the study was in my second task, where I had participants select an additional practice mode. However, this practice mode was kind of hard to find, so I had to guide users on where to find it.
I think in hindsight, I should have waited a bit longer and let them explore the app more to find it in order to make the study more authentic.
Both of my participants had full hearts for that task, so I had to figure out a way to make them lose a heart. The solution was for them to go to a different lesson, get an answer wrong, and then go back. I don’t think this impacted the study that much, but it was an interruption that could have impacted the users thought process.