Things to consider when doing usability testing with disabled people
Throughout the years, I’ve conducted several usability tests with disabled people. They are not only a great way to evaluate the accessibility of the interface, but they can also help to grow awareness of accessibility within the design and development team.
Seeing (disabled) people struggle to use the website you designed can be a real wake up call, and can help designers come to the conclusion that ”clever” solutions, may not always be so clever after all.
There are plenty of books and articles that help you conduct usability tests correctly, but few go into testing with disabled people. In this article, I share some of the things you need to keep in mind when conducting usability tests with disabled people (and that I sometimes need to remind myself of).
1. Recruiting people is hard
If you think recruiting the right participants for a ”regular” usability test is a lot of work, try finding disabled people for an accessibility test.
You probably already have a target audience for your ”regular” usability test. Still, for this test, you need to narrow it down even further by only testing with people with (specific) disabilities.
They might be harder to find, and even if you’re successful, it might take them too much time and energy to travel to your lab.
Find (online) communities for people with disabilities. Advocacy groups and charities can also be good starting points for finding participants.
Visit participants in places that are more convenient to them. For people with mobility issues, it can be a challenge getting to your lab, so sometimes it makes all the difference when you visit them at home.
Compensate the participants well, especially if you ask them to travel to you. The people you invite to do a test like this, are generally not the most well-off. And they often make a significant effort to come to the test.
Thanks for this suggestion, Dean!
That blind guy in accounting doesn’t count as a good participant. In general, don’t recruit people from inside the company as they have their own needs and workarounds.
2. You’ll need more time for everything
When we’re testing with disabled people, I generally recommend letting them use their own devices. They have their own preferences and their own devices that they need to use. This can be a braille display or their personal colour preferences for Windows High Contrast mode.
This means that setting up will take you a bit longer than your regular ”clear cookies and you’re good to go”. Give your participants some time to set up, about 15 minutes should be fine, but be flexible.
You’ll also need more time before the participants enter the lab, you might need to pick them up from a bus stop or train station. At least account for the extra time it takes them to come to your office.
Testing with four participants on a day is about as much as you’ll be able to do.
Provide the participants with enough assistance when they come to your office. Offer to book them a taxi to pick them up, or pick them up yourself.
User browser-based tools if you want to stream the session to an observation room or if you want to record it. That way, the participants don’t need to install any software on their personal devices.
I like to use Whereby.com (formerly known as appear.in) for this.
3. Only test when you’re ready for it
Before you invite people into your research lab, you should have at least done a decent enough job to make sure the website you’re testing is accessible. Not everything has to be perfect and meeting all the WCAG 2.1 requirements, but you should at least do the necessary checks to make sure you’re not just letting people struggle.
You should be testing the interface, not the participants’ patience.
There are some quick checks you can do to evaluate the accessibility of your website:
4. Not all disabilities are the same
When you want to test the accessibility of your website, don’t just invite ”a bunch of disabled people” (big air quotes here). Don’t play Accessibility Pokemon, where you ”gotta catch ’em all” by inviting one blind person, one deaf person, one person with a physical impairment, and a person with an intellectual disability and expect to have everything covered.
Rather spend one day with several people with visual impairments, you’ll notice soon enough that within that group there are overlapping preferences, but you’ll see just as many differences. They may use completely different software and devices to access your website. A person using VoiceOver on an iPad will have a different experience from someone using a braille display.
When you’ve done that, you can move on to improving it for a different group of people.
That’s not to say you shouldn’t test unless you have a representative group of people. If you have limited time and budget, it’s better to be creative and invite people with different disabilities, than not testing at all. Just don’t expect that the people you test with are a representation of everyone with similar disabilities.
Don’t expect that a single disabled person is a representation of everyone with a similar disability. People have different needs and preferences and the more people you invite, the better.
Invite the right people into the observation room. It should always be a good mix, but when you’re testing with blind people, you’ll catch more of the technical issues, so you want to have more developers in the room. On the other hand, when you’re testing with deaf people, you’ll probably be focusing more on the content, so you want to invite some extra content designers.
5. Check that your lab is accessible
This should probably be at the top of the list, but your lab should be accessible for the participants:
- Check the entrance and the way to the lab for any steps or kerbs
- How wide are the doors? Can a wheelchair fit through? (Also check if they should and can take turns)
- Offer to guide them and ask them how they like to be guided
- Inform people about obstacles (like steps) or things that are on the desk (like a glass of water)
- Also, check the accessibility of nearby train station, and bus- and metro stops. They may need extra assistance there.
6. Make the observation fun
Usability testing should always be a group activity, but testing with disabled people also allows you to grow awareness on accessibility in the team or your organisation.
Many people have never seen people using a screenreader, zoom functionality, or a switch control. Seeing this in action and hearing actual people talking about everyday issues they run into, helps to build empathy. Invite managers and people from other teams to watch in on a session and make it fun. Always provide (healthy) snacks.
Always ask for the participants’ consent before you stream and record the session.
Also, if they use screenreader software, ask them if they can change it to speak a bit slower. That makes everything a bit easier for the people in the observation room.
One last thing
Doing usability testing with disabled people is probably one of the part of my job that excites me most. It’s so rewarding to take accessibility beyond tools and checklists and seeing actual people benefiting from the improvements we made.
It’s definitely worth the (extra) effort.