Published by Peter McNally, Rachel Graham, and Jocelyn Bellas

Over the last few years, we ran several usability studies with participants with various disabilities (visual, cognitive, and motor).  We thought it would help others if we shared some of the main lessons learned when testing with individuals with disabilities.

Figure 1: Windows 10 high contrast mode of Google.com

Background

Several of our clients from a state government agency to a fortune 100 company came to us at the User Experience Center (UXC) for help with their websites. They wanted to make sure users with disabilities could access their site and accomplish their goals.  Typically, companies aim for compliance with accessibility guidelines such as the Web Content Accessibility Guidelines (WCAG 2.0).  While this is critical, it is also important to have users with disabilities try to accomplish real tasks on the site in usability testing. There may be gaps in the overall user experience, even if a site follows the accessibility guidelines and is technically accessible, users may not be able to accomplish their goals on the site.  

Lessons Learned

In most ways, usability testing with this population is no different than testing with anyone else.  However, there are several areas you need to pay just a bit more attention to so your sessions run smoothly.

1. BRING THEIR OWN EQUIPMENT: Allow and encourage participants to bring their own equipment. This way, you can truly see how people use assistive technology. This is a must for people using voice recognition software such as Dragon Naturally Speaking because they have trained the software to recognize their voice.
2. ALLOW ADDITIONAL TIME: Provide additional time in-between sessions.  Typically we like to reserve 30 minutes between participants.  However, when participants plan to bring in their own equipment additional time may be required for set up and resolving any issues that may arise.  When testing with individuals with disabilities we schedule an hour between sessions so we have extra time for setting up assistive technology and testing it.
3. CONFIRM PARTICIPANT NEEDS: Confirm what equipment participants will bring in and/or need supplied beforehand.  In our lab, we can connect external laptops (that in this case, were outfitted with special accessibility software and settings) to our 1Beyond system via a HDMI cable.  In a recent study, all of our participants’ laptops had HDMI ports. However, we forgot to check this beforehand. This is an example of a small but important thing to check to prevent show-stopping issues at the time of the test.
4. CONSIDER ADDITIONAL COST: Consider the cost of transportation in the incentive amount.  If feasible, consider providing an extra $25-$40 in your incentive amount so participants can take a taxi to and from your location.  Depending on access to public transportation and taxi rates in your area the amount may vary. Our participants came to the UXC in different ways – some more reliable and timely than others.
5. REVISE DIRECTIONS: Check the directions you provide for accessibility.  Make sure they include an accessible path into your building. Test them out beforehand. Do you need to provide additional signage?  If so, ensure all signs are clear, concise, and use plain-language directions. Will visually impaired participants be able to navigate? You may need to meet them outside and help guide them to the lab or testing location.
6. CONSIDER LOGISTICS: Consider remote usability testing as option. One of the benefits of bringing individuals with disabilities into the lab for usability testing is observing first hand participant’s use of the product or web site in question.  However, the logistics of getting to your location may be just too much for participants. If it’s possible to test remotely (we typically do this through GoToMeeting) it should be considered. This poses the additional challenge of making sure your process for capturing the remote session is compatible with all of the participant’s assistive technology, as well as accessible itself. Troubleshooting remotely is never fun, and could be more difficult with this population.
7. PRIOR TO THE STUDY: Send out the consent forms via email beforehand.  If you don’t, be prepared to read aloud the consent form and/or sign documents for some participants if necessary.

Figure 2: Amazon.com home page zoomed in at 250%.

Additional Insights

If you or your company have conducted user or customer research, you know the value of gathering feedback about the issues and benefits of products and systems. Testing with individuals with disabilities is no different, as you learn many insights that you would not gain otherwise.  However, an additional take-away for us was the realization that people use assistive technologies in different ways. 

An assumption might be someone that is blind only uses a screen reader such as JAWS and is an expert at it.  We found that people with visual impairments actually differ greatly in the level of support needed from assistive technology.  

• Some users need a screen reader for accessing all content.
• Some users with low vision only need to enlarge content or invert page colors to increase contrast.
• Others may need both. One visually impaired participant used both a screen reader along with the zoom function embedded in the web browser.She only used a screen reader for large paragraphs of text, but otherwise simply zoomed in with the web browser and got very close to the screen when navigating around the web site.

Furthermore, just like anyone, all users are not experts on the software they use.  While some users would consider themselves experts, some only learn enough about the software to accomplish what they need and no more.

If you like to discuss design and accessibility, the value of including people with disabilities into the design process, or usability testing please contact the UXC at uxc@bentley.edu

Peter McNally | LINKEDIN

A Senior Usability Consultant at the User Experience Center, I have more than 20 years’ experience in usability, information architecture, accessibility, and software engineering. My current areas of interests include the intersection of user experience (UX) and customer experience (CX), universal design, and internet of things (IoT). 

Regardless of the area or technology, I always focus on the human at the end of the product, so they can have the most seamless and engaging experience.  I have worked with users and designed digital experiences in the financial services, healthcare, government, education, energy, manufacturing, defense, and electronics industries.

Prior to joining the center, I was a Principal Consultant in the CSC User Experience Practice, Senior User Experience Architect at HP Enterprise Services, Research Officer at University of Hertfordshire in the UK, and Senior Software Engineer at 3M Health Information Systems.  I hold a Bachelor of Science in Computer Science from Northeastern University and a Master of Science in Computer Science from Rensselaer Polytechnic Institute (RPI) and am a member of the International User Experience Professionals' Association (UXPA), ACM CHI, Boston UXPA, and Boston CHI.

 RACHEL GRAHAM | LINKEDIN

I’m a research associate at the User Experience Center. I have a deep knowledge of the higher education market with a strong focus on digital and content marketing strategy using platforms like Drupal, WordPress and HubSpot. I’ve also spent time as both Project and Account Manager at an interactive agency in Boston, where I first became passionate about website design and user experience. My B.S. in Marketing and Media Arts is from Bentley University, where I also minored in Business Economics, spent three years running Cross Country and Track, and studied Film & Media in South Africa for a semester. I’m currently President of the User Experience Graduate Association and pursuing a Master of Science in Human Factors in Information Design. You can contact me at graham_rach@bentley.edu. 

JOCELYN BELLAS

I'm a Research Associate in the User Experience Center. Prior to joining the UXC, I worked in a variety of customer service roles at Disney, Florida State University, and John Wiley & Sons. My B.S. in Communication and Social Science is from Florida State University, and I have a certificate in Emergency Management, and a minor in Business. I am currently pursuing a Master of Science in Human Factors in Information Design at Bentley University, and I am involved in the User Experience Graduate Association and UXPA Boston. You can contact me at bellas_joce@bentley.edu. 

BACK TO ALL UXC BLOGS

Connect with the User Experience Center

Get in touch to learn more about Bentley UX consulting services and how we can help your organization.