Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
John Maeda
About Design Organizations
2019 • DesignOps Community
Mariesa Lenz
What Beekeeping Taught me about Product Teams
2025 • Rosenfeld Community
Christian Bason
Expand—Rethinking Design for Public Challenges
2022 • Civic Design Community
Meghan Hellstern
The Next 100 Years of Civic Design: How Might We Better Rise to Meet the Challenges of Today and Tomorrow?
2021 • Civic Design 2021
Gold
Jon Fukuda
Theme One Intro
2022 • DesignOps Summit 2022
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold
Jennifer Bolduc
What's involved with getting people back to work?: A panel discussion
2021 • DesignOps Community
Cennydd Bowles
Day 1 Panel
2024 • Designing with AI 2024
Gold
Dr. Nikki Smith
Research Strategy: Connecting Insights to Outcomes
2025 • Advancing Research 2025
Gold
Peter Van Dijck
Building the Rosenbot
2024 • Designing with AI 2024
Gold
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Amy Jiménez Márquez
The Atypical UX Manager Path
2020 • Enterprise Community
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Mike Davidson
Fireside Chat
2022 • Advancing Research 2022
Gold
Ana Ferreira
Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries
2024 • DesignOps Summit 2020
Gold

More Videos

Doug Powell

"Everything is a prototype."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"If a group session with middle schoolers goes off the rails, one-on-one sessions can save the research."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"Every parent we talked to had some variation of the question: this exists, can I use it right now? Where was this when I needed to apply?"

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"We never compromise on the business goals, but how we achieve them can be flexible and pragmatic."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"We created an AI persona called Dreamweaver to answer real-time team capacity and project questions."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Customers benchmark your company to their last best experience, often outside of your industry."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"Truly listening can push us beyond empathy to respect and reflection."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"Re-platforming is transformative not just for the product but also for the people, teams, and organization."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"I learned to be relentless about prioritization and to hone in on specific details and use cases."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023