Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Sheryl Cababa
Living in the Clouds: Adopting a Systems Thinking Mindset
2023 • Enterprise UX 2023
Gold
Jennifer Strickland
Fireside Chat: How Design Addresses a World on Fire
2022 • Civic Design Community
Tony Turner
Capturing Deep Insights
2021 • DesignOps Summit 2021
Gold
Jeff Gothelf
The Intersection of Lean and Design
2019 • Enterprise Community
Frances Yllana
DesignOps Exposed: What do our peers really think of us?
2025 • DesignOps Summit 2025
Conference
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold
Cornelius Rachieru
Handling Complexity: Framing a Scale of Design
2021 • Design at Scale 2021
Gold
Russell Blair
Killing the blank page
2024 • Designing with AI 2024
Gold
Indi Young
Paying Better Attention to the Problem with Indi Young
2019 • Advancing Research Community
Leah Buley
Closing Plenary: The Crisis of Digital
2020 • Advancing Research 2020
Gold
Zariah Cameron
Streamlining an Inclusive Design Practice
2023 • DesignOps Summit 2023
Gold
Jorge Arango
Scale Smart: AI-Powered Content Organization Strategies
2024 • DesignOps Summit 2024
Gold
Elizabeth Churchill
Exploring Cadence: You, Your Team, and Your Enterprise
2017 • Enterprise Experience 2017
Gold
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Kate Towsey
Ask Me Anything (AMA) with Kate Towsey
2025 • Rosenfeld Community

More Videos

Darian Davis

"Taking responsibility for our behaviors starts with an apology and continues with regular feedback and action."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Aaron Stienstra

"Iterative engagement models mean we test, learn, and adapt rather than seeking a perfect solution from the start."

Aaron Stienstra Lashanda Hodge

Leveraging Civic Design to Advance Equity and Rebuild Trust in the US Federal Government

December 8, 2021

Kavana Ramesh

"Assistive technology can feel like another person in the room during a research session, which you have to understand and include."

Kavana Ramesh

Meaningful inclusion: Practicing accessibility research with confidence

September 24, 2024

Helen Armstrong

"Building skepticism into users is essential because if you’re not skeptical as a designer, it’s hard to build it into your customers."

Helen Armstrong

Augment the Human. Interrogate the System.

June 7, 2023

Scott Plewes

"More math is coming our way into research into UX."

Scott Plewes

Why Isn't Your UX Approach Going Viral?: A Mathematical Model

March 28, 2023

Christian Bason

"Design thinking is really just a set of methodologies; there’s really no thinking in design thinking."

Christian Bason

Expand—Rethinking Design for Public Challenges

September 14, 2022

Peter Van Dijck

"Context design and context engineering mean figuring out and building what needs to go into that text file sent to the model."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!

July 30, 2025

Soma Ghosh

"AI doesn’t have a body or sensory experience, so it can’t replace human contextual understanding in research."

Soma Ghosh Savina Hawkins Dave Hoffer Rob Thomas Victor Udoewa

What emerging methods are advancing UX research [Advancing Research Community Workshop Series]

September 28, 2023

Steve Portigal

"You have to deliberately quiet that noise in your head and focus on humans in a world of humans."

Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke II

War Stories LIVE! Q&A-Discussion

March 30, 2020