Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Nancy Douyon
We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex
2018 • Enterprise Experience 2018
Gold
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Steve Portigal
Looking Back…to Look Ahead
2024 • Advancing Research 2024
Gold
Chris Geison
What's Next for Research?
2021 • Advancing Research Community
David Conrad
The Feeling of Data
2023 • Enterprise Community
Audrey Crane
Shadow Design–Where Else is Design Happening in Your Organization?
2023 • Enterprise Community
Aiyana Bodi
Three Key Climate Initiatives and How You Can Help
2024 • Climate UX Interest Group
Ted Booth
Discussion
2016 • Enterprise UX 2016
Gold
Erin Malone
Understanding the past to prepare for the future
2024 • Rosenfeld Community
Michelle Morrison
Culture Design
2020 • DesignOps Community
Scott Plewes
Why Isn't Your UX Approach Going Viral?: A Mathematical Model
2023 • Advancing Research 2023
Gold
Carol Massa
Designing Health: Integrating Service Design, Technology, and Strategy to Transform Patient and Clinician Experiences
2024 • Advancing Service Design 2024
Gold
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Uday Gajendar
Theme 1: Introduction
2021 • Design at Scale 2021
Gold
Uday Gajendar
From AI to Zeitgeist: Theory as the design antidote to AI hype
2025 • Rosenfeld Community
Doug Powell
DesignOps and the Next Frontier: Leading Through Unpredictable Change
2025 • DesignOps Summit 2025
Conference

More Videos

Sam Proulx

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

Sam Proulx

SUS: A System Unusable for Twenty Percent of the Population

December 9, 2021

Michael Land

"In government, the org chart really matters – it influences who talks to whom and who gets to interface."

Michael Land

Establishing Design Operations in Government

February 18, 2021

Shipra Kayan

"Our one metric that mattered was maximizing the number of VOС-tagged tickets solved – showing customer feedback made it to the roadmap."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Ian Swinson

"People care about titles way too much. I had a guy whose business card said star-bellied sneak—love that."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Isaac Heyveld

"The chief of staff serves as a bridge between the design program managers, executive assistants, and other ux and product chiefs of staff."

Isaac Heyveld

Expand DesignOps Leadership as a Chief of Staff

September 8, 2022

Amy Evans

"Self-service was a model with the most unknowns but possibly the biggest rewards."

Amy Evans

How to Create Change

September 25, 2024

Kate Koch

"Teleportation lets us move throughout the organization by building tools that gather disparate data for real-time decision making."

Kate Koch Prateek Kalli

Flex Your Super Powers: When a Design Ops Team Scales to Power CX

September 30, 2021

Dave Gray

"The only way to get outside your bubble is to act as if an alternative belief is true and test it."

Dave Gray

Liminal Thinking: Sense-making for systems in large organizations

May 14, 2015

Matt Duignan

"We wanted a living network of knowledge so yesterday’s insight connects to today’s evidence and tomorrow’s learning."

Matt Duignan

Atomizing Research: Trend or Trap

March 30, 2020