Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Jack Moffett
UX Metrics That Matter and The Future of our Design at Scale Conference: A Community Conversation
2022 • Enterprise Community
Sarah Flamion
Complex Problem? Add Clarity by Combining Research and Systems Thinking
2020 • Advancing Research 2020
Gold
Luca Rager
Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch
2021 • DesignOps Summit 2021
Gold
Jayne Engle
Civic Design for the Next Seven Generations—A Discussion on Sacred Civics
2022 • Civic Design Community
Taylor Jennings
Repository Retrospective: Learnings from Introducing a Central Place for UX Research
2022 • Advancing Research 2022
Gold
Bria Alexander
Welcome
2022 • DesignOps Summit 2022
Gold
Alnie Figueroa
The Future of Design Operations: Transforming Our Craft
2025 • DesignOps Summit 2025
Gold
Chris Hammond
Embedding sustainability into enterprise design and development: A journey towards "sustainability consciousness"
2025 • Climate UX Interest Group
Erin Weigel
Failure Friday #6: 90% of Everything I Do is Either Broken or Pointless
2025 • Rosenfeld Community
Neema Mahdavi
Operationalizing DesignOps
2018 • DesignOps Summit 2018
Gold
Jen Briselli
Learning is the north star: service design for adaptive capacity
2025 • Advancing Service Design 2025
Gold
Sarah Brooks
Theme Three Intro
2022 • Civic Design 2022
Gold
Jemma Ahmed
Theme 2 Intro
2024 • Enterprise Experience 2020
Gold
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • DesignOps Summit 2023
Gold
Victor Udoewa
Research in the Pluriverse
2023 • Advancing Research 2023
Gold

More Videos

Ben Reason

"Service blueprint is a knowledge graph – it gives AI intent and helps AI be more useful."

Ben Reason Aline Horta Majid Iqbal Fabiano Leoni

Making the system visible: The fastest path to better decisions

November 20, 2025

Sam Proulx

"Two-factor authentication accessibility improves when there are customizable options like text, email, or QR codes."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

March 10, 2022

Anna Nguyen

"If you have a sense of ownership in the work, you naturally feel more engaged with the outcome and output."

Anna Nguyen Emily Brogan

Why Our Voice of the Customer is Better Than Yours

March 10, 2022

Abby Covert

"A lot of these tools bring more noise into people’s day to day lives. It’s really hard to stay involved and still find time to get your own job done."

Abby Covert Tomer Sharon

Panel: Collaboration Tools

November 6, 2017

Phil Hesketh

"Fix It Fridays are a dedicated time for the team to fix accessibility issues and make steady progress."

Phil Hesketh

Designing Accessible Research Workflows

September 29, 2021

Irina Tikhonova

"If you fix the problems for marginalized communities, you really fix them for everyone."

Irina Tikhonova Kari Dietrich

Small Wins, Big Impact: Leveraging and Elevating User Engagement

December 9, 2021

Tim Frick

"Sustainability isn’t just about emissions or ecological destruction; it includes social and economic dimensions."

Tim Frick

The journey of building a sustainable design practice

April 23, 2025

Catherine Dubut

"The store mobile device is the indispensable silent partner enabling store employees to balance task productivity and customer engagement."

Catherine Dubut

Bridging Physical and Digital Spaces: Approaches to Retail Service Design

March 18, 2021

Justin Entzminger

"Embedding community members to lead design sessions helps shift power dynamics and surfaces authentic voices."

Justin Entzminger Terrance Smith Tracy M. Colunga Mai-Ling Garcia

Risk and Reward: How to Diversify the Field of Civic Innovators and Designers

November 17, 2022