Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

SUS: A System Unusable for Twenty Percent of the Population
Gold
Thursday, December 9, 2021 • Civic Design 2021
Share the love for this talk
SUS: A System Unusable for Twenty Percent of the Population
Speakers: Sam Proulx
Link:

Summary

Did you know that the SUS creates biases in your research, affecting one in five people? That’s right!  People with disabilities, especially those who use assistive technology, are not considered by most of the questions in the SUS. As a designer for the public sector, this could lead to you making design decisions that don’t take into account 20 percent of the visitors to your website.

Key Insights

  • Screen readers allow users to navigate web pages quickly by jumping through well-structured headings and content.

  • Low vision users often rely on system-wide screen magnification and color inversion settings to reduce eye strain.

  • Alternative navigation methods like head mice and voice dictation are selected based on task and user condition, not used exclusively.

  • Traditional lab-based usability research often prioritizes convenience over real assistive technology configurations, leading to inaccurate results.

  • Assistive technology users rarely use default settings; they extensively customize tools over time for optimal use.

  • The System Usability Scale (SUS) includes questions that confuse assistive technology users and do not capture their true experience.

  • Fable’s Accessible Usability Scale adapts SUS questions to better reflect the experience of assistive technology users.

  • Initial data shows screen reader users report significantly lower usability scores (mid-50s) compared to screen magnification users (low 60s).

  • Users’ lowered expectations after years of experience may mask the true severity of usability issues in assistive technology contexts.

  • Measuring accessibility usability accurately enables benchmarking and progress in improving digital products for all users.

Notable Quotes

"I’ve been using screen readers for 30 years and accessibility has always been a passion of mine."

"Screen readers let us jump from one heading to another, replicating the experience of skimming a page visually."

"Low vision users often invert colors and use magnification levels as high as 400 percent to reduce eye strain."

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

"In a usability session, the biggest challenge is not to talk at the same time the screen reader is talking."

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

"The System Usability Scale’s question about needing a technical person to use the system is confusing and inaccurate for screen reader users."

"Most people with my assistive technology might learn to use the system at a different pace than non-disabled users."

"Screen reader users scored significantly lower in usability tests, reflecting the most effort in interpreting content."

"If you’re doing research with people with disabilities, consider using the Accessible Usability Scale to get more accurate insights."

Ask the Rosenbot
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Dan Willis
Enterprise Storytelling Sessions
2017 • Enterprise Experience 2017
Gold
Indra Klavins
A Design Ops Girl in a Dev Ops World
2019 • DesignOps Summit 2019
Gold
Christian Bason
Expand—Rethinking Design for Public Challenges
2022 • Civic Design Community
Uday Gajendar
From AI to Zeitgeist: Theory as the design antidote to AI hype
2025 • Rosenfeld Community
Sofia Quintero
The Product Philosophy Behind EnjoyHQ
2021 • Advancing Research 2021
Gold
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
George Hinchliffe
Delivering Amazing Experiences
2021 • Design at Scale 2021
Gold
Sara Conklin
A UXer’s 12-Month Journey from Climate Concern to Climate Credibility
2025 • Climate UX Interest Group
Michal Anne Rogondino
Saving Outer Space: The First UX Design System for Our Nation’s Satellites
2024 • Enterprise Experience 2020
Gold
George Abraham
Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)
2021 • DesignOps Summit 2021
Gold
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Brigette Metzler
Research Repositories: A global project by the ResearchOps Community
2020 • Advancing Research Community
Adam Cutler
People + Places + Practices = Outcomes
2016 • Enterprise UX 2016
Gold
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold

More Videos

Bria Alexander

"Please read our code of conduct — even though we’re virtual, we are a community obligated to kindness and respect."

Bria Alexander

Opening Remarks

November 18, 2022

Hugh Dubberly

"Small companies like Descartes Labs built supercomputers entirely in AWS with no physical servers, showing a profound shift in how computing power is accessed."

Hugh Dubberly

Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving

June 19, 2019

April Reagan

"AI can be a great partner to humans if trained with fair and complete data, but we must be careful not to ship biased systems."

April Reagan

Look, Think, Act: The Futures-Smart Design Organization

October 1, 2021

Ricardo Martins

"Customers can answer what the researcher wants to hear in surveys, making attitude only 28% predictive of real behavior."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Maria Skaaden

"Google Design Sprint is the best inception strategy I know for design thinking."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Vanessa Varin

"Feedback is more like an investment and a contribution into not just the person's work, but also to their growth."

Vanessa Varin

Feedback: The Other F-Word

September 10, 2025

Jen Crim

"Half-day Fridays is a way for us to give back to ourselves and promote work-life balance."

Jen Crim Jess Quittner Saritha Kattekola Alex Karr Gurbani Pahwa

Culture, DIBS & Recruiting

June 11, 2021

Patrick Boehler

"Structure beats free form interaction—systematic query frameworks are essential to avoid pitfalls."

Patrick Boehler

Fishing for Real Needs: Reimagining Journalism Needs with AI

June 10, 2025

Julie Baher

"Net Promoter Score is just one question, but it’s a powerful way to measure loyalty and customer experience."

Julie Baher

Culture Change—My Journey

May 14, 2015