Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Jemma Ahmed
Theme Panel
2025 • Advancing Research 2025
Gold
Jill Fruchter
Inconvenient Insights: The Researcher's Role is to Stay Curious
2023 • Advancing Research 2023
Gold
Jaime Creixems
Best Practices when Creating and Maintaining a Design System
2023 • Enterprise UX 2023
Gold
Bria Alexander
Opening Remarks
2022 • DesignOps Summit 2022
Gold
Peter Van Dijck
Designing AI-first products on top of a rapidly evolving technology
2025 • Designing with AI 2025
Gold
Kate Kalcevich
Designing inclusively with AI
2024 • Designing with AI 2024
Gold
Chelsey Glasson
Exit Interview #3: Same as It Ever Was: What Leaving Tech Taught Me About Change
2025 • Rosenfeld Community
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
Peter Merholz
Customer-Centered Design Organizations
2017 • Enterprise Experience 2017
Gold
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Samuel Proulx
Designing beyond caricatures: Embracing real, diverse user needs
2024 • Advancing Service Design 2024
Gold
Nora Tejeda
Scaling Design Capabilities at BBVA Through a Self-service Design Model
2021 • Design at Scale 2021
Gold
Kat Vellos
Opener: The Other L Word
2024 • DesignOps Summit 2020
Gold
Marjorie Stainback
What Research Ops Professionals Have Learned from COVID-19
2020 • Advancing Research Community
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • DesignOps Summit 2022
Gold
Jake Burghardt
Finding More Inroads into Research Impact
2026 • Rosenfeld Community

More Videos

Bria Alexander

"Practical ops is about getting stuff done because it’s our job to scale the organizational ability to design and amplify the work of designers."

Bria Alexander

Theme Two Intro

October 3, 2023

Ariba Jahan

"Check-outs and reflections normalized pausing and being honest about what we don’t know."

Ariba Jahan

Team Resiliency Through a Pandemic

January 8, 2024

Cornelius Rachieru

"We tend to put the end user first, but we must also ask who we might be doing harm to beyond primary users."

Cornelius Rachieru

Handling Complexity: Framing a Scale of Design

June 9, 2021

Megan Clegg

"Compliance is not enough; we need to go beyond minimum legal requirements for true accessibility."

Megan Clegg Michael Haggerty-Villa Alexis Morin

Space for Everyone: Reframing Accessibility Through a Wider Lens

June 10, 2021

Saara Kamppari-Miller

"Who works here anyways? I knew about immediate teams but beyond that, it was a fog."

Saara Kamppari-Miller

Cartography for Design Communities

September 10, 2025

Kara Kane

"Legislation like the US executive order on federal customer experience has helped us rebuild trust in government."

Kara Kane

Theme One Intro

November 16, 2022

Natalia Radywyl

"Nearly a third of the 65 mental health commission recommendations call for co-design, which is pretty exciting."

Natalia Radywyl

Co-Designing New Power in Australia's Public Sector

November 16, 2022

Nancy Douyon

"Soap dispensers and water sensors often fail on darker skin because they rely on light reflection."

Nancy Douyon

We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex

June 15, 2018

Peter Van Dijck

"Co-creation between users and technology determines if this AI evolves to be useful or harmful."

Peter Van Dijck

Building the Rosenbot

June 4, 2024