Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI

Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Kate Towsey
Participant Recruitment and Management Tools
2026 • Advancing Research 2026
Gold
Snehal Pendharkar
Conducting pre-research with AI agent personas: Pressure-testing concepts for expert workflows
2026 • Designing with AI 2026
Conference
John Calhoun
Two Sides of the DesignOps Coin: Teams Ops and Product Ops
2024 • DesignOps Summit 2020
Gold
Heidi Trost
When AI Becomes the User’s Point Person—and Point of Failure
2025 • Rosenfeld Community
Soma Ghosh
What emerging methods are advancing UX research [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Jen Cardello
Standardizing Product Merits for Leaders, Designers, and Everyone
2018 • Enterprise Experience 2018
Gold
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold
Lisa Spitz
Building Trust Through Equitable Research Practices
2022 • Civic Design 2022
Gold
Randolph Duke II
War Stories LIVE! Randy Duke II
2020 • Advancing Research 2020
Gold
Uday Gajendar
From AI to Zeitgeist: Theory as the design antidote to AI hype
2025 • Rosenfeld Community
Dave Gray
Connection, Community, and the Future of Work
2026 • Rosenfeld Community
Josh Clark
Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)
2025 • Rosenfeld Community
Brigette Metzler
Scaling ResearchOps: Helping Researchers do Their Best Work
2020 • Advancing Research 2020
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Sam Proulx
Mobile Accessibility and You
2022 • Design at Scale 2022
Gold
Changying (Z) Zheng
Navigating Innovation with Integrity
2024 • DesignOps Summit 2024
Gold

More Videos

Yunyan Li

"Love principle is about emotionally supporting users, celebrating achievements, and creating visually beautiful products."

Yunyan Li Anna Le Jen Kim

UX Best Practices

June 11, 2021

Allan Lowson

"I felt this was a slippery slope—the blurring of lines between roles and responsibilities."

Allan Lowson

Rehashing the Double Diamond: Collaborating across functions with AI-assisted prototyping

June 9, 2026

Denise Jacobs

"It can be so tiring to be the only one constantly saying this has to be done differently."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Renee Bouwens

"Analytics and research should work closely to explain what is happening and why it is."

Renee Bouwens

Landing Product Impact: Aligning Research as a Foundational Driver for Delivering the World’s Best Products

December 15, 2023

Jemma Ahmed

"Junk in, junk out. Bad data is worse than no data at all."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Sam Ladner

"You have to be careful not to be too weird; mental models differ internationally."

Sam Ladner

How Research Can Drive Strategic Foresight

March 9, 2022

Yoel Sumitro

"Designers adhere to standards not scientific ones, but design rigor tailored to dealing with complexity."

Yoel Sumitro

Actions and Reflections: Bridging the Skills Gap among Researchers

March 9, 2022

Dave Hora

"Monitoring for weak signals and early signs of emergence is as important as intervention itself."

Dave Hora

Research in the Face of Complexity: New Sensibility for New Situations

August 27, 2025

Daniel J. Rosenberg

"If you don’t trust it, then there’s nothing here."

Daniel J. Rosenberg

Designing with and for Artificial Intelligence

August 11, 2022