Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI
Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Jemma Ahmed
Redefining the research toolkit: Expanding methodologies for a changing world
2025 • Advancing Research 2025
Gold
Sylvie Abookire
A Civic Designer's Guide to Mindful Conflict Navigation
2022 • Civic Design 2022
Gold
Frances Yllana
DesignOps Exposed: What do our peers really think of us?
2025 • DesignOps Summit 2025
Gold
Alla Weinberg
Healing Toxic Stress
2024 • DesignOps Summit 2024
Gold
John Cutler
The Alignment Trap
2023 • Design in Product 2023
Gold
Savannah Carlin
Don't botch the bot: Designing interactions for AI
2024 • Designing with AI 2024
Gold
Alla Weinberg
Design Teams Need Psychological Safety: Here’s How to Create It
2022 • DesignOps Summit 2022
Gold
Ron Bronson
Design, Consequences & Everyday Life
2022 • Civic Design 2022
Gold
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Suzan Bednarz
AccessibilityOps for All
2024 • DesignOps Summit 2020
Gold
Greg Petroff
Design is the Differentiator: Bringing New Design Innovations to a Very Antiquated and Very Large Industry
2021 • Design at Scale 2021
Gold
Lona Moore
Scaling Design Beyond Designers
2021 • Design at Scale 2021
Gold
Chris Moses
Stretching the Definition of DesignOps with Product Development
2018 • DesignOps Summit 2018
Gold
Nalini P. Kotamraju
Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"
2021 • Advancing Research 2021
Gold
Alana Washington
Theme 1: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Enterprise UX 2023
Gold

More Videos

Nalini Kotamraju

"For the first time, I was able to advocate for funding directly for the research function at high executive levels."

Nalini Kotamraju

Research After UX

March 25, 2024

Dean Broadley

"Society grows great when old people plant trees they know they shall never sit in."

Dean Broadley

Not Black Enough to be White

January 8, 2024

Denise Jacobs

"When we start changing our behavior, we start changing the voices and behavior around us."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Kim Fellman Cohen

"People problems find their way to design ops because we’re known problem solvers and people run to us when morale is low."

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

George Aye

"Why do designers decide how to design this? How do we wean design's addiction from whiteness?"

George Aye

That Quiet Little Voice: When Design and Ethics Collide

November 16, 2022

Nathan Curtis

"Don’t just think about design systems as an artifact but as a living ecosystem connecting people, tools, and products."

Nathan Curtis

Beyond the Toolkit: Spreading a System Across People & Products

June 9, 2016

Greg Petroff

"Our researchers talk regularly to the CEO and are closely involved in product decisions, making research highly impactful here."

Greg Petroff

The Compass Mission

March 10, 2021

Chloe Amos-Edkins

"Remote research methods can cast a wider net, but some briefs still require on-the-ground presence."

Chloe Amos-Edkins

A Cultural Approach: Research in the Context of Glocalisation

March 27, 2023

Mackenzie Cockram

"User journeys are the bedrock for deciding which data to collect and measure."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022