Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Designing inclusively with AI
Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
Designing inclusively with AI
Speakers: Kate Kalcevich
Link:

Summary

We know there’s massive opportunity for AI to be used for greater speed, growth, and insights, but how can we avoid the pitfalls? Head of Accessibility Innovation at Fable Kate Kalcevich will share how to best leverage current AI tools in your day-to-day work with inclusion in mind.

Key Insights

  • AI-generated images of people with disabilities often show racial and representation biases, such as all-white subjects and inaccurate assistive device depictions.

  • Disability representation in AI images is frequently inaccurate, e.g., guide dogs shown with leashes instead of harnesses and blindfolds over eyes.

  • AI-generated personas for users with disabilities can contain improbable feature combinations, such as a MacBook user reliant on Windows-only assistive tech like JAWS.

  • Personas created by AI lack the depth and nuance gained from real user research and testing, especially for disability accessibility considerations.

  • AI-generated mockups can have accessibility issues like poor contrast and unlabeled UI elements, requiring human accessibility review.

  • AI tools like Stark Sidekick improve design accessibility through automated issue detection, though they are not yet full replacements for experts.

  • AI-powered task management apps can enhance productivity and accommodation for neurodivergent users by organizing schedules and reducing cognitive load.

  • AI can help identify potential biases in accessibility automation projects, but requires careful prompt crafting and domain expertise.

  • Designing AI-powered interfaces comes with inclusion challenges, as users unfamiliar with prompting may struggle to get effective results.

  • Combining AI output with continuous user testing, especially involving people with disabilities, is essential to ensure accessibility and inclusion.

Notable Quotes

"If you have questions, I will take questions throughout."

"AI-generated images often have no racial diversity and show inaccurate assistive device representation."

"You are either navigating with a white cane or a service dog because you are not gonna want both of your hands full."

"Personas tend to not give you those deep insights that you would get from user research and user testing."

"Automation might not be able to capture the context of an image if you're trying to describe an image."

"AI is actually a pretty good productivity hack as a starting point for accessibility reviews."

"A lot of people will try a prompt and not get what they were hoping to get and maybe miss out on all the benefits of AI."

"For people who identify as autistic, AI tools that review writing tone can be especially helpful."

"If the code doesn’t match the label on screen, voice controls might not recognize the command."

"There always has to be a human option if you’re not getting what you need from the AI."

Ask the Rosenbot
Uday Gajendar
Day 2 Welcome
2024 • Designing with AI 2024
Gold
Farid Sabitov
Automatization for Large Enterprise Teams
2024 • DesignOps Summit 2020
Gold
Natalie Dunbar
DesignOps and Content Strategy: Envisioning the Future Together
2021 • DesignOps Summit 2021
Gold
Gabriela Barneva
Operationalizing Inclusive Design in Design Ops
2025 • DesignOps Summit 2025
Conference
Frances Yllana
D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)
2022 • Civic Design 2022
Gold
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Shazia Ali
Communication: Innovative techniques for making your voice heard [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Laine Riley Prokay
Carving a Path for Early Career DesignOps Practitioners
2022 • DesignOps Summit 2022
Gold
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Gina Mendolia
Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
2024 • Advancing Service Design 2024
Gold
Deanna Mitchell
Designing with culture: Unlocking impactful insights for Product and UX
2025 • Advancing Research 2025
Gold
Jacqui Frey
Flow and Superfluidity for Design Orgs
2018 • DesignOps Summit 2018
Gold
Taylor Klassman
Shaping the Next Era of UX Research: Collaborative Forum
2025 • Advancing Research 2025
Gold
Kit Unger
Theme 1 Intro
2022 • Design at Scale 2022
Gold
Joseph Meersman
Sweating the Pixel: Scaling Quality through Critique
2021 • Design at Scale 2021
Gold

More Videos

Joshua Graves

"Try to observe the situation like a security camera—stick to factual, objective descriptions."

Joshua Graves

We Need To Talk: Addressing Unmet Expectations (Part 2 of 3)

April 28, 2025

Uday Gajendar

"Craft is a pet issue for me; it remains central even in massive enterprise projects."

Uday Gajendar

Theme 1: Introduction

June 9, 2021

Audrey Crane

"The alternative to good design isn’t no design, it’s bad design."

Audrey Crane

Shadow Design–Where Else is Design Happening in Your Organization?

April 20, 2023

Gina Mendolia

"Taking ugly notes together without worrying about making things pretty lowers barriers and increases engagement."

Gina Mendolia

Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems

December 3, 2024

Sheri Byrne-Haber

"Accessibility isn't about dumbing down interfaces; except for color, most guidelines do not require changing the look or behavior of components."

Sheri Byrne-Haber

The Importance of Accessible Design Systems

January 8, 2024

"Southwest Airlines focused on quality of experience and recorded 24 straight years of profitability despite never dominating market share."

How to Identify and Increase your "Experience Quotient"

June 15, 2018

Adel Du Toit

"After swapping our remote meeting solution in early 2020, ESAT went from 69% to 92%, a huge improvement."

Adel Du Toit

Get Your CFO To Say: 'Our Strategic Goal is User Obsession'

June 10, 2022

Kavana Ramesh

"Think of assistive technology in three categories: screen readers, screen magnification, and alternative navigation."

Kavana Ramesh

Meaningful inclusion: Practicing accessibility research with confidence

September 24, 2024

Tricia Wang

"When infrastructure breaks down, it is human infrastructure — communities and organizers — that saves the day."

Tricia Wang

Spatial Collapse: Designing for Emergent Culture

January 8, 2024