Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.

  • Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.

  • AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.

  • Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.

  • Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.

  • Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.

  • Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.

  • Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.

  • Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.

  • Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.

Notable Quotes

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

"Algorithms that adjust care based on race don’t actually help and embed disparities instead."

"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."

"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

"All commercially available facial recognition software perform worse on darker females."

"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."

"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."

"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."

"The best transformation happens by asking the right questions, even when the problem feels overwhelming."

Ask the Rosenbot
Kit Unger
Theme 2: Discussion
2024 • Enterprise Experience 2020
Gold
Maggie Dieringer
Cutting through the Noise
2020 • DesignOps Community
Soma Ghosh
What emerging methods are advancing UX research [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Bria Alexander
Day 3 Welcome
2024 • DesignOps Summit 2024
Gold
Heidi Trost
When AI Becomes the User’s Point Person—and Point of Failure
2025 • Rosenfeld Community
Jemma Ahmed
Collaboration: learning from other fields beyond our own [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Irina Tikhonova
Small Wins, Big Impact: Leveraging and Elevating User Engagement
2021 • Civic Design 2021
Gold
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
Bob Baxley
Theme 4: Intro
2024 • Enterprise Experience 2020
Gold
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold
Jake Burghardt
Stop wasting research: Create new value with insight summaries
2025 • Rosenfeld Community
Sarah Kinkade
Design Management Models in the Face of Transformation
2022 • Design at Scale 2022
Gold
Kim Holt
A Salesforce Panel Discussion on Values-Driven DesignOps
2022 • DesignOps Summit 2022
Gold
Alan Williams
Designing essential financial services for those in need
2022 • Civic Design Community
Bob Baxley
Theme 4: Discussion
2024 • Enterprise Experience 2020
Gold

More Videos

Brad Peters

"Having a well-thought-out data package really helps us see a fuller picture."

Brad Peters Anne Mamaghani

Short Take #1: UX/Product Lessons from Your Industry Peers

December 6, 2022

Lona Moore

"You don’t have to be a designer to participate; over 90% of our community are not design professionals."

Lona Moore

Scaling Design Beyond Designers

June 11, 2021

Josh Clark

"The interface becomes a radically adaptive surface—an intelligent canvas that reacts to your behavior and context."

Josh Clark Veronika Kindred

Sentient Design, AI, and the Radically Adaptive Experience (1st of 3 seminars)

January 15, 2025

Erin May

"People will talk to customers whether you want them to or not. The question is how to make it a better experience."

Erin May Roberta Dombrowski Laura Oxenfeld Brooke Hinton

Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org

March 10, 2022

Tara Tressel

"If I really just needed only quantitative data from a survey, then I would not go the AI moderated route."

Tara Tressel

Investigating qualitative depth of AI-moderated interviews

March 10, 2026

Charles Lee

"Workshops often reveal gaps in documentation, which is a spicy take but true."

Charles Lee Jennie Yip

Building a New Home for the Atlassian Design System

October 22, 2020

Kristin Skinner

"Technology can revolutionize how we think about education."

Kristin Skinner

Five Years of DesignOps

September 29, 2021

Megan Blocker

"Without adoption, value is zero."

Megan Blocker

Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research

March 26, 2024

Shanti Mathew

"Service delivery is led by frontline staff who tend to have the least amount of power in the system."

Shanti Mathew Natalie Sims Natalia Radywyl

Civic Design at Scale: Introducing the Public Policy Layer Cake

December 9, 2021