Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
Speakers: Dr. Jamika D. Burge
Link:

Summary

AI-based modeling plays an ever-increasing role in our daily lives. Our personal data and preferences are being used to build models that serve up anything from movie recommendations to medical services with a precision designed to make life decisions easier. But, what happens when those models are not precise enough or are unusable because of incomplete data, incorrect interpretations, or questionable ethics? What's more, how do we reconcile the dynamic and iterative nature of AI models (which are developed by a few) with the expansive nature of the human experience (which is defined by many) so that we optimize and scale UX for AI applications equitably? This talk will push our understanding of the sociotechnical systems that surround AI technologies, including those developing the models (i.e., model creators) and those for whom the models are created (i.e., model customers). What considerations should we keep top of mind? How might the research practice evolve to help us critique and deliver inclusive and responsible AI systems?

Key Insights

  • AI systems in healthcare often embed racial biases that exacerbate disparities, particularly impacting Black and Hispanic populations.

  • Algorithms based on past healthcare spending can unfairly dictate future levels of care, disadvantaging poorer patients.

  • AI hiring tools can perpetuate bias by training on historically discriminatory data, reducing trust among candidates.

  • Dark patterns in UI design, when combined with AI, may manipulate users into unintended decisions, especially vulnerable groups.

  • Deepfakes, created via AI neural networks, pose serious risks of misinformation, reputational harm, and psychological damage.

  • Facial recognition software shows significant racial and gender bias, performing worst on darker-skinned females as shown by Joy Buolamwini.

  • Biases in AI extend beyond technical factors to include systemic and human biases embedded in organizations and data collection.

  • Ethical failures like the Tuskegee Syphilis Study highlight the urgency of integrating ethics and do-no-harm principles into AI research and design.

  • Women and minorities remain vastly underrepresented in technical AI roles, limiting the diversity of perspectives in AI development.

  • Emerging regulations like GDPR and CPRA hold organizations accountable for ethical data use, reinforcing the responsibility of AI practitioners.

Notable Quotes

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

"Algorithms that adjust care based on race don’t actually help and embed disparities instead."

"Dark patterns are not new in UX, but AI amplifies their impact by deceiving users in new ways."

"Deepfakes can disrupt democratic elections and inflict lasting psychological harm on victims."

"Biases out of sight are biases out of mind — facial recognition misidentifies Black faces leading to false arrests."

"All commercially available facial recognition software perform worse on darker females."

"Doing no harm is a core tenet of user experience research, whether or not you are trained in IRB processes."

"Only 12% of AI researchers globally are women, and 6% of professional software developers are women of color."

"Policy like the California Privacy Rights Act requires us to explain data use and comply with user deletion requests."

"The best transformation happens by asking the right questions, even when the problem feels overwhelming."

Ask the Rosenbot
Kate Towsey
The State of ResearchOps: More Than Just Theory
2019 • DesignOps Community
Sarah Kinkade
Design Management Models in the Face of Transformation
2022 • Design at Scale 2022
Gold
Angelos Arnis
Navigating the Rapid Shifts in Tech's Turbulent Terrain
2023 • DesignOps Summit 2023
Gold
Francesca Barrientos, PhD
You Need Your Own Definition of Design Maturity
2022 • Design at Scale 2022
Gold
Ruzanna Rozman
Getting in Flow with Your Team
2024 • DesignOps Summit 2020
Gold
Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
Benjamin Real
Maturity Models: A Core Tool for Creating a DesignOps Strategy
2021 • DesignOps Summit 2021
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Jason Mesut
Shaping design, designers and teams
2018 • DesignOps Summit 2018
Gold
Sheryl Cababa
Integrating Systems Thinking Into Your Practice as a Designer
2025 • Rosenfeld Community
Veevi Rosenstein
Building for Scale: Creating the Zendesk UX Research Practice
2024 • Enterprise Experience 2020
Gold
Samuel Proulx
Designing beyond caricatures: Embracing real, diverse user needs
2024 • Advancing Service Design 2024
Gold
Carla Casariego
DesignOps in Wonderland
2019 • DesignOps Summit 2019
Gold
Uday Gajendar
10 Years of Enterprise UX: Reflecting on the community and the practice
2025 • Enterprise Community
Rachael Dietkus, LCSW
Leading through the long tail of trauma
2022 • Advancing Research Community
Kat Vellos
Opener: The Other L Word
2024 • DesignOps Summit 2020
Gold

More Videos

Doug Powell

"The user experience will allow us to win."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"Power dynamics exist in every session. People don’t want to be embarrassed or feel put on the spot."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"Signals are something that’s happening at the horizon, percolating at the fringes of society or in a narrow subset of the population."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"We can’t complain about not having a seat at the table if we don’t offer one of our own."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"We created an AI persona called Dreamweaver to answer real-time team capacity and project questions."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"When executives start questioning your research details, that’s a good sign they’re engaged and trusting the process."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"I felt like a journalist pretending to be a designer."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"The curse of hyper-focus means you lose peripheral vision and miss the larger context and customer needs."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"At General Assembly, I learned that product management is focused more on outcomes and iterative discovery."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023