Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Benjamin Real
Showing the Value of DesignOps by Not Having a DesignOps Team
2020 • DesignOps Summit 2020
Gold
Heidi Trost
To Protect People, You Have to Protect Information: A Human-Centered Design Approach to Cybersecurity
2025 • Rosenfeld Community
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
George Abraham
Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)
2021 • DesignOps Summit 2021
Gold
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Florence Okoye
AfroFuturism and UX Research
2023 • Advancing Research 2023
Gold
Sheri Byrne-Haber
The Importance of Accessible Design Systems
2024 • DesignOps Summit 2020
Gold
Nicole Aleong
What UX research can learn from other research practices [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Adam Thomas
Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way
2022 • Design in Product 2022
Gold
Tutti Taygerly
Make Space to Lead
2021 • Design at Scale 2021
Gold
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Doug Powell
DesignOps and the Next Frontier: Leading Through Unpredictable Change
2025 • DesignOps Summit 2025
Conference
Molly Fargotstein
Multipurpose Communication & UX Research Marketing
2019 • DesignOps Community

More Videos

Amanda Kaleta-Kott

"Text-based campaigns have been very successful for reaching senior citizens."

Amanda Kaleta-Kott Lea Martin, PhD

The Joys and Dilemmas of Conducting UX Research with Older Adults

March 11, 2022

Nalini P. Kotamraju

"I fumed that my idea had been ignored, wondering if it was because I was new, a woman, brown, or had a PhD."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021

Phil Gilbert

"Culture isn’t just behavior over time, it’s the artifacts that culture creates and embraces over time."

Phil Gilbert

A Consistent Culture of Design

May 14, 2015

Jeff Sussna

"You can’t separate the front end from the back end and you can’t separate the inside from the outside."

Jeff Sussna

What DesignOps Can Learn From DevOps

November 6, 2017

Michelle Morrison

"IKEA's instruction manual is rigid; you must do exactly what you're told to get a specific result."

Michelle Morrison

Practice What You Preach

January 8, 2024

Sharon Bautista

"Browsers are considered absolutely critical to business given organizations' reliance on web apps and software that's easy to update."

Sharon Bautista

Time to Make the Donuts: How User Research Helped Bridge Disparate Teams

January 8, 2024

PJ Buddhari

"We capture component design decisions and health transparently with checklists; unchecked items correspond to open tickets for prioritization."

PJ Buddhari Nate Baldwin

Meet Spectrum, Adobe’s Design System

June 9, 2021

Rachael Dietkus, LCSW

"Having a Slack channel for chat and resources creates ongoing conversations and historical artifacts beyond the live talks."

Rachael Dietkus, LCSW Victor Udoewa Jennifer Strickland

Everything You Need to Know about the Civic Design 2022 Call for Presentations

May 17, 2022

Sha Hwang

"Participation without the redistribution of power is an empty and frustrating process for the powerless."

Sha Hwang

The First Fifty Years of Civic Design

November 16, 2022