Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Scott Jensen
Short Take #2: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Nalini Kotamraju
Research After UX
2024 • Advancing Research 2024
Gold
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
John Calhoun
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
2024 • DesignOps Summit 2024
Gold
Daniel Orbach
Zero to One: Co-Creating Operating Models with your Team
2024 • DesignOps Summit 2024
Gold
Craig Villamor
Design Systems for Ethical Design
2023 • Enterprise Community
Jon Fukuda
Theme One Intro
2023 • DesignOps Summit 2023
Gold
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Michelle Chin
The DesignOps Starter Kit
2021 • DesignOps Summit 2021
Gold
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • Rosenfeld Community
Dr. Jamika D. Burge
Advancing the Inclusion of Womxn in Research Practices
2022 • Advancing Research Community
Yasmine Khan
Checking Bias and Listening to Financially Vulnerable Americans
2020 • Advancing Research 2020
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Nathan Curtis
Discussion
2016 • Enterprise UX 2016
Gold
Dave Gray
Group Activity: Making Sense of DesignOps
2017 • DesignOps Summit 2017
Gold

More Videos

Kavana Ramesh

"Adapt your protocol as needed to support different users; equity may mean not treating everyone exactly the same."

Kavana Ramesh

Meaningful inclusion: Practicing accessibility research with confidence

September 24, 2024

Rima Campbell

"AI makes data more accessible to everyone but won’t replace the need for validated, empathetic UX research."

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Bria Alexander

"The Slack channel is the fastest way to get help, whether you’re virtual or in person."

Bria Alexander Louis Rosenfeld

Opening Remarks Day 1

March 25, 2024

Kim Lenox

"Sharing meals is a really critical part of building trust which seems kind of strange but breaking bread together makes a difference."

Kim Lenox

Leading Distributed Global Teams

May 20, 2019

Louis Rosenfeld

"We took thousands of hours of curated content and built a chatbot around it so it’s not garbage in, garbage out."

Louis Rosenfeld

The Rosenbot and the Rosenverse: An AMA with Lou Rosenfeld

June 5, 2024

Steve Portigal

"People in organizations are already, to some extent, out there talking to users."

Steve Portigal

Looking Back…to Look Ahead

March 26, 2024

Maggie Dieringer

"Getting designers into a cadence of reviewing work with senior stakeholders was not a muscle the team had built."

Maggie Dieringer

Creating Consistency Through Constant Change

January 8, 2024

Brigette Metzler

"Research Ops needs an operational brain — it’s not the kind of brain researchers usually carry."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community

August 27, 2020

Sam Proulx

"Involving people with disabilities throughout the product development cycle is crucial, from prototype reviews through to acceptance testing."

Sam Proulx

Prototype Reviews, People With Disabilities, and You

October 1, 2021