Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
Joshua Graves
We Need To Talk: Addressing Unmet Expectations (Part 2 of 3)
2025 • Rosenfeld Community
Matt Bernius
Trauma-informed Research: A Panel Discussion
2021 • Advancing Research Community
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • DesignOps Summit 2021
Gold
Jonathon Colman
How to Maximize the Impact of Content Design
2024 • DesignOps Summit 2020
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold
Sarah Gallimore
Inspire Progress with Artifacts from the Future
2022 • Civic Design 2022
Gold
Jon Fukuda
Theme 3 Intro
2024 • DesignOps Summit 2024
Gold
Jacqui Frey
Setting the Table for Dynamic Change
2019 • DesignOps Summit 2019
Gold
Mansi Gupta
Women-Centric Research: What, Why, How
2023 • Advancing Research 2023
Gold
Rachael Dietkus, LCSW
Leading through the long tail of trauma
2022 • Enterprise Community
Charlotte Vorbeck
Pipeline to Civic Design
2021 • Civic Design 2021
Gold
Tutti Taygerly
Make Space to Lead
2021 • Design at Scale 2021
Gold
Marjorie Stainback
What Research Ops Professionals Have Learned from COVID-19
2020 • Advancing Research Community
Spencer L. A. Stultz
Why Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
2023 • DesignOps Summit 2023
Gold

More Videos

Mary-Lynne Williams

"I wanted to be in a place where I was making people feel better, not worse."

Mary-Lynne Williams

Exit Interview #4: From Product Design Leadership to Sound Healing

January 14, 2026

Elana Chapman

"Assistive technology is a tool; we need to make sure we enable that tool to do its job."

Elana Chapman Li Wen Huang Divyen Sanganee Annabel Weiner

Getting started with accessibility research

February 20, 2025

Sarah Barrett

"This header took six minutes of IA work—but two years to make happen."

Sarah Barrett

The "How" of Enterprise Information Architecture

June 6, 2023

Mariesa Lenz

"Every bee understands the same purpose — the survival of the hive for winter."

Mariesa Lenz

What Beekeeping Taught me about Product Teams

October 29, 2025

Jemma Ahmed

"Duplicative research will become less acceptable; researchers must act as natural connectors to reduce it and enhance organizational knowledge sharing."

Jemma Ahmed Megan Blocker Eduardo Ortiz

Redefining the research toolkit: Expanding methodologies for a changing world

March 12, 2025

James Lang

"Scaling communities is about propagation, not just growing as big as possible."

James Lang

If you can design an app, you can design a community

May 22, 2025

Clemens Janssen

"I transitioned out of academia because I wanted to work faster and create immediate impact rather than slow, layered processes."

Clemens Janssen Jane Davis

Efficiently Scaling Research as a Team of One

March 28, 2023

Lija Hogan

"Being fluent in multiple data languages and understanding behavioral and attitudinal data will be essential."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Etienne Fang

"The person sitting next to you probably solved that problem already—just talk to one another."

Etienne Fang

Power of Insights: Why sharing is better than silos with Uber’s Insights Platform

December 16, 2019