Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Emily Eagle
Can't Rewind: Radio and Retail
2019 • Enterprise Experience 2019
Gold
Phil Gilbert
A Consistent Culture of Design
2015 • Enterprise UX 2015
Gold
Lin Nie
When Thought-worlds Collide: Collaborating Between Research and Practice
2021 • Advancing Research 2021
Gold
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Dan Saffer
Why AI projects fail (and what we can do about it)
2025 • Rosenfeld Community
Aurobinda Pradhan
Introduction to Collaborative DesignOps using Cubyts
2022 • DesignOps Summit 2022
Gold
Mike Oren
Improving Democratized Research with CustomGPTs and Gems
2026 • Rosenfeld Community
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold
Amy Jiménez Márquez
The Atypical UX Manager Path
2020 • Enterprise Community
Ron Bronson
Design, Consequences & Everyday Life
2022 • Civic Design 2022
Gold
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • Designing with AI 2025
Gold
Brendan Jarvis
Framing Tomorrow by Questioning Today
2022 • Design at Scale 2022
Gold
Jon Fukuda
Theme One Intro
2023 • DesignOps Summit 2023
Gold
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • DesignOps Summit 2022
Gold
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold

More Videos

Alfred Kahn

"Organizing your team in a fractal structure ensures product, engineering, and design are always represented as partners."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Peter Merholz

"Design impact doesn’t exist separate from the work with others – it’s part of the initiative's success."

Peter Merholz

The Trials and Tribulations of Directors of UX

July 13, 2023

Leisa Reichelt

"When negotiating, bring plan A—show what you need and what good looks like—even if it seems outrageous."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Sabrina Mach

"Design ends up being a means to fix what development has already delivered."

Sabrina Mach Nina Wainwright

How to Design Your Design Operating Model

September 29, 2021

Natalie Dunbar

"Creating frameworks helps identify responsibilities, dependencies, and critical handoffs across disciplines."

Natalie Dunbar

DesignOps and Content Strategy: Envisioning the Future Together

October 1, 2021

Peter Merholz

"Career growth is mostly about skills development, responsibility, and influence rather than just compensation or titles."

Peter Merholz

The 2025 State of UX/Design Organizational Health

November 12, 2025

Chris Geison

"By sitting at the table with sales and marketing, we can catch bias and produce more rigorous research."

Chris Geison Cristen Torrey Eric Mahlstedt

What is Research Strategy?: A Panel of Research Leaders Discuss this Emergent Question

March 4, 2021

Dr. Jamika D. Burge

"We have to figure out the strategic, unique value to our clients, stakeholders, and customers."

Dr. Jamika D. Burge Steve Portigal Alba Villamil Sam Ladner

The Future of Research: Bridging the Gaps

July 29, 2021

Louis Rosenfeld

"If your talk isn’t accepted, it doesn’t mean it’s bad—it might just not fit the program’s narrative arc or available slots."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024