Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Hana Nagel
Turning Research Ripples into Waves
2018 • DesignOps Summit 2018
Gold
Mark Interrante
AI for Prioritization (3rd of 3 seminars)
2024 • Rosenfeld Community
Eniola Oluwole
Lessons From the DesignOps Journey of the World's Largest Travel Site
2019 • DesignOps Summit 2019
Gold
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Tutti Taygerly
Make Space to Lead
2021 • Design at Scale 2021
Gold
Brianna Sylver
Lead With Purpose
2020 • Advancing Research 2020
Gold
Prayag Narula
Empowering Designers to do Good Research
2022 • Advancing Research 2022
Gold
Dave Hora
Advice for Establishing Research
2022 • Advancing Research Community
Louis Rosenfeld
Do you want to work on climate? (Climate UX Discussion Series)
2023 • Climate UX Interest Group
Roberta Dombrowski
Making Research a Team Sport
2022 • Advancing Research 2022
Gold
Russell Blair
Killing the blank page
2024 • Designing with AI 2024
Gold
Dan Saffer
Why AI projects fail (and what we can do about it)
2025 • Rosenfeld Community
Jennifer Kanyamibwa
Creating the Blueprint: Growing and Building Design Teams
2018 • DesignOps Summit 2018
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
Ben Reason
Making the system visible: The fastest path to better decisions
2025 • Advancing Service Design 2025
Gold

More Videos

Sara Asche Anderson

"Excessive messaging leads to change fatigue, especially during times like a pandemic."

Sara Asche Anderson Jamie Kaspszak

Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy

January 8, 2024

Mansi Gupta

"Our methodologies today are prone to producing one-size-fits-men outcomes under the guise of being gender-neutral."

Mansi Gupta

Drawing from Feminist Practice to Make Inclusive Design Operational

September 9, 2022

Jemma Ahmed

"What that chance encounter does is provide a small moment of hope and stillness that gives him a chance to renew his energy."

Jemma Ahmed

Theme Three Intro

March 29, 2023

Bassel Deeb

"These tools aren’t about adding bureaucracy but enabling teams to perform at their best under pressure."

Bassel Deeb Will Osborn

Do More With Less: Equip and Lead Design Orgs Through Adversity

October 2, 2023

Adam Cutler

"Don’t be cool, be good. Work hard at leading and managing your teams."

Adam Cutler Karen Pascoe Ian Swinson Susan Worthman

Discussion

June 8, 2016

Crystal Yan

"Research office hours sound good, but many people feel nervous or don’t know if they’re ready to participate."

Crystal Yan

Building a Customer-Centric Culture

March 30, 2020

Sabrina Mach

"Design ends up being a means to fix what development has already delivered."

Sabrina Mach Nina Wainwright

How to Design Your Design Operating Model

September 29, 2021

Marc Fonteijn

"A regular heartbeat and familiar structure are really important principles to keep a community alive."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Anna Poznyakov

"Research has layers like an onion; we operate at multiple layers, from discovery to concept evaluation."

Anna Poznyakov Richa Prajapati

Get The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact

March 12, 2021