Rosenverse
Managing the Human Engagement Risks of AI
Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Shipra Kayan
How Tess Dixon Facilitates Team Engagement and Collaboration at Condé Nast Using Miro 
2021 • DesignOps Summit 2021
Gold
Frances Yllana
The Big Question about Impact: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Angelos Arnis
Navigating the Rapid Shifts in Tech's Turbulent Terrain
2023 • DesignOps Summit 2023
Gold
Etienne Fang
The Power of Care: From Human-Centered Research to Humanity-Centered Leadership
2021 • Advancing Research 2021
Gold
Ryan Matthew
DesignOps without Boundaries: Building More with What You Have
2025 • DesignOps Summit 2025
Conference
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
2022 • DesignOps Community
Shan Shen
Translating UX Terms into Business Contexts
2023 • Design in Product 2023
Gold
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Satyam Kantamneni
Do You Have an Experience Vision?
2023 • Enterprise Community
Matt Stone
Scaling Empathy, A Case Study in Change Management
2021 • Design at Scale 2021
Gold
Joseph Williams
Unlocking impact and influence through inclusive hiring in research
2021 • Advancing Research Community
Tony Turner
Capturing Deep Insights
2021 • DesignOps Summit 2021
Gold
Peter Morville
The Architecture of Understanding
2015 • Enterprise UX 2015
Gold
Dr. Jamika D. Burge
Advancing the Inclusion of Womxn in Research Practices
2022 • Advancing Research Community
Tracy McGoldrick
IBM User Experience Program—The What, Why and How
2021 • Advancing Research Community
Steve Sanderson
Discussion
2015 • Enterprise UX 2015
Gold

More Videos

Sarah Barrett

"Windows of opportunity happen when problem, solution, and political streams align."

Sarah Barrett

The "How" of Enterprise Information Architecture

June 6, 2023

Peter Van Dijck

"You have to build a detailed definition of what is good for my system to do meaningful automated evals."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 2: Evals are your moat

July 23, 2025

Laura Gatewood

"The vast majority of what makes up meaningful human communication is harder for us to access screen to screen."

Laura Gatewood Laine Prokay

Beyond Buzzwords: Adding Heart to Effective Slack Communication

September 23, 2024

Bria Alexander

"The primary way to convene with other attendees is through our Slack channel."

Bria Alexander

Opening Remarks

January 8, 2024

Chris Geison

"Six million people worldwide have died of COVID. Millions more are dealing with lasting effects."

Chris Geison

Theme 1 Intro

March 9, 2022

Mansi Gupta

"In that moment, I looked down at my chest and failed to find a breast pocket of my own."

Mansi Gupta

Drawing from Feminist Practice to Make Inclusive Design Operational

September 9, 2022

Shipra Kayan

"Our one metric that mattered was maximizing the number of VOС-tagged tickets solved – showing customer feedback made it to the roadmap."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Alexia Cohen

"Federal funding does not provide food for participants, so we had to plan accordingly to make participation irresistible."

Alexia Cohen Adriane Ackerman

Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona

December 4, 2024

Sarah Auslander

"Civic action as proof of concept to get government moving, to get the wheels turning was really interesting."

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022