Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Managing the Human Engagement Risks of AI

Gold
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the Designing with AI 2025 playlist and 1 more.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

Ask the Rosenbot
Lais de Almeida
Designing Data Services
2024 • Advancing Service Design 2024
Gold
Ebru Namaldi
Designing the Designer’s Journey: Scaling Teams, Culture, and Growth Through DesignOps
2025 • DesignOps Summit 2025
Gold
Alicia Mooty
Design Staffing Models
2021 • DesignOps Summit 2021
Gold
Gabriela Barneva
Operationalizing Inclusive Design in Design Ops
2025 • DesignOps Summit 2025
Gold
Lena Shenkarenko
Collaborative Wireframing for Creating Team Alignment and Shipping Better Products
2020 • DesignOps Summit 2020
Gold
Megan Blocker
Positioning insight: Structuring teams, roles and careers for a changing research landscape
2025 • Advancing Research 2025
Gold
Ovetta Sampson
Research in the Automated Future
2022 • Advancing Research 2022
Gold
Aaron Stienstra
Leveraging Civic Design to Advance Equity and Rebuild Trust in the US Federal Government
2021 • Civic Design 2021
Gold
Natalia Radywyl
Co-Designing New Power in Australia's Public Sector
2022 • Civic Design 2022
Gold
Ariel Kennan
Building a Design Culture
2017 • Enterprise Experience 2017
Gold
Jemma Ahmed
Research at an inflection point: Adapting to a new era of collaboration, equity, and innovation
2025 • Advancing Research 2025
Gold
Nicole Wright
Democratizing Research at HoneyBook
2022 • Advancing Research 2022
Gold
Dave Malouf
Theme 3: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Ben Reason
Making the system visible: The fastest path to better decisions
2025 • Advancing Service Design 2025
Gold
Anna Nguyen
Why Our Voice of the Customer is Better Than Yours
2022 • Advancing Research 2022
Gold
Changying (Z) Zheng
Practical DesignOps: From Ideas to Tools That Teams Actually Use
2025 • Rosenfeld Community

More Videos

Chris Geison

"Most people have a marketing or sales strategy, but many research leaders look puzzled when asked about their research strategy."

Chris Geison Cristen Torrey Eric Mahlstedt

What is Research Strategy?: A Panel of Research Leaders Discuss this Emergent Question

March 4, 2021

Steve Baty

"The empathy in defending the opposing team’s idea is very, very powerful and helps find a third way in deadlocked conflicts."

Steve Baty Richard Dalton Maria Giudice Harry Max

Discussion

June 9, 2016

Louis Rosenfeld

"Find the loudest respected person on the team and get their buy-in; they can fast track weeks of work in getting everyone else on board."

Louis Rosenfeld Jose Coronado Rachel Posman Guneet Singh Crystal Yan

The Bigger Picture: A Panel Discussion

October 23, 2019

Kristin Skinner

"We didn’t want to be navel-gazy; we wanted to understand the value our teammates would find in a role like this."

Kristin Skinner

Theme 1 Intro

September 29, 2021

George Hinchliffe

"We organized UI components hierarchically so designers focus on solving user needs rather than building basic elements."

George Hinchliffe Joy Liu

Delivering Amazing Experiences

June 10, 2021

Louis Rosenfeld

"Every job really is a climate job—there are ways to bring climate focus into any role."

Louis Rosenfeld Matt Jones Olga Khroustaleva Michael Leggett Karol Munoz

Do you want to work on climate? (Climate UX Discussion Series)

November 15, 2023

Dr Chloe Sharp

"We used affinity mapping with stakeholders to get them really into the mindset of the users and advocate for them."

Dr Chloe Sharp

Using Evidence and Collaboration for Setting and Defending Priorities

November 29, 2023

Stephanie Wade

"Measuring impact is not just nice to have, it’s critical to hold ourselves accountable and to catalyze change at scale."

Stephanie Wade

Building and Sustaining Design in Government

December 8, 2021

Jon Fukuda

"Aligning on a north star problem gets everyone excited and aligned from the beginning."

Jon Fukuda Amy Evans Ignacio Martinez Joe Meersman

The Big Question about Innovation: A Panel Discussion

September 25, 2024