Rosenverse

This video is only accessible to conference ticket holders.

Log in Create account

For 90 days after a conference, only paid ticket holders can watch conference videos. After that, all Gold members have access.

Contact Support if you are having any issues.

Managing the Human Engagement Risks of AI
Conference ticket
Tuesday, June 10, 2025 • Designing with AI 2025

This video is featured in the AI and UX playlist.

Share the love for this talk
Managing the Human Engagement Risks of AI
Speakers: Ovetta Sampson
Link:

Summary

AI is fast transforming the way companies operate and design products and services. New design practices like “vibe coding,” or “prompt engineering,” are accelerating product launches at speed and scale. But with the rush to market; how can teams maintain their commitment to human-centered design, minimize the risks to consumers and keep up with the increase in developer velocity? We showcase various tactics companies are using to manage the risk of implementing AI into their products while keeping their ever tightening GTM timelines.

Key Insights

  • Generative AI does not think or understand; it excels at pattern recognition and filling in blanks like a souped-up mad lib.

  • AI design requires a paradigm shift from building products to designing dynamic behaviors within multi-agent ecosystems.

  • UX designers must move beyond traditional user-device interaction design to address machine learning's stochastic and contextual outputs.

  • Generative AI models have no moral code and can generate harmful outputs without human-guided guardrails and feedback loops.

  • Red teaming, adversarial testing, and safety classifiers are essential tools to proactively identify and mitigate AI risks.

  • Data and model cards documenting dataset provenance and model training are crucial for auditing bias and risk in AI systems.

  • Automation bias can cause humans to overtrust AI outputs, requiring design strategies that clearly signal uncertainty and keep users in control.

  • Machine learning models can unlearn harmful behaviors given correct feedback, but timely monitoring is critical before embedded biases solidify.

  • Mindful AI design entails co-design, simulations of multiple futures, and iterative testing with diverse users to understand AI behavior in context.

  • UX professionals hold unique responsibility and agency to embed ethical, equitable practices in AI design and deployment.

Notable Quotes

"Generative AI is a souped up mad lib, its job is to fill in the blank."

"Good UX and good design need not only to make a comeback but lead the way in this era of AI."

"In the post software era, we have to think about how products will behave, not just how they are built."

"The user and machine can both be in control; technology affordance changes over time and outcomes are dynamic."

"Generative AI has no moral code; you have to discipline it so it can learn right from wrong."

"There is no unbiased creation of machine learning or artificial intelligence; the idea of agnostic algorithms is a myth."

"You have to build guardrails and test models with red teaming to ensure they behave ethically and safely."

"Minimize anthropomorphism in AI design so users know they are engaging with machines, not humans."

"Machine learning models can unlearn behaviors, but we need robust feedback loops to detect when they go awry."

"The more that you engage with AI technology, the better confidence you will have in designing with it."

David Cronin
Discussion
2015 • Enterprise UX 2015
Gold
Lija Hogan
Practical Principles of Inclusive Research
2023 • Advancing Research 2023
Gold
Charlotte Lee
Theme 1 Intro
2021 • Civic Design 2021
Gold
Angelos Arnis
State of DesignOps: Learnings from the 2021 Global Report
2021 • DesignOps Summit 2021
Gold
Kelly Dern
AI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process
2023 • DesignOps Summit 2023
Gold
Kit Unger
Theme 1 Intro
2022 • Design at Scale 2022
Gold
Sahibzada Mayed
Cultivating Design Ecologies of Care, Community, and Collaboration
2023 • DesignOps Summit 2023
Gold
Kit Unger
Theme 3 Intro
2022 • Design at Scale 2022
Gold
Corey Nelson
Layoffs
2022 • Advancing Research Community
Tutti Taygerly
Videconference: How to Work with Difficult People with Tutti Taygerly
2020 • Enterprise Community
Sam Ladner
How Research Can Drive Strategic Foresight
2022 • Advancing Research 2022
Gold
Rachel Posman
A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)
2020 • DesignOps Community
Pippa Lomas
Paving the Path for Neurodiversity in Design
2023 • DesignOps Summit 2023
Gold
Savannah Carlin
Don't botch the bot: Designing interactions for AI
2024 • Designing with AI 2024
Gold
Peter Merholz
The Mysterious Case of the Missing UX Career Path
2022 • DesignOps Community
Uday Gajendar
10 Years of Enterprise UX: Reflecting on the community and the practice
2025 • Enterprise Community

More Videos

Llewyn Paine

"Prompting is a specialized skill, just like statistics or in-depth interviewing; the wrong wording can bias your results."

Llewyn Paine

Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration

March 11, 2025

Sam Yen

"If your best customers say they want to engage through design thinking, that’s where you get the most traction."

Sam Yen

Driving Organizational Change Through Design? Do more of this and less of that

June 9, 2017

Meredith Black

"Burnout isn’t caused by working too much; it happens when people aren’t stimulated."

Meredith Black

Scaling Design Culture

November 6, 2017

Paula Bach

"Consistency with legacy habits versus new platforms depends on knowing your audience and the likelihood of pain from breaking habits."

Paula Bach

Improving Legacy Software: How Much Better Does it Have to Be?

March 11, 2022

Gordon Ross

"The assistant deputy minister described this to me as a design problem, even if his peers couldn’t articulate it that way."

Gordon Ross

12 Months of COVID-19 Design and Digital Response with the British Columbia Government

December 8, 2021

Joi Freeman

"Diversity at the bottom without diversity at the top means narrowing perspective from the top down."

Joi Freeman

A New Vantage Point: Building a Pipeline for Multifaceted Research(ers)

March 30, 2020

Saara Kamppari-Miller

"The guiding question I’ve adopted is who might we be excluding now."

Saara Kamppari-Miller

Theme Three Intro

September 9, 2022

Louis Rosenfeld

"Think of AI more like a junior associate, a really powerful one, not a replacement."

Louis Rosenfeld Billy Carlson Jon Fukuda Maria Taylor

How AI will Change DesignOps Tooling

October 3, 2023

Corey Long

"Don’t read canned questions during interviews; make it conversational and human."

Corey Long

Hiring in DesignOps: A Critical Study on How to Hire and Get Hired

September 23, 2024