Rosenverse
Why AI projects fail (and what we can do about it)

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • User-centered design alone often identifies problems unsuitable for AI solutions.

  • AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Holly Cole
Panel Discussion: Growing People and Teams
2018 • DesignOps Summit 2018
Gold
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group
Sarah Williams
A Framework for CX Transformation
2021 • Design at Scale 2021
Gold
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
William Newton
How to Lead With Data, and Without Data
2023 • Enterprise UX 2023
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Uday Gajendar
From AI to Zeitgeist: Theory as the design antidote to AI hype
2025 • Rosenfeld Community
Vitorio Miliano
Don’t call it AI: Turn words into numbers with quantitative ethnography
2026 • Advancing Research 2026
Gold
Iram Shah
Closing Keynote: The View from the Top
2019 • Enterprise Experience 2019
Gold
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
George Abraham
Design Systems To-Go: Reimagining Developer Handoff, and Introducing App Builder (Part 2)
2021 • DesignOps Summit 2021
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Tess Dixon
C'mon Get Happy
2021 • DesignOps Summit 2021
Gold
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Husani Oakley
Theme Two Intro
2023 • Enterprise UX 2023
Gold

More Videos

Bria Alexander

"When teams are connected to real strategic missions and own their key results, OKRs become positive."

Bria Alexander Benson Low Natalya Pemberton Stephanie Goldthorpe

OKRs—Helpful or Harmful?

January 20, 2022

Jeff Gothelf

"Nordstrom decided that innovation needed to be a corporate value, not a lab value."

Jeff Gothelf

Innovation Studios: the Engines of Enterprise Experimentation

May 14, 2015

Wyatt Hayman

"We send out surveys weekly, but respondents have two weeks to answer, accommodating different time zones and bandwidth."

Wyatt Hayman

Global Research Panels

August 8, 2020

Gillian Salerno-Rebic

"We balance AI speed with human depth to maximize value without sacrificing empathy or rigor."

Gillian Salerno-Rebic Mark Micheli

From Insight to Impact: How JourneySpark Used WEVO Pulse + Pro to Drive a 50% Lift in Ad Engagement

June 11, 2025

Yalenka Mariën

"Digitizing doesn’t mean removing your frontline; frontline workers must be empowered to assist less digital citizens."

Yalenka Mariën Marie Mervaillie

Designing for Digital Inclusion in the Belgian Government

December 8, 2021

Angelos Arnis

"AI can standardize work, but it's on the designer’s court now to bring humanity, emotion, and originality to the work."

Angelos Arnis

Navigating the Rapid Shifts in Tech's Turbulent Terrain

October 2, 2023

Benjamin Real

"After four years we were able to see in a concrete way what happened, the changes that we have been able to do for the organization."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Magdalena Zadara

"Meeting stakeholders where they are means using language and concepts they understand, not just tech jargon."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Billy Carlson

"A great design solution for a poorly defined problem is infinitely worse than an average solution applied to a well-defined problem."

Billy Carlson

Ideation tips for Product Managers

December 6, 2022