Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • User-centered design alone often identifies problems unsuitable for AI solutions.

  • AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Daniel Orbach
Zero to One: Co-Creating Operating Models with your Team
2024 • DesignOps Summit 2024
Gold
Michael Land
Establishing Design Operations in Government
2021 • DesignOps Community
Todd Healy
Driving Change with CX Metrics
2023 • Enterprise UX 2023
Gold
Melissa Tsang
From Insights to Action: Driving Business Values through DesignOps
2024 • DesignOps Summit 2020
Gold
Peter Merholz
Customer-Centered Design Organizations
2017 • Enterprise Experience 2017
Gold
Catt Small
Moving from Execution to Strategy as a Designer
2022 • Design in Product 2022
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • Rosenfeld Community
Christopher Noessel
AI of the now: Designing for Agents
2024 • Rosenfeld Community
Kate Towsey
ResearchOps AMA with Kate Towsey & Jake Burghardt
2025 • Advancing Research Community
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
Lija Hogan
Doing more with more: Lessons from the Front Lines of Democratization
2022 • Advancing Research 2022
Gold
Alicia Mooty
Design Staffing Models
2021 • DesignOps Summit 2021
Gold
Billy Carlson
Pro-level UI Tips for Beginners
2022 • DesignOps Summit 2022
Gold
Dana Chisnell
The Sensemaking Business
2026 • Advancing Research 2026
Gold
Jim Kalbach
Peace is waged with sticky notes: Mapping Real-World Experiences
2018 • Enterprise Experience 2018
Gold
Sean Baker
Weaving Knowledge Management into the Fabric of Our Design Practice
2025 • DesignOps Summit 2025
Gold

More Videos

Sarah Auslander

"Who creates the conditions for civic design to slow down? Can we afford to slow down? Can we afford not to slow down?"

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022

Lukas Moro

"The interaction comes to the foreground mainly when there’s a negotiation of intentionality between the user and the technology."

Lukas Moro

“Feels Like Paper!”: Interfacing AI through Paper

June 11, 2025

Andreas Huebner

"We used virtual sketching where users could move components into spaces representing their actual workflows on the platform."

Andreas Huebner Amy Takata Craig Brookes

What Is It Like To Be Part of The UX Team at Compass?

March 11, 2021

Benjamin Real

"The model was the work of everybody, it was not just my job."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Alla Weinberg

"Psychological safety means if I make a mistake on our team, the people won’t disconnect from me."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 8, 2022

Meredith Black

"One in three designers on our team is from another country — diversity improves relevance for our global users."

Meredith Black

Scaling Design Culture

November 6, 2017

Jill Fruchter

"Averages are an enemy; segmenting reveals the true nature of customer intent and sentiment."

Jill Fruchter

Inconvenient Insights: The Researcher's Role is to Stay Curious

March 29, 2023

Gabrielle Verderber

"Avoid walls of text and create visual hierarchy to ensure scanability by using headers, sections, lists, and images."

Gabrielle Verderber

Documentation Your Team Will Actually Use

October 3, 2023

Llewyn Paine

"Let’s all advocate for AI products and processes that keep the human at the center."

Llewyn Paine

Day 1 Using AI in UX with Impact

June 10, 2025