Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)
Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • User-centered design alone often identifies problems unsuitable for AI solutions.

  • AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Peter Levin
Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility
2024 • Advancing Research 2024
Gold
Uday Gajendar
Day 1 Welcome
2024 • Designing with AI 2024
Gold
Jennifer Kong
Journeying toward AI-assisted documentation in healthcare
2024 • Designing with AI 2024
Gold
Etienne Fang
Power of Insights: Why sharing is better than silos with Uber’s Insights Platform
2019 • Advancing Research Community
Bria Alexander
Theme Two Intro
2023 • DesignOps Summit 2023
Gold
Anna Avrekh
Expert Panel: Leading in and with Research
2022 • Advancing Research 2022
Gold
Sarah Barrett
AI in Real Life: Using LLMs to Turbocharge Microsoft Learn
2025 • Rosenfeld Community
Samuel Proulx
Designing for Disability, Innovating for Everyone
2025 • Advancing Research 2025
Gold
Tom Armitage
Day 2 Panel: Looking ahead: Designing with AI in 2026
2025 • Designing with AI 2025
Gold
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Reginé Gilbert
Asking the Right Questions: Life, Hope and Moving Forward During the Pandemic
2022 • Design at Scale 2022
Gold
Lena Shenkarenko
Collaborative Wireframing for Creating Team Alignment and Shipping Better Products
2020 • DesignOps Summit 2020
Gold
Dr Chloe Sharp
Using Evidence and Collaboration for Setting and Defending Priorities
2023 • Design in Product 2023
Gold
Emily Danielson
“I mean, I can lift a shovel”: Design Skills in Disaster Response
2022 • Design at Scale 2022
Gold
Jemma Ahmed
Democratization: Working with it, not against it [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Eduardo Ortiz
Theme 3 Intro
2025 • Advancing Research 2025
Gold

More Videos

Daniel Gloyd

"After holding a warm cup of coffee, people perceive others as more trustworthy and engaging."

Daniel Gloyd

Designing Warmth

February 26, 2025

Steve Portigal

"Visualizing data in a way executives can interpret is key to shifting from I think to I know."

Steve Portigal Chris Chapo Kelly Goto Christian Rohrer

Discussion

May 13, 2015

Harry Max

"Start with yourself in prioritization, and you’ll likely infect your team and then the organization."

Harry Max Jim Meyer

Prioritization for Leaders (2nd of 3 seminars)

June 27, 2024

Brigette Metzler

"One ops person to five researchers seems pretty legit, according to Kate Towsie."

Brigette Metzler

Scaling ResearchOps: Helping Researchers do Their Best Work

March 30, 2020

Jennifer Kanyamibwa

"Process can make things scalable and help you take the smallest problems or the biggest problems and really bring folks together in ways that are not even imaginable."

Jennifer Kanyamibwa

Creating the Blueprint: Growing and Building Design Teams

November 8, 2018

Emilia Åström

"Reflection after sessions increases ongoing collaboration design and continuous improvement."

Emilia Åström Jim Kalbach

Unlock Your Team’s Intelligence with Collaboration Design

June 9, 2022

Christian Rohrer

"Having no competition can be a problem because there’s no pressure to create a great user experience."

Christian Rohrer

Insight Types That Influence Enterprise Decision Makers

May 13, 2015

George Zhang

"Moving research upstream in the product process helps solve problems faster and save millions of dollars."

George Zhang Molly Stevens

UX Research Excellence Framework

March 11, 2021

Bria Alexander

"Cohorts are one of the most highly rated parts of our conferences every single year, you get to meet somebody new from a different company."

Bria Alexander

Opening Remarks

September 30, 2021