Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • •

    Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • •

    The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • •

    User-centered design alone often identifies problems unsuitable for AI solutions.

  • •

    AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • •

    Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • •

    Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • •

    Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • •

    Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • •

    AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • •

    There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Chris Geison
Theme Two Intro
2023 • Advancing Research 2023
Gold
Bria Alexander
State of DesignOps Panel
2021 • DesignOps Summit 2021
Gold
Nora Tejeda
Scaling Design Capabilities at BBVA Through a Self-service Design Model
2021 • Design at Scale 2021
Gold
Emily Lessard
RFPs Without Tears: Writing Inclusive RFPS that Don't Scare Away Talent
2021 • Civic Design 2021
Gold
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Dave Malouf
Panel: Design Systems and Documentation
2017 • DesignOps Summit 2017
Gold
Jorge Arango
The Best of Both Worlds: How to Integrate Paper and Digital Notes (1st of 3 seminars)
2024 • Rosenfeld Community
Louis Rosenfeld
Coffee with Lou: Should You Write a (UX) Book?
2024 • Rosenfeld Community
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • DesignOps Summit 2025
Gold
Fatimah Richmond
The Future of ReOps as a Strategic Function: A Roadmap for Getting There
2024 • Advancing Research 2024
Gold
Patrizia Bertini
DesignOps + KPIs = Measure your Impact!
2024 • DesignOps Summit 2020
Gold
Sabrina Mach
How to Design Your Design Operating Model
2021 • DesignOps Summit 2021
Gold
Sarah Fathallah
Lessening the Research Burden on Vulnerable Communities
2020 • Advancing Research 2020
Gold
Erin Malone
Understanding the past to prepare for the future
2024 • Rosenfeld Community

More Videos

Victor Udoewa

"Our ways of being are roots that grow into ways of knowing, doing, and methods, all nourished by systems of value."

Victor Udoewa

Beyond Methods and Diversity: The Roots of Inclusion

March 26, 2024

Peter Morville

"When you’re in a culture that celebrates complexity, making things simple is still a very hard thing to do."

Peter Morville

The Architecture of Understanding

May 13, 2015

Sam Ladner

"I am swimming in a sea of information, like a little fish, and eating all the stuff."

Sam Ladner

How Research Can Drive Strategic Foresight

March 9, 2022

Michelle Bejian Lotia

"We created research indexes with search blocks that update over time to help people focus on the most recent insights."

Michelle Bejian Lotia Anne-Marie Morell

Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization

March 28, 2023

Leisa Reichelt

"At Atlassian, many confident non-researchers were already doing lots of research, which was an opportunity and a challenge."

Leisa Reichelt

Opening Keynote: Operating in Context

November 7, 2018

Sarah Gallimore

"The mayor wanted to shrink the neighborly bonds that the city was known for by investing in a 2030 living Monument of collaboration and community."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Joerg Beringer

"The system is live now analyzing audience-submitted tasks and producing results in minutes."

Joerg Beringer Thomas Geis

Scaling User Research with AI: Continuous Discovery of User Needs in Minutes

September 10, 2025

Peter Merholz

"Team leadership means managing down to empower your team, managing across to build stakeholder relationships, and managing up with executives."

Peter Merholz

Customer-Centered Design Organizations

June 8, 2017

Dalia El-Shimy

"The single most important thing we can do at the table is expanding the boundaries of knowledge for decision makers."

Dalia El-Shimy

So You've Got a Seat at the Table. Now What?

March 31, 2020