Rosenverse
Why AI projects fail (and what we can do about it)

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Why AI projects fail (and what we can do about it)

Wednesday, May 14, 2025 • Rosenfeld Community
Share the love for this talk
Why AI projects fail (and what we can do about it)
Speakers: Dan Saffer
Link:

Summary

Most AI projects fail. Somewhere between 50-90% of them, which is double the rate of more traditional tech projects. This Rosenverse Session will draw on years of Carnegie Mellon HCII research to dive into the five traps that AI projects can fall into, and then talk about what designers and project managers can do to avoid those traps. Including one startling finding: user-centered design alone isn’t enough.

Key Insights

  • •

    Most AI projects fail due to choosing the wrong problems rather than poor execution.

  • •

    The AI innovation gap occurs because data scientists focus on technically hard but low-value problems.

  • •

    User-centered design alone often identifies problems unsuitable for AI solutions.

  • •

    AI works best for low-risk, moderate accuracy (~90%), narrow tasks like spam filtering or smart text suggestions.

  • •

    Executives push AI fast to avoid falling behind, risking deployments without clear user or business value.

  • •

    Consequences scanning is a vital method to detect and mitigate ethical risks before AI product launch.

  • •

    Matchmaking AI capabilities with real user needs and organizational goals increases project success.

  • •

    Agents add new AI capabilities to sense, think, and act, requiring adapted design processes.

  • •

    AI lacks context, aesthetic taste, and common sense—designers must provide these to create effective AI products.

  • •

    There is currently little corporate or legal accountability for AI ethics; responsibility largely falls to product teams.

Notable Quotes

"Welcome to the AI party, we've got 10 years of research to share."

"Executives said it's better to go fast and fail than to go slow and succeed."

"AI can be magical, but it's also just not that smart."

"Don Norman was named the top accomplished woman in UX by AI, even though Don is a man."

"Companies hunt for AI innovation in technically challenging areas instead of simpler, more valuable places."

"Most AI fails because projects require near perfect accuracy, which AI can't reliably deliver."

"The traditional user-centered design process doesn't work well for AI problems."

"Matchmaking connects AI capabilities to user needs so you find the low hanging fruit projects."

"AI struggles with context, taste, and common sense, and designers bring that to the table."

"Consequences scanning helps detect unintended harms and risks before launching AI features."

Ask the Rosenbot
Jack Moffett
SAFe or Sorry?
2019 • Enterprise Community
John Calhoun
Bring your DesignOps Story to Life! The Definitive DesignOps Book Jam
2023 • DesignOps Summit 2023
Gold
Sheri Byrne-Haber
Accessibility at Scale
2021 • Design at Scale 2021
Gold
Chris Moses
Stretching the Definition of DesignOps with Product Development
2018 • DesignOps Summit 2018
Gold
Shipra Kayan
How Tess Dixon Facilitates Team Engagement and Collaboration at Condé Nast Using Miro 
2021 • DesignOps Summit 2021
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
Prayag Narula
Dialing for Research: How to Reach the Unreachable
2022 • Advancing Research 2022
Gold
Stefanie Hutka
Shaping the Tools that Shape Us
2026 • Shift UX 2026
Conference
Victor M. Gonzalez
Practicing Learners and Learning Practitioners
2021 • Advancing Research 2021
Gold
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Kate Towsey
Participant Recruitment and Management Tools
2026 • Advancing Research 2026
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Design at Scale 2022
Gold
Jennifer Kong
Journeying toward AI-assisted documentation in healthcare
2024 • Designing with AI 2024
Gold
Taiye Akin-Akinyosoye
Amplifying voices and enhancing user research through group interviews
2025 • Advancing Research 2025
Gold
Kritika Yadav
Optimizing AI Conversations: A Case Study on Personalized Shopping Assistance Frameworks
2025 • Designing with AI 2025
Gold
Benjamin Real
Showing the Value of DesignOps by Not Having a DesignOps Team
2020 • DesignOps Summit 2020
Gold

More Videos

Dr Chloe Sharp

"There was a lot of communication needed around roles, especially since our software developer was external and not always involved in day-to-day decisions."

Dr Chloe Sharp

Using Evidence and Collaboration for Setting and Defending Priorities

November 29, 2023

Dr. Karl Jeffries

"Design ops has the opportunity to set a new default future, not follow the path set by the past."

Dr. Karl Jeffries

The Science of Creativity for DesignOps

January 8, 2024

Amber Knabl

"AI is currently automating and amplifying human biases that exist in society today."

Amber Knabl

Empowering innovation: The critical role of inclusive product development in the AI era

June 4, 2024

Dorelle Rabinowitz

"When we have genuine relationships and we trust each other, it leads to so much goodness."

Dorelle Rabinowitz

The Magic Word is Trust

June 15, 2018

Dan Willis

"The more specific a story gets, the more universal the themes are felt by everybody."

Dan Willis

Enterprise Storytelling Sessions

June 3, 2019

Courtney Kaplan

"I was much more interested in talking to clients, understanding the scope, and translating that into what designers would build."

Courtney Kaplan

Taking it to the next level: Career paths in DesignOps

November 8, 2018

Bethany Brown

"I spent 40% of my day verifying what the system already knows."

Bethany Brown

Rewiring operations with service design and AI

November 20, 2025

Laura Klein

"Well-set-up AI systems pulling data from multiple company sources will have more context, but it’s still limited compared to human understanding."

Laura Klein

Human vs. machine: Testing AI’s ability to synthesize and analyze research

March 11, 2026

Weidan Li

"Qualitative synthesis is like watching a movie in a cinema; AI synthesis feels like watching a recap on YouTube—fast but missing the immersive experience."

Weidan Li

Qualitative synthesis with ChatGPT: Better or worse than human intelligence?

June 4, 2024

Latest Books All books