Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Erika Flowers
AI-Readiness: Preparing NASA for a Data-Driven, Agile Future
2025 • Designing with AI 2025
Gold
Dr. Jamika D. Burge
The Future of Research: Bridging the Gaps
2021 • Advancing Research Community
Gina Mendolia
Coordinated collaboration: a Service Design & DesignOps love story
2025 • Advancing Service Design 2025
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Adrian Howard
Sturgeon’s Biases
2024 • DesignOps Summit 2024
Gold
John Taschek
Making People the X-Factor in the Enterprise
2018 • Enterprise Experience 2018
Gold
Christopher Geison
Theme 1 Intro
2024 • Advancing Research 2024
Gold
Frances Yllana
The Big Question about Impact: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Dave Hoffer
UX Job Search AMA #3 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Joseph Meersman
Sweating the Pixel: Scaling Quality through Critique
2021 • Design at Scale 2021
Gold
Bria Alexander
Reflect and Chart Forward
2021 • Civic Design 2021
Gold
Jules Monza
Use These Words and Count These Things
2024 • DesignOps Summit 2024
Gold
Erin Malone
Understanding the past to prepare for the future
2024 • Rosenfeld Community
Sara Asche Anderson
Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy
2024 • Enterprise Experience 2020
Gold
Sean Baker
Weaving Knowledge Management into the Fabric of Our Design Practice
2025 • DesignOps Summit 2025
Gold

More Videos

John Cutler

"I never thought I'd say this, but can someone please for love of God, just make a RACI so we can align on who's responsible for things."

John Cutler

The Alignment Trap

November 29, 2023

Samuel Proulx

"An inaccessible tool costs more than just the person with accessibility needs; it costs their friends and family too."

Samuel Proulx

From Standards to Innovation: Why Inclusive Design Wins

November 19, 2025

Alastair Simpson

"A shared understanding of how you got here creates a sense of purpose, not just urgency."

Alastair Simpson

Debunking the Myths of Cross-Disciplinary Collaboration

October 24, 2019

Kit Unger

"In our desire to see the big table, we often forget that sometimes it’s more important not to seek validation at the table but to offer it to someone else."

Kit Unger Lada Gorlenko

Theme 3 Intro

June 10, 2022

JD Buckley

"Ignore the research: it’s ultimately a business decision."

JD Buckley Margot Dear Jim Kalbach Janaki Kumar

COMMUNICATE: Discussion

June 14, 2018

Husani Oakley

"If we don’t have a shared context, we’re not actually on the same page."

Husani Oakley

Theme Two Intro

June 6, 2023

Llewyn Paine

"Now a year later, the landscape is completely different with dozens of new AI design tools for all stages."

Llewyn Paine

Day 1 Using AI in UX with Impact

June 10, 2025

Scott Plewes

"Math by itself can be misleading and full of biases, but combined with humanities research, it can be very powerful."

Scott Plewes

Why Isn't Your UX Approach Going Viral?: A Mathematical Model

March 28, 2023

Mark Interrante

"Many organizations silo data inputs and prioritize within categories before combining results."

Mark Interrante Harry Max

AI for Prioritization (3rd of 3 seminars)

July 11, 2024