Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?

Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Sean Baker
Weaving Knowledge Management into the Fabric of Our Design Practice
2025 • DesignOps Summit 2025
Gold
PJ Buddhari
Meet Spectrum, Adobe’s Design System
2021 • Design at Scale 2021
Gold
Robin Beers
Navigating organizational systems: Rethinking researcher’s role in driving change
2025 • Advancing Research 2025
Gold
Jerome “Axle” Brown
How to Use Self-Directed Learning to Ensure Your Research Insights are Heard and Acted Upon
2021 • Advancing Research 2021
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Caroline Jarrett
Garbage in, garbage out? Measuring error rates to get ready for AI
2026 • Rosenfeld Community
Jon Fukuda
Theme 3 Intro
2024 • DesignOps Summit 2024
Gold
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Maverick Chan
From Doodle to Demo: AI as Our Storytelling Partner
2025 • Rosenfeld Community
Amy Bucher
Harnessing behavioral science to uncover deeper truths
2025 • Advancing Research 2025
Gold
Roberta Dombrowski
Making Research a Team Sport
2022 • Advancing Research 2022
Gold
Ovetta Sampson
Managing the Human Engagement Risks of AI
2025 • Designing with AI 2025
Gold
Roy Opata Olende
How Zapier Uses ‘All Hands Research’ to Increase Exposure to Users
2020 • Advancing Research Community
The AI + Design wave has arrived: Making sense of the 2026 AI + Design Report
2026 • Rosenfeld Community
Dolly Parikh
Exit Interview #7: Journey of a Social Entrepreneur
2026 • Rosenfeld Community

More Videos

Jon White

"We are not just doing research to find stuff out; we need to show a return on investment on everything."

Jon White Erin May

Unsticking Research for Better Information Flow

March 11, 2026

Luz Bratcher

"Creating time to do nothing is creating runway to tackle the next thing."

Luz Bratcher

This Is a Talk for Tired People

June 10, 2022

Ignacio Martinez

"The system encourages growth not just in craft but also in client engagement and team collaboration."

Ignacio Martinez

Fair and Effective Designer Evaluation

September 25, 2024

Bria Alexander

"If design ops only serves designers, it risks silos; instead it must apply design thinking across R&D and product teams - Patrizia."

Bria Alexander Patrizia Bertini Peter Boersma Jon Fukuda Dave Malouf Theresa Slate Changying (Z) Zheng

Charting the future of DesignOps: A community workshop

April 18, 2024

Michael Weir

"The part of the brain we’re aware of is like the press release office; the real decisions happen behind closed doors."

Michael Weir

Mixed Methods and Behavioural Science

May 26, 2023

Sam Proulx

"People with disabilities often pay more to buy from stores they trust are accessible and where they feel confident."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Alla Weinberg

"Managers with more burnout score 34 points lower on feeling safe to speak up than those with none."

Alla Weinberg

People Are Sick of Change: Psychological Safety is the Cure

July 20, 2023

Amy Paris

"We want to do no harm — collecting sensitive data requires care about how it’s used, shared, and stored."

Amy Paris Danielle Thierry

Delivering Equity: Government Services for All Ages, Languages, Sexual Orientations, and Gender Identities

December 9, 2021

Francesca Barrientos, PhD

"Design maturity results from behaviors and structures that enable delivering value to customers, business, ourselves, and society."

Francesca Barrientos, PhD

You Need Your Own Definition of Design Maturity

June 8, 2022