Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Bob Baxley
Leading with Design Operations Past and Present
2019 • DesignOps Community
Harry Max
Prioritization for Leaders (2nd of 3 seminars)
2024 • Rosenfeld Community
Iain McMaster
Design and Product: from Frenemy to Harmony
2023 • Design in Product 2023
Gold
Suzan Bednarz
AccessibilityOps for All
2024 • DesignOps Summit 2020
Gold
Margot Bloomstein
Fostering Trust in Your Brand and Beyond
2020 • Enterprise Community
Jess Greco
Claiming your power: Practical tools for amplifying your unique voice
2025 • Advancing Research 2025
Gold
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Brad Orego
Bringing Customer Research to More Internal Teams
2022 • Advancing Research 2022
Gold
Soma Ghosh
What emerging methods are advancing UX research [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Richard Buchanan
Creativity and Principles in the Flourishing Enterprise
2018 • Enterprise Experience 2018
Gold
Tricia Wang
Spatial Collapse: Designing for Emergent Culture
2024 • Enterprise Experience 2020
Gold
Liam Thurston
Why Your Design Team Is Quitting, And How To Fix It
2022 • Design at Scale 2022
Gold
Sarah Fathallah
Lessening the Research Burden on Vulnerable Communities
2020 • Advancing Research 2020
Gold
Dawn Ressel
Full-Stack User Experiences: A Marriage of Design and Technology
2016 • Enterprise UX 2016
Gold
Edgar Anzaldua Moreno
Using Research to Determine Unique Value Proposition
2021 • Advancing Research 2021
Gold
Theresa Marwah
How Atlassian is Operationalizing Respect in Research
2020 • Advancing Research Community

More Videos

Darian Davis

"By observing Jeff's concerns and asking questions, I was able to show empathy and distract myself from getting defensive."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Aaron Stienstra

"Citizen participation is like eating spinach—everyone agrees it’s good, but few actually do it meaningfully."

Aaron Stienstra Lashanda Hodge

Leveraging Civic Design to Advance Equity and Rebuild Trust in the US Federal Government

December 8, 2021

Kavana Ramesh

"Watching users navigate with assistive technology exposes you to barriers that people with disabilities face online."

Kavana Ramesh

Meaningful inclusion: Practicing accessibility research with confidence

September 24, 2024

Helen Armstrong

"Trust erodes very quickly the moment an AI prediction is a little off or wrong."

Helen Armstrong

Augment the Human. Interrogate the System.

June 7, 2023

Scott Plewes

"Personas and journeys are all models we use in UX to manage complexity."

Scott Plewes

Why Isn't Your UX Approach Going Viral?: A Mathematical Model

March 28, 2023

Christian Bason

"Designers can design both at nano levels and interplanetary scales like Mars architecture."

Christian Bason

Expand—Rethinking Design for Public Challenges

September 14, 2022

Peter Van Dijck

"All the complicated-sounding techniques are just ways to put the relevant text into the context window."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!

July 30, 2025

Soma Ghosh

"It’s not answers we’re after but further questions to keep the conversation and understanding alive."

Soma Ghosh Savina Hawkins Dave Hoffer Rob Thomas Victor Udoewa

What emerging methods are advancing UX research [Advancing Research Community Workshop Series]

September 28, 2023

Steve Portigal

"Reliance on foundational basics applies whether you’ve been a researcher for two months or twenty years."

Steve Portigal Susan Simon-Daniels Tamara Hale Randolph Duke II

War Stories LIVE! Q&A-Discussion

March 30, 2020