Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Jemma Ahmed
Collaboration: learning from other fields beyond our own [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2019 • Enterprise Experience 2019
Gold
Sarah Coyle
Design and Analytics with Sarah Coyle
2020 • DesignOps Community
Crystal Yan
Building a Customer-Centric Culture
2020 • Advancing Research 2020
Gold
Dantley Davis
Leadership & Diversity—A Fireside Chat with Dantley Davis
2020 • Enterprise Community
Dan Saffer
Why AI projects fail (and what we can do about it)
2025 • Rosenfeld Community
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Clara Kliman-Silver
UX Futures: The Role of Artificial Intelligence in Design
2023 • Enterprise UX 2023
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps Summit 2024
Gold
James Chudley
Decarbonising User Journeys: How minimising enables us to do more with less
2025 • Climate UX Interest Group
Sylvie Abookire
A Civic Designer's Guide to Mindful Conflict Navigation
2022 • Civic Design 2022
Gold
Kara Kane
Theme One Intro
2022 • Civic Design 2022
Gold
Jorge Arango
Design as an Antidote to VUCA
2019 • Enterprise Community
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Peter Merholz
The Mysterious Case of the Missing UX Career Path
2022 • DesignOps Community

More Videos

Michele Wong

"Manage the toolkit as a project—have a backlog, prioritize changes, and dedicate sprints to keep it updated."

Michele Wong

Helping Them Help Us

January 8, 2024

Nicole Aleong

"Expectation describes how things ought to be and reinforces the distinction between the present and what is not yet here."

Nicole Aleong

Future Orientations to Everyday Life: Futures Anthropology as a Methodology

March 26, 2024

Christian Bason

"Design makes the abstract tangible by transforming concepts into something with form, shape, and expression."

Christian Bason

Innovating With People: Unleashing the Potential of Civic Design

December 8, 2021

Bria Alexander

"We have the entire Rosenfeld Media staff on hand to support you throughout this conference."

Bria Alexander Louis Rosenfeld

Opening Remarks Day 2

March 26, 2024

Josina Vink

"Instead of perpetuating professional design as the dominant approach, we should support people's many ways of shaping systems."

Josina Vink

Navigating the pitfalls of systems thinking in service design

December 4, 2024

Stephen Pollard

"The landing charge is the key thing airlines look at and that needs to stay flat."

Stephen Pollard

Closing Keynote: Getting giants to dance - what can we learn from designing large and complex public infrastructure?

November 7, 2017

Sam Proulx

"Accessibility doesn’t stop at the desktop just because you’ve made your desktop website accessible."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

November 17, 2022

Melinda Belcher

"If you hire by portfolio visual impact, industry experience, and price alone, you’re probably making the wrong choice."

Melinda Belcher

Insider preview of Enterprise Experience 2020

May 28, 2020

Aurobinda Pradhan

"Traditional devops tools are more task and issue-centric, but designers think in processes and workflows."

Aurobinda Pradhan Shashank Deshpande

Introduction to Collaborative DesignOps using Cubyts

September 9, 2022