Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Erik Flowers
Introduction to MURAL for UX
2021 • Design at Scale 2021
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Advancing Research 2023
Gold
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold
Kyria Stephens
Power to Heal: Civic Design in the Aftermath of Tragedy
2022 • Civic Design 2022
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Enterprise UX 2023
Gold
Peter Merholz
Customer-Centered Design Organizations
2017 • Enterprise Experience 2017
Gold
Josina Vink
Navigating the pitfalls of systems thinking in service design
2024 • Advancing Service Design 2024
Gold
Jorge Arango
The Best of Both Worlds: How to Integrate Paper and Digital Notes (1st of 3 seminars)
2024 • Rosenfeld Community
Nina Jurcic
The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
2023 • DesignOps Summit 2023
Gold
Joel Branch
Humanizing AI: Filling the Gaps with Multi-faceted Research
2021 • Advancing Research 2021
Gold
Ed Mullen
Designing the Unseen: Enabling Institutions to Build Public Trust
2022 • Civic Design 2022
Gold
Erin May
Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
2022 • Advancing Research 2022
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Melinda Belcher
Insider preview of Enterprise Experience 2020
2020 • Enterprise Community
Jemma Ahmed
Theme 2 Intro
2022 • Advancing Research 2022
Gold
Mark Interrante
AI for Prioritization (3rd of 3 seminars)
2024 • Rosenfeld Community

More Videos

Daniel J. Rosenberg

"The marketplace is going to decide if it’s close enough in cost-benefit tradeoff."

Daniel J. Rosenberg

Designing with and for Artificial Intelligence

August 11, 2022

Dan Mall

"A lot of people think about design systems as a project, but they’re really about how an organization gets work done."

Dan Mall

“Ask Me Anything” with Dan Mall, Author of Upcoming Rosenfeld Title, Design that Scales

October 2, 2023

Peter Morville

"Categories are dangerous but not bad."

Peter Morville

The Architecture of Understanding

May 13, 2015

Kyria Stephens

"Tops was more than a grocery store; it was a bank, a pharmacy, and a community meeting place."

Kyria Stephens Marlon Kerner

Power to Heal: Civic Design in the Aftermath of Tragedy

November 17, 2022

Nathan Shedroff

"You wouldn’t use design thinking to optimize inventory—you use Six Sigma."

Nathan Shedroff

Redefining Value: Bridging the Innovation Culture Divide

May 14, 2015

Kim Fellman Cohen

"Our team’s improvement wasn’t the final state—we set ambitious goals to be in the 80th to 90th percentile next year."

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

Bria Alexander

"When I think about resilience, I think about flexibility, like trees that bend in strong winds instead of breaking — that starts with mindset and communication."

Bria Alexander Laura Gatewood Corey Long Daniel Orbach Laine Prokay Deanna Smith

The Big Question about Resilience: A panel discussion

September 23, 2024

Bria Alexander

"If these two voices don’t work for you, you can reach out at conferences@rosenfeldmedia.com."

Bria Alexander Louis Rosenfeld

Opening Remarks Day 2

March 26, 2024

Changying (Z) Zheng

"Our approach to career development is thinking about two dimensions: what you’re good at and enjoy, and what you’re good at but don’t enjoy."

Changying (Z) Zheng

Practical DesignOps: From Ideas to Tools That Teams Actually Use

September 25, 2025