Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Vasileios Xanthopoulos
A Top-Down and Bottom-Up Approach to User-Centric Maturity at Scale
2024 • Enterprise Experience 2020
Gold
Nicole Wright
Democratizing Research at HoneyBook
2022 • Advancing Research 2022
Gold
Brennan Hartich
Communicating and Establishing DesignOps as a New Function
2018 • DesignOps Summit 2018
Gold
Melinda Belcher
Insider preview of Enterprise Experience 2020
2020 • Enterprise Community
Ilana Lipsett
Anticipating Risk, Regulating Tech: A Playbook for Ethical Technology Governance
2021 • Civic Design 2021
Gold
Steve Portigal
Discussion
2015 • Enterprise UX 2015
Gold
Emily DiLeo
Stronger Together: Lessons Learned from UX Research Ops
2024 • DesignOps Summit 2024
Gold
Prayag Narula
How to Empower Your Designers to Do Good Research – And Why You Want To
2022 • Design at Scale 2022
Gold
Rich Mironov
How Can Product Managers and UXers Help Each Other (and Why are Product Folks so Annoying Sometimes)?
2022 • Design in Product 2022
Gold
Daniel Gloyd
Warming the User Experience: Lessons from America's first and most radical human-centered designers
2024 • Rosenfeld Community
Amy Parness
Scaling Sustainability: Complementary strategies that drive long-term success
2025 • Climate UX Interest Group
Dave Hora
Advice for Establishing Research
2022 • Advancing Research Community
Peter Van Dijck
Designing AI-first products on top of a rapidly evolving technology
2025 • Designing with AI 2025
Gold
Nancy Douyon
We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex
2018 • Enterprise Experience 2018
Gold
Jeff Ephraim Bander
Eye Tracking Gamechanger: Why Smartphone Eye Tracking will Revolutionize Your UX Research
2022 • Advancing Research 2022
Gold
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold

More Videos

Jon Fukuda

"Through DP&M, designers get more contextual detail than what might be captured in a Jira ticket."

Jon Fukuda Ellie Krysl

Design Planning and Management Support

October 3, 2023

Saara Kamppari-Miller

"Who works here anyways? I knew about immediate teams but beyond that, it was a fog."

Saara Kamppari-Miller

Cartography for Design Communities

September 10, 2025

Kevin Bethune

"We’re serving a hyper-connected, multifaceted, intersectional world, and even the planet is a stakeholder."

Kevin Bethune

Reimagining Design: Unlocking Strategic Innovation

June 8, 2022

Daniel Orbach

"One-on-ones aren’t just reflective of a point in time but are a journey you go on together."

Daniel Orbach

Zero to One: Co-Creating Operating Models with your Team

September 23, 2024

Marc Fonteijn

"High quality personal connections mean you can engage on a level playing field with serious professionals."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Clara Kliman-Silver

"When you hand over something to a machine, what is really yours? How does it affect your identity as a professional or your relationship to your work?"

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Robin Beers

"Persistence is about beating the drum over and over, even if it feels annoying."

Robin Beers Nalini Kotamraju Andy Warr

Panel: Excellence in Communicating Insights

March 26, 2024

Ana Maria Montero Barrantes

"With enough creativity and imagination, you can make anything interesting."

Ana Maria Montero Barrantes Aditi Dhar Michelle Kaplan Nate Osborne Matt Laurence

The Authentic UX Talent Show

January 8, 2024

Kristen Honey

"If you can’t measure it, it doesn’t exist is hubris in medicine and science."

Kristen Honey

"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design

November 18, 2021