Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Friday, February 18, 2022 • QuantQual Interest Group
Share the love for this talk
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Speakers: Brian T. O’Neill , Maria Cipollone , Luis Colin , Manuel Dahm and Mike Oren
Link:

Summary

Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.

Key Insights

  • UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.

  • Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.

  • Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.

  • Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.

  • Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.

  • Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.

  • Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.

  • Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.

  • Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.

  • No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.

Notable Quotes

"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."

"Design is about creating that shared space where we look at problems through different lenses."

"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."

"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."

"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."

"You need to prototype with real or believable data so the data doesn’t change the context of use."

"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."

"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."

"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."

"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."

Ask the Rosenbot
Emily Lessard
RFPs Without Tears: Writing Inclusive RFPS that Don't Scare Away Talent
2021 • Civic Design 2021
Gold
Trisha Terhar
Empathizing with the Empowered: Non-Researcher Responses to Democratization
2022 • Advancing Research 2022
Gold
Indi Young
Paying Better Attention to the Problem with Indi Young
2019 • Advancing Research Community
Bria Alexander
State of DesignOps Panel
2021 • DesignOps Summit 2021
Gold
Bassel Deeb
Do More With Less: Equip and Lead Design Orgs Through Adversity
2023 • DesignOps Summit 2023
Gold
Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
Tutti Taygerly
Make Space to Lead
2021 • Design at Scale 2021
Gold
Pippa Lomas
Paving the Path for Neurodiversity in Design
2023 • DesignOps Summit 2023
Gold
Mary-Lynne Williams
Exit Interview #4: From Product Design Leadership to Sound Healing
2026 • Rosenfeld Community
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Taylor Klassman
Shaping the Next Era of UX Research: Collaborative Forum
2025 • Advancing Research 2025
Gold
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Erin Weigel
Real-world lessons to improve your conversion rates
2024 • Rosenfeld Community
Amber Knabl
Empowering innovation: The critical role of inclusive product development in the AI era
2024 • Designing with AI 2024
Gold
Gregg Bernstein
Opportunistic Research with Gregg Bernstein
2019 • Advancing Research Community
Lin Nie
When Thought-worlds Collide: Collaborating Between Research and Practice
2021 • Advancing Research 2021
Gold

More Videos

Gina Mendolia

"Small feedback comments like, 'Hey, how’s that going?' keep the momentum alive in addition to structured meetings."

Gina Mendolia Jasmine Toy

Coordinated collaboration: a Service Design & DesignOps love story

November 19, 2025

Jennifer Bolduc

"There’s a class distinction in who can or cannot avoid going to the office; 70% of workers never left the office during the pandemic."

Jennifer Bolduc Diane Gregorio Emily Day

What's involved with getting people back to work?: A panel discussion

July 1, 2021

Robin Beers

"Great research doesn’t automatically have the intended impact because people may not want their minds changed."

Robin Beers

Beyond Insights: Researchers as Organizational Change Catalysts

March 25, 2024

Florence Okoye

"The trickster role in African storytelling reveals observersities in systems and shows new ways things should be done."

Florence Okoye

AfroFuturism and UX Research

March 27, 2023

Anat Fintzi

"When I started, the team didn’t understand what the point of UX on the back end was; I had to win their hearts and minds."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Anna Avrekh

"The transition from hands-on research to managing is a big mind shift where outcomes become more intangible."

Anna Avrekh Dr. John Pagonis Klara Pelcl Sina Schreiber

Expert Panel: Leading in and with Research

March 10, 2022

Sam Proulx

"Disability is normal, not shameful or rare, and everyone can benefit from accessibility innovations."

Sam Proulx

Accessibility: An Opportunity to Innovate

June 8, 2022

Michelle Morrison

"You are never too busy to send a handwritten note. It deepens relationships, creates trust, and enriches the cultural fabric."

Michelle Morrison

Culture Design

May 21, 2020

Dharani Perera

"Stakeholders began referring to user groups by name—humanizing conversations and keeping us focused."

Dharani Perera

The mandala of service design: unlocking alignment and action through service design

November 20, 2025