Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
Summary
Join us for a different type of Quant vs. Qual discussion: instead of discussing how data science and quantitative research methods can power UX research and design, we’re going to talk about designing enterprise data products and tools that put ML and analytics into the hands of users. Does this call for new, different, or modified approaches to UX research and design? Or do these technologies have nothing to do with how we approach design for data products? The session’s host will be Brian T. O’Neill who is also the host of the Experiencing Data podcast and founder of Designing for Analytics, an independent consultancy that helps data products leaders use design-driven innovation to deliver better ML and analytics user experiences. In this session, we’ll be sharing some rapid [slides-optional] anecdotes and stories from the attendees and then open up the conversation to everyone. We hope to get perspectives from both enterprise data teams doing “internal” data analytics or ML/AI solutions development as well as software/tech companies as well who may offer data-related platform tools, intelligence/SAAS products, BI/decision support solutions, etc. Slots are open to both experienced UX practitioners as well as data science / analytics / technical participants who may have participated in design or UX work with colleagues. Please share! If folks are too quiet in the session, you may be subject to a drum or tambourine solo from Brian. Nobody has all of this “figured out yet” and experiments and trials are welcome.
Key Insights
-
•
UX researchers in ML-heavy environments often face steep domain knowledge gaps requiring strong interview facilitation to bridge communication.
-
•
Designing for AI data products demands a shift from generic user interfaces to tools tailored for expert users who need explainability over polish.
-
•
Cross-functional collaboration thrives by creating shared spaces that respect differing expertise while focusing on a unifying product goal.
-
•
Decision culture focusing on how decisions are made is more useful than a vague data culture mindset when designing AI/ML products.
-
•
Besides customers and stakeholders, data labelers who curate training data form a critical, often overlooked human part of the ML ecosystem.
-
•
Model interpretability can be more important than raw accuracy because users must trust a system to adopt it and generate value.
-
•
Prototyping data products requires believable data and can be facilitated by testing edge cases like false positives and trust thresholds early.
-
•
Incremental learning loops involving user feedback can improve models over time and should be designed into AI products.
-
•
Contextualizing data model outputs with business rules and user environment (e.g., seasonal factors) increases relevance and trust.
-
•
No-code data science tools and scenario building help teams quickly experiment with data products despite the usual slow model training cycles.
Notable Quotes
"Smoke comes out of my ears when I'm talking to some of the more advanced applications in machine learning."
"Design is about creating that shared space where we look at problems through different lenses."
"Decision culture is a better lens than data culture for thinking about what decisions we're trying to facilitate with AI."
"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."
"If nobody uses this because they don’t trust it, it doesn’t matter how accurate the model is."
"You need to prototype with real or believable data so the data doesn’t change the context of use."
"Some things that look like blockers, like domain expertise gaps, actually force you to become a better interviewer."
"We’re not always trying to use machine learning everywhere—sometimes the answer is to ignore it."
"Users want to understand why this happened, what will happen, and how can we make it happen—explainability is critical."
"The presenting problem might be a dashboard with a score, but the real need is deciding what to do with that information."
Or choose a question:
More Videos
"Bots is like a perfect on-ramp for UX for AI."
Greg NudelmanDesigning Conversational Interfaces
November 14, 2019
"Day two is all about design ops at scale, the maturing and evolution of design ops teams."
Bud CaddellTheme 2 Intro
September 30, 2021
"Removing unnecessary fields like birthday on signup avoids tripping users up — get them started fast."
Billy CarlsonPro-level UI Tips for Beginners
September 9, 2022
"Research participants are demanding that we see them as equals rather than vessels to extract knowledge from."
Robert Fabricant Sahibzada Mayed Nidhi Singh RathoreIndustry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]
October 2, 2024
"Journeys Spark drove a 50% CTR increase by speeding up persona-driven content optimization with AI before human validation."
Gillian Salerno-Rebic Mark MicheliFrom Insight to Impact: How JourneySpark Used WEVO Pulse + Pro to Drive a 50% Lift in Ad Engagement
June 11, 2025
"Excel is the mental model of the scientific research world—a two-dimensional grid of literal data."
Victor Lombardi Ted Booth HK Dunston Andrew OtwellBridging Design and Climate Science
February 14, 2024
"If they can do it, so can I – that’s the takeaway from these transformation stories."
Lada GorlenkoTheme 1: Intro
January 8, 2024
"It’s never too late to apologize. And don’t be afraid to say, I’m sorry."
Dan WillisEnterprise Storytelling Sessions
June 3, 2019
"It's wrong to expect people to want to do more work with less motivation."
Emily Williams Nora FioreWhen UX Research and Institutional Racism Collide: A Case Study
March 12, 2021
Latest Books All books
Dig deeper with the Rosenbot
What inexpensive methods can validate physical design concepts before investing in hardware?
How might cultural figures like Taylor Swift influence future movements demanding fair treatment of creators in AI?'
What causes the collapse of critique processes in teams using AI-generated design artifacts?