Summary
In this talk, Jon explores the often misunderstood relationship between data scientists and designers and how their collaboration can create significant customer value. Drawing on his experience at Intuit, he discusses developing an enterprise-wide A/B testing platform that initially struggled with adoption until working closely with designers to identify key leading metrics like platform ease of use and time to deployment, rather than just lagging outcomes. Jon also recounts a retail case where overwhelming daily data was distilled by studying top-performing store managers in context, enabling the team to design a focused dashboard that increased sales conversion by 10%. Lastly, he highlights eBay's approach to extracting meaningful signals from vast transaction data by partnering designers with 300 data analysts to streamline workflows and reduce insight time from weeks to a day, democratizing access across the company. He concludes that integrating data science fully into product teams alongside design and development creates more delightful, insight-driven products.
Key Insights
-
•
Less than 1% of global data is used to generate human insights, showing the value in efficient data utilization.
-
•
A data scientist combines statistical skills, hacking mindset, and entrepreneurial thinking to solve complex problems.
-
•
At Intuit, pairing designers with data scientists uncovered that ease of use and awareness were key to increasing adoption of an A/B testing platform.
-
•
Lagging indicators like number of experiments performed or features deployed are insufficient alone; leading indicators such as time to consumption and user awareness better drive adoption.
-
•
Ethnographic studies can reveal user behaviors and misalignments in what enterprise teams measure versus what actually drives business outcomes.
-
•
Studying the best-performing store leaders on the floor uncovered critical behaviors that informed a focused dashboard replacing overwhelming 424-page daily reports.
-
•
A simple data product tracking live customer-to-employee ratios, transaction progress, staff availability, and service queue length helped retail store managers improve performance and customer experience.
-
•
eBay’s approach to ‘signal extraction’ from massive data involves time series analysis, decision trees, and close designer-analyst collaboration to cut insight delivery from weeks to a day.
-
•
Democratizing data insights across an enterprise empowers broader decision-making beyond traditional analyst-only access.
-
•
Integrating data science as a core pillar alongside design, development, and product management can transform teams into more effective problem solvers and innovators.
Notable Quotes
"Since we started this presentation, enough data has been collected to fill everyone’s hard drive about 500,000 times, and fewer than 1% is ever used for insights."
"Data scientists are part stats geek, part hacker, and part entrepreneur."
"Develop deep customer empathy — really understand who’s using a product and why — walk a day in their shoes."
"Rapid experimentation with customers is key. Iterate, iterate, iterate."
"It doesn’t matter what the insight is if no one does anything with it."
"We realized we were measuring the wrong things — we focused on lagging indicators rather than the leading ones we could influence."
"Surprisingly, many developers didn’t even know our testing platform existed."
"By studying the best store leaders, we learned patterns that informed a dashboard reducing 424 pages of data to a few key, actionable metrics."
"eBay cut the time it took to get answers from weeks to one day by integrating design with data analysis workflows."
"Bringing data science into the core product process is like adding a fourth leg to the stool — design, development, product management, and data together create delight."
Or choose a question:
More Videos
"Decision makers often have to try to piece all the pieces of the puzzle together themselves."
Brad Peters Anne MamaghaniShort Take #1: UX/Product Lessons from Your Industry Peers
December 6, 2022
"Rituals symbolize community identity and shared values, helping scale human-centered habits across boundaries."
Lona MooreScaling Design Beyond Designers
June 11, 2021
"Radically adaptive experiences bend to your current wants and needs, creating unique, unpredictable journeys."
Josh Clark Veronika KindredSentient Design, AI, and the Radically Adaptive Experience (1st of 3 seminars)
January 15, 2025
"People will talk to customers whether you want them to or not. The question is how to make it a better experience."
Erin May Roberta Dombrowski Laura Oxenfeld Brooke HintonDistributed, Democratized, Decentralized: Finding a Research Model to Support Your Org
March 10, 2022
"From setup to analysis, AI moderation still requires a good understanding of what quality research looks like and how to interpret qualitative data."
Tara TresselInvestigating qualitative depth of AI-moderated interviews
March 10, 2026
"Kate said she considers live code to be the source of truth because she can't trust the design site due to inconsistencies."
Charles Lee Jennie YipBuilding a New Home for the Atlassian Design System
October 22, 2020
"Personalization is not just a trend; it’s a necessity."
Kristin SkinnerFive Years of DesignOps
September 29, 2021
"Hoard and reuse your jewels. Those proven frameworks can amplify every hour of your work."
Megan BlockerGetting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research
March 26, 2024
"Even the best-designed public-facing tools need operational and policy support to succeed."
Shanti Mathew Natalie Sims Natalia RadywylCivic Design at Scale: Introducing the Public Policy Layer Cake
December 9, 2021
Latest Books All books
Dig deeper with the Rosenbot
What role do research projects with customizable data inputs play in Survicate's new product?
What features does Rosenbot offer to help UX learners deepen their knowledge through follow-up prompts?
How can large enterprises design pilot programs that effectively test new research tools without excessive delays?