The Five Dysfunctions of Democratized Research at Scale
Summary
Key Insights
-
•
Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.
-
•
There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.
-
•
Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.
-
•
Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.
-
•
Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.
-
•
Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.
-
•
Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.
-
•
Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.
-
•
Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.
-
•
Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.
Notable Quotes
"More people being closer to customers is very likely to help us achieve better products and services."
"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."
"A developer pretending to be a product manager is not reliable participant data."
"People can make up an opinion about anything if asked, which leads to the query effect."
"False positives in research give teams misleading evidence that their solutions are working."
"Designers are not trusted unless they bring data to the table."
"Research should reduce the risk of investing in the wrong place or in the wrong way."
"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."
"Qualitative research gives us the richer, more nuanced and complex conversations."
"Product managers ultimately make the call about research investments and prioritization."
Or choose a question:
More Videos
"My Roomba vacuums while I’m at work. It does its work out of my attention."
Christopher NoesselAI of the now: Designing for Agents
July 31, 2024
"UX research is a journey, not just a switch you flick on; many challenges you face have been experienced by others in the community."
Caroline VizeThe State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
March 9, 2022
"Telepathy as mind reading means proactively building a talent pipeline and improving recruitment processes."
Kate Koch Prateek KalliFlex Your Super Powers: When a Design Ops Team Scales to Power CX
September 30, 2021
"If someone asks you for a cup of coffee, here’s the cup of coffee. You did it. But always have proposals on the side for more."
Jessamyn EdwardsSurviving Your UX Career in Enterprise Design
December 2, 2021
"Democratization is not research anarchy where anybody can talk to any customer for any reason with no oversight."
Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher NashDemocratization: Working with it, not against it [Advancing Research Community Workshop Series]
July 24, 2024
"SAFE is essentially a software development methodology, and fitting experience design into it has been a constant trial and error."
Sharbani DharBreathing Room for Delight
January 8, 2024
"Measuring impact means linking research insights back to decisions and how they affect roadmaps and priorities."
Ned Dwyer Emily Stewart James WallisThe Intersection of Design and ResearchOps
September 24, 2024
"I want to know about bruises, lessons learned, not just best practices."
Bud CaddellTheme 2 Intro
September 30, 2021
"Using a mixture of behavioral science, analytics, and experiments helps prioritize changes that truly move KPIs."
Michael WeirMixed Methods and Behavioural Science
May 26, 2023
Latest Books All books
Dig deeper with the Rosenbot
What is 'vibe coding' and how does it reflect changing expectations for junior design and product roles?
How did AI factor into Rachel’s feelings about leaving the workforce and her current creative processes?
What core skills should UX researchers develop to thrive in the evolving AI-enhanced product environment?