The Five Dysfunctions of Democratized Research at Scale
Summary
As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.
Key Insights
-
•
Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.
-
•
There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.
-
•
Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.
-
•
Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.
-
•
Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.
-
•
Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.
-
•
Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.
-
•
Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.
-
•
Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.
-
•
Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.
Notable Quotes
"More people being closer to customers is very likely to help us achieve better products and services."
"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."
"A developer pretending to be a product manager is not reliable participant data."
"People can make up an opinion about anything if asked, which leads to the query effect."
"False positives in research give teams misleading evidence that their solutions are working."
"Designers are not trusted unless they bring data to the table."
"Research should reduce the risk of investing in the wrong place or in the wrong way."
"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."
"Qualitative research gives us the richer, more nuanced and complex conversations."
"Product managers ultimately make the call about research investments and prioritization."
Or choose a question:
More Videos
"Craft isn't gonna help you feel more fulfilled, influential or powerful. It cannot meet that need."
Jess GrecoClaiming your power: Practical tools for amplifying your unique voice
March 13, 2025
"With bespoke UI, you don’t present every single path to the user; you let the right tool appear immediately for the job."
Josh Clark Veronika KindredSentient Scenes and Radically Adaptive Experiences
June 11, 2025
"Survey tools are no longer just survey tools; the boundaries between categories are increasingly blurry."
Aleksandra Korczynska Caroline Jarrett Justyna ParmeeSurvey Tools
March 12, 2026
"When I started, the team didn’t understand what the point of UX on the back end was; I had to win their hearts and minds."
Anat Fintzi Rachel MinnicksDelivering at Scale: Making Traction with Resistant Partners
June 9, 2022
"When you’re in a culture that celebrates complexity, making things simple is still a very hard thing to do."
Peter MorvilleThe Architecture of Understanding
May 13, 2015
"Process and discipline got thrown out the window, but we really need new processes that incorporate these new AI tools."
Paul FordNew work, new words: A glossary for AI
June 9, 2026
"We’ll hear about creating healthy cultures and ways of working across many organizational silos, including vendors."
Ariel KennanTheme Two Intro
November 17, 2022
"Don’t forget cars are fun—get behind the wheel and enjoy a road trip once in a while; UX is meant to enhance that experience."
James RamptonThe Basics of Automotive UX & Why Phones Are a Part of That Future
July 25, 2024
"Qualitative research is excellent for discovering needs but poor at quantifying their scale or priority."
Michaela MoraAdvanced Concept Testing Approaches To Guide Product Development and Business Decisions
March 11, 2022