Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold
Jon White
Unsticking Research for Better Information Flow
2026 • Advancing Research 2026
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Civic Design 2022
Gold
Erin Weigel
Real-world lessons to improve your conversion rates
2024 • Rosenfeld Community
Bilan Hashi
The Tension Between Story Collecting and Story Telling in Research
2021 • Advancing Research 2021
Gold
Tony Turner
Capturing Deep Insights
2021 • DesignOps Summit 2021
Gold
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
Andy Barraclough
From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme
2024 • DesignOps Summit 2024
Gold
Greg Petroff
Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian
2025 • Rosenfeld Community
JJ Kercher
A Roadmap for Maturing Design in the Enterprise
2018 • Enterprise Experience 2018
Gold
Dr Chloe Sharp
Using Evidence and Collaboration for Setting and Defending Priorities
2023 • Design in Product 2023
Gold
Brennan Hartich
Communicating and Establishing DesignOps as a New Function
2018 • DesignOps Summit 2018
Gold
Russ Unger
Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started
2016 • Enterprise UX 2016
Gold
Sara Conklin
Exit Interview: 20 Years of Tech, One Very Big Bet, and a Lot of Heat Pumps
2026 • Rosenfeld Community
Chris Moses
Stretching the Definition of DesignOps with Product Development
2018 • DesignOps Summit 2018
Gold
Jennifer Strickland
Adopting a "Design By" Method
2021 • Civic Design 2021
Gold

More Videos

Jess Greco

"If I had waited for someone else to give me permission, I'd still be waiting. We'd all be waiting."

Jess Greco

Claiming your power: Practical tools for amplifying your unique voice

March 13, 2025

Josh Clark

"Radically adaptive experiences are conceived and compiled in real time, responding to context and intent."

Josh Clark Veronika Kindred

Sentient Scenes and Radically Adaptive Experiences

June 11, 2025

Aleksandra Korczynska

"The key to combating survey fatigue is short surveys triggered at the right context, making respondents feel listened to and valued."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Anat Fintzi

"We consolidated over ten tools into one supply chain management system with a user-led team of 4 UXers, 2 product managers, and 6 engineers."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Peter Morville

"When you’re in a culture that celebrates complexity, making things simple is still a very hard thing to do."

Peter Morville

The Architecture of Understanding

May 13, 2015

Paul Ford

"Everything is disposable now; we’re drowning in prototypes and reviewmiliation—so much work shows up overnight."

Paul Ford

New work, new words: A glossary for AI

June 9, 2026

Ariel Kennan

"We’ll hear about creating healthy cultures and ways of working across many organizational silos, including vendors."

Ariel Kennan

Theme Two Intro

November 17, 2022

James Rampton

"Rivian never allowed Apple CarPlay or Android Auto because they want to own the experience and data."

James Rampton

The Basics of Automotive UX & Why Phones Are a Part of That Future

July 25, 2024

Michaela Mora

"No research method is perfect; use a mix and triangulate results depending on whether you want to predict or explain."

Michaela Mora

Advanced Concept Testing Approaches To Guide Product Development and Business Decisions

March 11, 2022