Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Rebecca Gimenez
Work in Progress: Service Design at Airbnb
2024 • Advancing Service Design 2024
Gold
Andrew Webster
Scaling Design Capability: How Involved Should You Be?
2021 • DesignOps Summit 2021
Gold
Sharon Bautista
Time to Make the Donuts: How User Research Helped Bridge Disparate Teams
2024 • Enterprise Experience 2020
Gold
Kaitlin Tasker
Fast and Fearless Inclusive Research
2023 • Advancing Research 2023
Gold
Alba Villamil
Stereotyped by Design: Pitfalls in Cross-Cultural User Research
2020 • Advancing Research 2020
Gold
Jake Burghardt
Stop wasting research: Create new value with insight summaries
2025 • Rosenfeld Community
Nathan Shedroff
How Will Design be Taught When the Schools Shut Down?
2026 • Rosenfeld Community
Frances Yllana
The Big Question about Impact: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Frances Yllana
DesignOps Exposed: What do our peers really think of us?
2025 • DesignOps Summit 2025
Gold
Jessamyn Edwards
Surviving Your UX Career in Enterprise Design
2021 • Enterprise Community
Emily DiLeo
Stronger Together: Lessons Learned from UX Research Ops
2024 • DesignOps Summit 2024
Gold
Spencer L. A. Stultz
Why Social Justice Frameworks are Necessary for Successful DEI/JEDI Initiatives
2023 • DesignOps Summit 2023
Gold
Ovetta Sampson
Turning UX Passion into Real Product Influence
2023 • Enterprise UX 2023
Gold
Lena Shenkarenko
Collaborative Wireframing for Creating Team Alignment and Shipping Better Products
2020 • DesignOps Summit 2020
Gold
Jon Temple
Panel: Stacks, Security, and Stakeholders: The Hidden Work of UXR Tool Procurement
2026 • Advancing Research 2026
Conference
Kurt McCulloch
Faster alone, further together: Rebuilding collaboration in the age of AI research
2026 • Advancing Research 2026
Conference

More Videos

Giff Constable

"Everyone bounces off this at first, but you just have to start, be humble, and get better with practice."

Giff Constable

Financial fluency for product leaders: AMA with Giff Constable

April 11, 2024

Kritika Yadav

"Users frequently felt overwhelmed and misunderstood; this was a significant barrier to a positive shopping experience."

Kritika Yadav

Optimizing AI Conversations: A Case Study on Personalized Shopping Assistance Frameworks

June 10, 2025

Alla Weinberg

"Deep listening means hearing not just what is said but what isn’t said, especially for people with different language or cultural backgrounds."

Alla Weinberg

How to Build and Scale Team Safety

January 8, 2024

Harry Brignull

"User research then focused on basic usability problems like removing reset buttons that wiped forms."

Harry Brignull Mark Leiser Robert Stribley

Beyond Clicks and Tricks: Why deceptive design has grown into a regulatory faultline

January 16, 2026

Kit Unger

"These products are often part of an ecosystem of other complex products."

Kit Unger

Theme 2: Introduction

June 10, 2021

Sofia Quintero

"There are almost two languages in organizations: what we say and what we really mean."

Sofia Quintero

Beyond Tools: The Messy Business of Implementing Research Repositories

March 10, 2022

Jeff Gothelf

"You want to move from ‘make me a thing’ to ‘change customer behavior.’ That’s where the magic is."

Jeff Gothelf

The Intersection of Lean and Design

January 10, 2019

Kit Unger

"It can be very uncomfortable to be the only woman in that room."

Kit Unger Lada Gorlenko

Theme 3 Intro

June 10, 2022

Jorge Arango

"I resist premature optimization of tagging and structure; I want to be 80% there rather than perfect and distracted."

Jorge Arango

The Best of Both Worlds: How to Integrate Paper and Digital Notes (1st of 3 seminars)

April 5, 2024