Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Michelle Bejian Lotia
Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
2023 • Advancing Research 2023
Gold
Nicole Aleong
What UX research can learn from other research practices [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Noel Lamb
Cultivating Business Partnerships to Grow Research Ops
2022 • Advancing Research Community
Kelly Goto
Emotion Economy: Ethnography as Corporate Strategy
2015 • Enterprise UX 2015
Gold
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
World Usability Day Panel Discussion
2022 • DesignOps Community
Theresa Slate
Why Changing Hearts & Minds Doesn’t Work When Promoting DE&I Efforts, but Checklists Do
2023 • DesignOps Summit 2023
Gold
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Dave Hora
Advice for Establishing Research
2022 • Advancing Research Community
Liz Ebengo
The Burden on Children: The Cost of Insufficient Post-Conflict Services and Pathways Forward
2024 • Advancing Service Design 2024
Gold
Louis Rosenfeld
Do you want to work on climate? (Climate UX Discussion Series)
2023 • Climate UX Interest Group
Malini Rao
Lessons Learned from a 4-year Product Re-platforming Journey
2021 • Design at Scale 2021
Gold
Séamus Byrne
Aligning Teams with Choreography
2024 • Enterprise Experience 2020
Gold
David Eisner
From prototype to production: Vibe coding design for real engineering systems
2026 • Designing with AI 2026
Conference
Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
James Lang
Hopeful Futures for UX Research
2026 • Rosenfeld Community

More Videos

Yunyan Li

"Visual specs must include behavior, styles, accessibility, and edge cases so developers build accurately."

Yunyan Li Anna Le Jen Kim

UX Best Practices

June 11, 2021

Allan Lowson

"We went from stitching wireframes to actually designing in code—no more simulacra."

Allan Lowson

Rehashing the Double Diamond: Collaborating across functions with AI-assisted prototyping

June 9, 2026

Denise Jacobs

"I appreciate your allyship when I'm in the room, but I appreciate it more when I'm not in the room."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Renee Bouwens

"We’re capturing requirements usually without research, design has to be 100% baked before development starts."

Renee Bouwens

Landing Product Impact: Aligning Research as a Foundational Driver for Delivering the World’s Best Products

December 15, 2023

Jemma Ahmed

"It’s not about power, it’s about guidance—helping people collect and use good data, even in qualitative settings."

Jemma Ahmed Steve Carrod Chris Geison Dr. Shadi Janansefat Christopher Nash

Democratization: Working with it, not against it [Advancing Research Community Workshop Series]

July 24, 2024

Sam Ladner

"You have to have a point of view, and you have to lead in order to get that impact in strategic foresight."

Sam Ladner

How Research Can Drive Strategic Foresight

March 9, 2022

Yoel Sumitro

"Students cannot be taught what they need to know; they can only be coached to absorb it through demonstration."

Yoel Sumitro

Actions and Reflections: Bridging the Skills Gap among Researchers

March 9, 2022

Dave Hora

"Helping teams is much more like managing a children’s party than engineering a bridge."

Dave Hora

Research in the Face of Complexity: New Sensibility for New Situations

August 27, 2025

Daniel J. Rosenberg

"Human beings understand graphical user interfaces as composed of objects and actions—this grammar is key to AI design."

Daniel J. Rosenberg

Designing with and for Artificial Intelligence

August 11, 2022