Rosenverse
The Five Dysfunctions of Democratized Research at Scale

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Laura Weiss
Turn Down the Heat: 3 Ways to Handle Conflict in the Moment
2024 • Rosenfeld Community
Jamie Beck Alexander
How can you find your role in climate?
2024 • Climate UX Interest Group
Jeff Gothelf
Innovation Studios: the Engines of Enterprise Experimentation
2015 • Enterprise UX 2015
Gold
Bria Alexander
Opening Remarks Day 2
2024 • Advancing Research 2024
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps Summit 2024
Gold
Nina Jurcic
The Design System Rollercoaster: From Enabler and Bottleneck to Catalyst for Change
2023 • DesignOps Summit 2023
Gold
Jorge Arango
Design as an Antidote to VUCA
2019 • Enterprise Community
Daniel J. Rosenberg
Designing with and for Artificial Intelligence
2022 • Enterprise Community
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Gold
Bob Baxley
Leading with Design Operations Past and Present
2019 • DesignOps Community
Amber Knabl
Empowering innovation: The critical role of inclusive product development in the AI era
2024 • Designing with AI 2024
Gold
Tatyana Mamut
Opening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart
2019 • Enterprise Experience 2019
Gold
Shelby Switzer
Making Space for Community Knowledge-sharing in a Distributed World
2021 • Civic Design 2021
Gold
Natalie Dunbar
DesignOps and Content Strategy: Envisioning the Future Together
2021 • DesignOps Summit 2021
Gold
Smitha Papolu
Theme 3 Discussion
2019 • Enterprise Experience 2019
Gold

More Videos

Daniel Gloyd

"It’s psychopathic behavior to know about harm in a product’s sourcing and ignore it rather than seek transparency."

Daniel Gloyd

Designing Warmth

February 26, 2025

David Sternberg

"Human behavior is anything but linear, rational, or predictable."

David Sternberg

Uncovering the hidden forces shaping user behavior

July 17, 2025

Karen McGrane

"The biggest challenge with LLMs was keeping them on task instead of writing their own Reddit posts."

Karen McGrane Jeff Eaton

AI for Information Architects: Are the robots coming for our jobs?

November 21, 2024

John Cutler

"If you need to cross a dangerous river, you don’t paddle straight into it; you use the opposing force of the river to move forward."

John Cutler

Oxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)

December 3, 2024

Erin Hoffman-John

"In a hyperconnected world, design leadership is under constant personal inspection, making kindness and empathy really hard."

Erin Hoffman-John

This Game is Never Done: Design Leadership Techniques from the Video Game World

November 6, 2017

Matt Duignan

"We treat foundational and tactical research differently, using more flexible structures where rigid reports don’t fit."

Matt Duignan

HITS, Microsoft's internal human insight system: From research library to living body of knowledge

July 16, 2019

Peter Merholz

"Without a shared purpose, a team is simply a collection of individuals."

Peter Merholz

Design at Scale is People!

June 11, 2021

Davis Neable

"The experience vision provided a tangible goal that everyone could work towards, focusing on the user's experience rather than the technology."

Davis Neable Guy Segal

How to Drive a Design Project When you Don’t Have a Design Team

June 10, 2021

Catt Small

"Early involvement of product helps you frame the problem better and co-create a solution that others will buy into."

Catt Small

Craft a Vision that Actually Gets Shipped

April 30, 2026