Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Melissa Eggleston
Practical People Skills for Building Trust on Teams and with Partners
2021 • Civic Design 2021
Gold
Harry Max
Prioritization for Leaders (2nd of 3 seminars)
2024 • Rosenfeld Community
Sam Ladner
How Research Can Drive Strategic Foresight
2022 • Advancing Research 2022
Gold
Taylor Klassman
Shaping the Next Era of UX Research: Collaborative Forum
2025 • Advancing Research 2025
Gold
Emilia Åström
Unlock Your Team’s Intelligence with Collaboration Design
2022 • Design at Scale 2022
Gold
Ovetta Sampson
Research in the Automated Future
2022 • Advancing Research 2022
Gold
Paula Bach
Improving Legacy Software: How Much Better Does it Have to Be?
2022 • Advancing Research 2022
Gold
Heidi Trost
To Protect People, You Have to Protect Information: A Human-Centered Design Approach to Cybersecurity
2025 • Rosenfeld Community
Cassandra Piester
Developing and Deploying Your Design Operations Strategy
2024 • DesignOps Summit 2024
Gold
Hana Nagel
Turning Research Ripples into Waves
2018 • DesignOps Summit 2018
Gold
Randolph Duke II
War Stories LIVE! Randy Duke II
2020 • Advancing Research 2020
Gold
Tim Parmee
Changing Our Design Pressure Points
2023 • DesignOps Summit 2023
Gold
Nathan Shedroff
Double Your Mileage: Use Your Research Strategically
2020 • Advancing Research 2020
Gold
Louis Rosenfeld
The Rosenbot and the Rosenverse: An AMA with Lou Rosenfeld
2024 • Designing with AI 2024
Gold
Cennydd Bowles
Responsible Design in Reality
2021 • Design at Scale 2021
Gold
Brendan Jarvis
It was the Best of Times. It was the Worst of Times.
2024 • DesignOps Summit 2024
Gold

More Videos

Randolph Duke II

"Carol was really grateful for our conversation."

Randolph Duke II

War Stories LIVE! Randy Duke II

March 30, 2020

Corey Nelson

"This is a moment to rethink identity beyond UX and get creative with income streams and career paths."

Corey Nelson Amy Santee

Layoffs

November 15, 2022

Landon Barnes

"You shouldn’t send another survey unless you’ve implemented changes and allowed time for behavior to adapt."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Amy Bucher

"Self-report is not the end all; people often don’t know why they behave certain ways, especially regarding automatic motivations like deeply ingrained mental models."

Amy Bucher

Harnessing behavioral science to uncover deeper truths

March 12, 2025

David Sternberg

"When you observe a quantum particle, its possible states collapse to one—just like user intent collapses to action."

David Sternberg

Uncovering the hidden forces shaping user behavior

July 17, 2025

Deanna Smith

"If you dug deeper into feedback, you'd find ways to address issues without a big change."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Jennifer Strickland

"Provocative prototypes provoke conversation and surface unspoken values."

Jennifer Strickland

Adopting a "Design By" Method

December 9, 2021

Rachel Posman

"COVID has maybe made us a better global team by forcing us to figure out how to work across time zones and communication preferences."

Rachel Posman John Calhoun

A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)

November 19, 2020

Gina Mendolia

"Service design is 10% connecting the dots and 90% creating the conditions that bring those dots closer together."

Gina Mendolia

Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems

December 3, 2024