Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Natalia Radywyl
Co-Designing New Power in Australia's Public Sector
2022 • Civic Design 2022
Gold
Matteo Gratton
Can Data and Ethics Live Together?
2021 • DesignOps Summit 2021
Gold
Karen Pascoe
Developing Experience Teams and Talent in the Enterprise
2016 • Enterprise UX 2016
Gold
Renee Reid
Becoming a ResearchH.E.R (Highly Enterprise Ready)
2019 • Enterprise Experience 2019
Gold
Simon Wardley
Maps and Topographical Intelligence
2019 • Enterprise Community
Bria Alexander
The Big Question about Resilience: A panel discussion
2024 • DesignOps Summit 2024
Gold
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 2: Evals are your moat
2025 • Rosenfeld Community
Frances Yllana
D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)
2022 • Civic Design 2022
Gold
Joanna Vodopivec
One Research Team for All - Influence Without Authority
2022 • Advancing Research 2022
Gold
Shipra Kayan
Make your research synthesis speedy and more collaborative using a canvas
2025 • Rosenfeld Community
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
2022 • DesignOps Community
Taylor Jennings
Research Debate Club
2026 • Advancing Research 2026
Conference
Bria Alexander
OKRs—Helpful or Harmful?
2022 • DesignOps Community
Silke Bochat
5 Antifragile Strategies for a DesignOps 2.0
2024 • DesignOps Summit 2024
Gold
Saara Kamppari-Miller
DesignOps for Inclusive Design and Accessibility
2022 • DesignOps Community
Ana Ferreira
Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries
2024 • DesignOps Summit 2020
Gold

More Videos

Christian Crumlish

"When something good happens, I want to name the person who did it and say yes, they passed through me. When something bad happens, I say let me and my team go look into it."

Christian Crumlish Wendy Johansson Rich Mironov Aditi Ruiz Adam Thomas

Afternoon Insights Panel

December 6, 2022

Michael Land

"In government, the org chart really matters – it influences who talks to whom and who gets to interface."

Michael Land

Establishing Design Operations in Government

February 18, 2021

Bria Alexander

"Sponsor sessions are not sales pitches. They offer incredibly high-quality content and are free to anyone to attend."

Bria Alexander Louis Rosenfeld

Welcome

September 8, 2022

Surya Vanka

"There are 8 billion creative humans on planet Earth, far beyond the number of trained designers."

Surya Vanka

Unleashing Swarm Creativity to Solve Enterprise Challenges

June 10, 2021

Cassandra Piester

"The most important step we often forget is moving on to the next problem."

Cassandra Piester

Developing and Deploying Your Design Operations Strategy

September 24, 2024

Ariel Kennan

"Our presenters have iterated on their talks to provide rich resources and new knowledge."

Ariel Kennan

Theme 2 Intro

December 9, 2021

Joshua Graves

"The amygdala acts as the brain’s smoke detector, smashing a button when it senses anything unsafe."

Joshua Graves

We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)

April 14, 2025

Peter Van Dijck

"Models are stateless; they have no memory and forget everything after each response."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!

July 30, 2025

Bria Alexander

"Sponsor sessions are completely free and offer content just as valuable as the main program sessions."

Bria Alexander

Opening Remarks

November 17, 2022