Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Jessamyn Edwards
Surviving Your UX Career in Enterprise Design
2021 • Enterprise Community
Christian Crumlish
AMA with Christian Crumlish, author of Product Management for UX People
2022 • Enterprise Community
Alla Weinberg
Design Teams Need Psychological Safety: Here’s How to Create It
2022 • DesignOps Summit 2022
Gold
Laura Weiss
Turn Down the Heat: 3 Ways to Handle Conflict in the Moment
2024 • Rosenfeld Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Design at Scale 2022
Gold
Gretchen Anderson
Scaling the Human Center
2017 • Enterprise Experience 2017
Gold
Clemens Janssen
Efficiently Scaling Research as a Team of One
2023 • Advancing Research 2023
Gold
Andrea Gallagher
The Problem Space
2019 • Advancing Research Community
Kristin Sundermeyer
Design Ops Metrics
2021 • DesignOps Summit 2021
Gold
Adam Cutler
Discussion
2016 • Enterprise UX 2016
Gold
Harry Max
Failure Friday #5: Lessons from a SaaS Design Failure
2025 • Rosenfeld Community
Marc Fonteijn
First Insights from the 2025 Service Design Salary(+) Report
2024 • Advancing Service Design 2024
Gold
Ariel Kennan
Theme 2 Intro
2021 • Civic Design 2021
Gold
Mitchell Bernstein
Organizing Chaos: How IBM is Defining Design Systems with Sketch for an Ever-Changing AI Landscape
2021 • DesignOps Summit 2021
Gold
Vincent Brathwaite
Opener: Past, Present, and Future—Closing the Racial Divide in Design Teams
2020 • DesignOps Summit 2020
Gold
Dr. Jamika D. Burge
A Genuine Conversation about the Future of UX Research
2024 • Advancing Research Community

More Videos

Sam Proulx

"More ways to contact support—chat, email, phone—are essential because different disabilities require different options."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Ignacio Martinez

"We’ve had to retire the studio development category because it wasn’t prioritized due to project work demands."

Ignacio Martinez

Fair and Effective Designer Evaluation

September 25, 2024

Sarah Kinkade

"We needed to revise our plan; we couldn’t just keep adding talented folks and hope everything falls into place."

Sarah Kinkade Mariana Ortiz-Reyes

Design Management Models in the Face of Transformation

June 8, 2022

Daniel Gloyd

"Belonging means understanding the shape, size, and cause of your community and your unique purpose in it."

Daniel Gloyd

Warming the User Experience: Lessons from America's first and most radical human-centered designers

May 9, 2024

Patrick Boehler

"We spend millions on content that nobody wanted and that didn’t actually help people navigate their lives."

Patrick Boehler

Fishing for Real Needs: Reimagining Journalism Needs with AI

June 10, 2025

Andy Barraclough

"How do you build in the context of what you specifically need rather than having tools drive the research process? That’s the future."

Andy Barraclough Betsy Nelson

From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme

September 23, 2024

Alexandra Schmidt

"How do we as designers connect more deeply with the policy world? That’s still a developing thing."

Alexandra Schmidt

Why Ethics Can't Save Tech

November 18, 2022

Louis Rosenfeld

"A lot of DevOps is about empathy — doing our piece but thinking about the larger connection."

Louis Rosenfeld

Discussion: What Operations can teach DesignOps

November 6, 2017

Mitchell Bernstein

"Every decision in a design system is like a puzzle piece; together they form a bigger picture that evolves over time."

Mitchell Bernstein

Organizing Chaos: How IBM is Defining Design Systems with Sketch for an Ever-Changing AI Landscape

September 29, 2021