Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Kathleen Asjes
Research Democratization: the Good, the Bad and the Ugly
2022 • Advancing Research 2022
Gold
Jemma Ahmed
Theme 2 Intro
2022 • Advancing Research 2022
Gold
Bob Baxley
Theme 4: Intro
2024 • Enterprise Experience 2020
Gold
Mariesa Lenz
What Beekeeping Taught me about Product Teams
2025 • Rosenfeld Community
Dr. Jamika D. Burge
A Genuine Conversation about the Future of UX Research
2024 • Advancing Research Community
Phil Hesketh
Designing Accessible Research Workflows
2021 • DesignOps Summit 2021
Gold
Tracy McGoldrick
IBM User Experience Program—The What, Why and How
2021 • Advancing Research Community
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Amy Thibodeau
Opening Keynote: Process and Ambiguity
2019 • DesignOps Summit 2019
Gold
Kate Towsey
The State of ResearchOps: More Than Just Theory
2019 • DesignOps Community
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Sarah Barrett
The "How" of Enterprise Information Architecture
2023 • Enterprise UX 2023
Gold
Sheri Byrne-Haber
Accessibility at Scale
2021 • Design at Scale 2021
Gold
Rachael Dietkus, LCSW
Leading through the long tail of trauma
2022 • Enterprise Community
Melissa Eggleston
Practical People Skills for Building Trust on Teams and with Partners
2021 • Civic Design 2021
Gold
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold

More Videos

Brad Peters

"Even when we bring the data together, we’re still not necessarily synthesizing very well."

Brad Peters Anne Mamaghani

Short Take #1: UX/Product Lessons from Your Industry Peers

December 6, 2022

Lona Moore

"You don’t have to be a designer to participate; over 90% of our community are not design professionals."

Lona Moore

Scaling Design Beyond Designers

June 11, 2021

Josh Clark

"We can’t design for every possible outcome anymore. We have to anticipate fuzzy ranges of results and channel behavior accordingly."

Josh Clark Veronika Kindred

Sentient Design, AI, and the Radically Adaptive Experience (1st of 3 seminars)

January 15, 2025

Erin May

"75% of democratized studies don’t see full reporting mostly because stakeholders don’t have time for the analysis."

Erin May Roberta Dombrowski Laura Oxenfeld Brooke Hinton

Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org

March 10, 2022

Tara Tressel

"AI moderation did not make the research easier. It made it possible under real world constraints."

Tara Tressel

Investigating qualitative depth of AI-moderated interviews

March 10, 2026

Charles Lee

"Code is definitely considered the source of truth; we're working hard to make Figma an extension of that."

Charles Lee Jennie Yip

Building a New Home for the Atlassian Design System

October 22, 2020

Kristin Skinner

"Equipped with the right tools, we can truly meet students where they are."

Kristin Skinner

Five Years of DesignOps

September 29, 2021

Megan Blocker

"Without adoption, value is zero."

Megan Blocker

Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research

March 26, 2024

Shanti Mathew

"A standard framework for collecting and responding to family feedback at the organizational level can increase their voice’s impact."

Shanti Mathew Natalie Sims Natalia Radywyl

Civic Design at Scale: Introducing the Public Policy Layer Cake

December 9, 2021