Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Jennifer Bolduc
What's involved with getting people back to work?: A panel discussion
2021 • DesignOps Community
Russ Unger
Onboarding: The Ecosystem, not the Afterthought
2017 • DesignOps Summit 2017
Gold
Lais de Almeida
Designing Data Services
2024 • Advancing Service Design 2024
Gold
Uday Gajendar
From AI to Zeitgeist: Theory as the design antidote to AI hype
2025 • Rosenfeld Community
Xenia Adjoubei
Empowering Communities Through the Researcher in Residence Program
2023 • Advancing Research 2023
Gold
Bria Alexander
OKRs—Helpful or Harmful?
2022 • DesignOps Community
Bria Alexander
Theme Two Intro
2023 • DesignOps Summit 2023
Gold
Alla Weinberg
Design Teams Need Psychological Safety: Here’s How to Create It
2022 • DesignOps Summit 2022
Gold
John Calhoun
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
2024 • DesignOps Summit 2024
Gold
Sheryl Cababa
Living in the Clouds: Adopting a Systems Thinking Mindset
2023 • Enterprise UX 2023
Gold
Steve Portigal
Discussion
2015 • Enterprise UX 2015
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Conference
Theresa Slate
Why Changing Hearts & Minds Doesn’t Work When Promoting DE&I Efforts, but Checklists Do
2023 • DesignOps Summit 2023
Gold
Dan Willis
Enterprise Storytelling Sessions
2015 • Enterprise UX 2015
Gold
Kyria Stephens
Power to Heal: Civic Design in the Aftermath of Tragedy
2022 • Civic Design 2022
Gold

More Videos

Frances Yllana

"Hope locates itself in the spaciousness of uncertainty and the belief that what we do matters even if outcomes are unknown."

Frances Yllana

DesignOps–Leading the Path to Parity

April 27, 2023

Kim Holt

"Always remember the power of your values in times of doubt and change; lean into what your team really stands for."

Kim Holt Emma Wylds Pearl Koppenhaver Maisee Xiong

A Salesforce Panel Discussion on Values-Driven DesignOps

September 8, 2022

Amy Thibodeau

"We measure qualitative sentiment alongside quantitative metrics to design how we support the team."

Amy Thibodeau

Opening Keynote: Process and Ambiguity

October 23, 2019

Laura Gatewood

"Slack clips can add tone and body language that is too hard to capture in the written word."

Laura Gatewood Laine Prokay

Beyond Buzzwords: Adding Heart to Effective Slack Communication

September 23, 2024

Patrick Boehler

"Structure beats free form interaction—systematic query frameworks are essential to avoid pitfalls."

Patrick Boehler

Fishing for Real Needs: Reimagining Journalism Needs with AI

June 10, 2025

Marc Fonteijn

"We design a community that would be missed if it were gone for our service design leaders."

Marc Fonteijn Ru Butler

Increase your confidence, influence, and impact (through a Professional Community)

December 3, 2024

Bria Alexander

"Today's theme around researchers thriving in the organization is led by Steve Portico."

Bria Alexander

Opening Remarks

January 8, 2024

Laura Weiss

"People tend to support what they actively help create."

Laura Weiss

There is No Playbook: Leader as Coach During Challenging Times

April 26, 2024

Tony Turner

"A taxonomy system is basically a way to tag different findings and reports so you can find relationships later."

Tony Turner

Capturing Deep Insights

September 30, 2021