Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Daniel J. Rosenberg
Digital Medicine Design
2019 • Enterprise Community
Sean McKay
Whole Product Thinking: Expanding beyond problem and solution space thinking
2024 • Rosenfeld Community
Nicole Bergstrom
AccessibilityOps: Moving beyond “nice to have”
2024 • DesignOps Community
Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Husani Oakley
Bias Towards Action: Building Teams that Build Work
2018 • Enterprise Experience 2018
Gold
Ana Maria Montero Barrantes
The Authentic UX Talent Show
2024 • Enterprise Experience 2020
Gold
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Adam Thomas
Survival Metrics – Making Change in a Fast, Data-Informed, and Politically Safe Way
2022 • Design in Product 2022
Gold
Natalia Radywyl
Co-Designing New Power in Australia's Public Sector
2022 • Civic Design 2022
Gold
Russell Blair
Killing the blank page
2024 • Designing with AI 2024
Gold
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Conference
Abbey Smalley
Today’s Design Ops and Programs Landscape & Career Paths
2023 • DesignOps Summit 2023
Gold
Emily Eagle
Can't Rewind: Radio and Retail
2019 • Enterprise Experience 2019
Gold
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Kate Stern
Scaling Learning for the Future
2022 • DesignOps Summit 2022
Gold
Gordon Ross
12 Months of COVID-19 Design and Digital Response with the British Columbia Government
2021 • Civic Design 2021
Gold

More Videos

Bria Alexander

"Slack has over 650 people actively talking and sharing experiences right now."

Bria Alexander

Opening Remarks

November 18, 2022

Hugh Dubberly

"Digital twins are replicas of dynamic systems that help both individuals and populations manage and improve operations."

Hugh Dubberly

Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving

June 19, 2019

April Reagan

"AI can be a great partner to humans if trained with fair and complete data, but we must be careful not to ship biased systems."

April Reagan

Look, Think, Act: The Futures-Smart Design Organization

October 1, 2021

Ricardo Martins

"Cluster analysis will definitely replace personas, the fictional personas of course."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Maria Skaaden

"How can developers and designers work together to create great products? That’s the question."

Maria Skaaden

Continuous Design: One eye on the horizon and the other on the next wave

November 8, 2018

Vanessa Varin

"Match the moment. Calibrate the altitude and the type of feedback to the context."

Vanessa Varin

Feedback: The Other F-Word

September 10, 2025

Jen Crim

"The team is passionate, creative, and innovative, building experiences that people love."

Jen Crim Jess Quittner Saritha Kattekola Alex Karr Gurbani Pahwa

Culture, DIBS & Recruiting

June 11, 2021

Patrick Boehler

"We’re moving from theory of change to theory of service: starting with what people actually need before creating anything."

Patrick Boehler

Fishing for Real Needs: Reimagining Journalism Needs with AI

June 10, 2025

Julie Baher

"At Stanford’s d.school, we got the religion: design thinking could be bigger than just about products."

Julie Baher

Culture Change—My Journey

May 14, 2015