Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale

Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Dave Malouf
Closing Keynote: Amplify. Not Optimize.
2019 • DesignOps Summit 2019
Gold
Megan Blocker
Panel: Excellence in Impact
2024 • Advancing Research 2024
Gold
Steve Sanderson
Discussion
2015 • Enterprise UX 2015
Gold
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
John Paul de Guzman
10k Screens Later: How We Became a Data-Driven Design Organization
2024 • DesignOps Summit 2024
Gold
Bas Raijmakers, PhD (RCA)
What Design Research can Learn from Documentary Filmmaking
2022 • Advancing Research 2022
Gold
Jaime Creixems
Best Practices when Creating and Maintaining a Design System
2023 • Enterprise UX 2023
Gold
Melissa Tsang
From Insights to Action: Driving Business Values through DesignOps
2024 • DesignOps Summit 2020
Gold
Jon Temple
Panel: Stacks, Security, and Stakeholders: The Hidden Work of UXR Tool Procurement
2026 • Advancing Research 2026
Gold
Sarah Rink
Remote User Research: Dos and Don'ts from the Virtual Field
2020 • Advancing Research Community
Erin Malone
Understanding the past to prepare for the future
2024 • Rosenfeld Community
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Marjorie Stainback
What Research Ops Professionals Have Learned from COVID-19
2020 • Advancing Research Community
Gordon Ross
12 Months of COVID-19 Design and Digital Response with the British Columbia Government
2021 • Civic Design 2021
Gold
Nathan Reiff
Research, from Unimaginable to Presently Possible: A Future-Casting Sticky-Note Sprint
2026 • Advancing Research 2026
Gold

More Videos

Kate Towsey

"Building ecosystems of interconnected tools that work seamlessly is crucial for modern, fast-moving organizations."

Kate Towsey Basel Fakhoury Oren Friedman Graham Gardner

Participant Recruitment and Management Tools

March 12, 2026

Jennifer Kong

"The unpredictable black box nature of LLMs means we had to zoom in on small problems first before seeing the bigger picture."

Jennifer Kong

Journeying toward AI-assisted documentation in healthcare

June 5, 2024

Kevin M. Hoffman

"Project management is the ecosystem connecting everyone to deliver the best branded dog communication technology."

Kevin M. Hoffman

Theme 2: Enterprise Team Journey

June 3, 2019

Amy Marquez

"You have to understand how the business actually makes money to align UX with it."

Amy Marquez

INVEST: Discussion

June 15, 2018

Gretchen Anderson

"Silicon Valley feels fast, but when the grid goes down, it can be catastrophic."

Gretchen Anderson

Scaling the Human Center

June 8, 2017

April Reagan

"The pandemic and climate change have shown how tightly interwoven and global our challenges truly are."

April Reagan

Look, Think, Act: The Futures-Smart Design Organization

October 1, 2021

Dante Guintu

"We matched designers with new peer buddies outside their usual domains to encourage learning and networking."

Dante Guintu

How to Crush the Talent Crunch

September 8, 2022

Jaime Creixems

"Try to make your component hierarchy as flat as possible because tall, nested structures become hard to navigate."

Jaime Creixems

Best Practices when Creating and Maintaining a Design System

June 7, 2023

Billy Carlson

"Wireframes are like an idea in time you can use for conversation."

Billy Carlson

Principles of Team Wireframing

October 2, 2023