Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

The Five Dysfunctions of Democratized Research at Scale
Gold
Monday, March 30, 2020 • Advancing Research 2020
Share the love for this talk
The Five Dysfunctions of Democratized Research at Scale
Speakers: Leisa Reichelt
Link:

Summary

As more organizations embrace research we move on to the second generation question – how do we scale this user centered behavior to support demand? There are fairly systemic issues that can undermine our best intentions. At best, it can render our research wasteful and inefficient, and at worst it can introduce significant risks in the decision making that our teams make. This talk considers five of the most likely challenges you’ll rub up against – speed, silos, teams, numbers and failure to mature. We’ll then discuss what causes these and what you can do to mitigate risks for your organization.

Key Insights

  • Democratizing research is more than encouraging everyone to talk to customers; it requires activating existing team tools thoughtfully.

  • There are two distinct jobs in customer research: building empathy for diverse contexts and gathering evidence to validate decisions.

  • Speed pressure often leads teams to cut corners in participant recruitment and data analysis, risking biased and invalid research.

  • Silo dysfunction arises when teams focus narrowly on their feature without considering the broader user context, increasing false positives.

  • Weaponizing research happens when teams use data combatively in internal disputes rather than as a tool for collaboration and understanding.

  • Quantitative data is often privileged because numbers are easier to digest, but mixed methods that combine qualitative insights yield richer results.

  • Early-stage usability testing with just one user can be beneficial to get started, but should not be the permanent approach for mature teams.

  • Risk management framing helps match research effort to the potential impact of decisions, optimizing resource use.

  • Involving product managers, designers, and engineers together in quality conversations enhances shared understanding and decision making.

  • Focusing on end-to-end user journeys rather than isolated features helps teams avoid narrow perspectives and create better product experiences.

Notable Quotes

"More people being closer to customers is very likely to help us achieve better products and services."

"Speed matters, but saving time in recruitment and analysis can lead to invalid and biased research."

"A developer pretending to be a product manager is not reliable participant data."

"People can make up an opinion about anything if asked, which leads to the query effect."

"False positives in research give teams misleading evidence that their solutions are working."

"Designers are not trusted unless they bring data to the table."

"Research should reduce the risk of investing in the wrong place or in the wrong way."

"Testing one user is 100 percent better than testing none, but it’s not a strategy forever."

"Qualitative research gives us the richer, more nuanced and complex conversations."

"Product managers ultimately make the call about research investments and prioritization."

Ask the Rosenbot
Erin Weigel
Failure Friday #6: 90% of Everything I Do is Either Broken or Pointless
2025 • Rosenfeld Community
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Design at Scale 2021
Gold
Craig Villamor
Resilient Enterprise Design
2017 • Enterprise Experience 2017
Gold
Nalini P. Kotamraju
Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"
2021 • Advancing Research 2021
Gold
Marc Fonteijn
Increase your confidence, influence, and impact (through a Professional Community)
2024 • Advancing Service Design 2024
Gold
Smitha Papolu
Theme 3 Discussion
2019 • Enterprise Experience 2019
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Jilanna Wilson
Distributed Design Operations Management
2019 • DesignOps Summit 2019
Gold
Russell Blair
Killing the blank page
2024 • Designing with AI 2024
Gold
Sarah Kinkade
Design Management Models in the Face of Transformation
2022 • Design at Scale 2022
Gold
Giff Constable
Financial fluency for product leaders: AMA with Giff Constable
2024 • Rosenfeld Community
John Cutler
The Alignment Trap
2023 • Design in Product 2023
Gold
Alëna Iouguina
Designing Systems at Scale
2018 • DesignOps Summit 2018
Gold
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community

More Videos

Sam Proulx

"Alternative navigation users pick different tools based on the task and how they feel at that moment, not just one technology."

Sam Proulx

SUS: A System Unusable for Twenty Percent of the Population

December 9, 2021

Michael Land

"You don’t want so much governance that people can’t be creative, but you need just enough to help their process."

Michael Land

Establishing Design Operations in Government

February 18, 2021

Shipra Kayan

"We had to push through analysis paralysis and trust our guts as customer-facing teams that knew the business."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Ian Swinson

"Innovation is about being fearless, breaking rules, and refusing to stop at the way things have always been done."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Isaac Heyveld

"Emotional intelligence is key to navigating situations with senior leaders multiple grades above you."

Isaac Heyveld

Expand DesignOps Leadership as a Chief of Staff

September 8, 2022

Amy Evans

"What if product and business actually manage designs and deliver digital communications?"

Amy Evans

How to Create Change

September 25, 2024

Kate Koch

"Super strength is the resilience and endurance to partner and drive strategic impact backed by customer obsession."

Kate Koch Prateek Kalli

Flex Your Super Powers: When a Design Ops Team Scales to Power CX

September 30, 2021

Dave Gray

"Most enterprises are not emotionally safe places; people can’t just leave emotions at the door."

Dave Gray

Liminal Thinking: Sense-making for systems in large organizations

May 14, 2015

Matt Duignan

"Durable insights can emerge organically based on usage and referencing patterns in the system."

Matt Duignan

Atomizing Research: Trend or Trap

March 30, 2020