Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Research Operations at Scale

Gold
Tuesday, November 7, 2017 • DesignOps Summit 2017
Share the love for this talk
Research Operations at Scale
Speakers: Christian Rohrer
Link:

Summary

Research is an inherent part of design, and operationalizing how insights are generated and used is one of the more challenging aspects of design operations. Overcoming the lack of good tools, taming the complexity of research at scale, and navigating the world of different forms of inquiry are just a few of the obstacles to overcome. That said, we are starting to see a range of benefits from implementing research operations in modern design organizations.

Key Insights

  • Banks like Capital One must manage research amid strict regulations and risk, requiring innovative recruiting and study designs.

  • Specialized research operations and curricula teams at Capital One remove procedural inefficiencies and ensure compliance.

  • Risk Management at Capital One includes embedded 'risk-sharper' roles that guide research teams through compliance challenges.

  • 'Wine and dine' sessions enable rich qualitative insights by combining customer interaction with large employee participation, without traditional focus group pitfalls.

  • User research approaches must vary by business context: B2C, B2B, and internal IT differ greatly in user access and needs.

  • Agile’s dominance in software development creates challenges for user-centered design integration, requiring adaptations or use of Lean methodologies.

  • Qualitative research provides rich user insight and inspiration for designers, while executives rely more on quantitative data for decision-making.

  • The PURE method formalizes usability evaluation through expert ratings, producing measurable scores that drive executive action and product improvement.

  • The user experience talent pool is small and highly sought after, making it critical to structure roles to attract and retain specialists rather than expecting 'unicorns.'

  • Effective knowledge management for UX research must support the entire insight lifecycle—from ideation to product impact—to avoid losing valuable learnings.

Notable Quotes

"Capital One is really a bank ... it’s out to change banking for good by becoming a tech company that just happens to do banking."

"We call it research goo: regulation, risk, inefficiencies that complicate research in banks."

"Risk-sharper is a role assigned within Risk Management that carries teams through the risk process to mitigate it."

"Wine and dine is not a focus group; it’s more employees than customers interacting in small groups to get deep insights."

"In business-to-business contexts, finding a champion within your sales or account management teams can open access to users for research."

"You need to know tomorrow’s user as well as today’s because their characteristics differ greatly according to diffusion of innovation."

"Agile is here to stay because engineering managers won’t hire for anything else; user-centered design must fit into Agile somehow."

"Designers thrive on qualitative research because it gives them access to the root of the problem beyond just numbers."

"Never put numbers on qualitative research findings; qualitative insights are about richness and why, not how many."

"The PURE method measures usability by scoring cognitive load and friction at each step of fundamental user tasks."

Ask the Rosenbot
Taylor Klassman
Shaping the Next Era of UX Research: Collaborative Forum
2025 • Advancing Research 2025
Gold
Ellen Chisa
The Values of Design
2023 • Design in Product 2023
Gold
Terry Buckman
Wargaming (An Introduction)
2023 • Enterprise Community
Peter Van Dijck
Hands-on AI #2: Understanding evals: LLM as a Judge
2025 • Rosenfeld Community
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Steve Turbek
Designing Interactive Graphics with AI Code Help
2026 • Rosenfeld Community
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold
Caitlyn Hampton
Compass 101: Growing Your Career In A Startup World
2021 • Design at Scale 2021
Gold
Mike Oren
Improving Democratized Research with CustomGPTs and Gems
2026 • Rosenfeld Community
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Civic Design 2021
Gold
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Jose Coronado
People First - Design at JP Morgan
2021 • Design at Scale 2021
Gold
Holly Cole
Panel Discussion: Growing People and Teams
2018 • DesignOps Summit 2018
Gold
Nick Cochran
Growing in Enterprise Design through Making Connections
2019 • Enterprise Community
Dan Willis
Enterprise Storytelling Sessions
2018 • Enterprise Experience 2018
Gold
George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold

More Videos

Tamara Hale

"I have learned not to hold on too tightly to the ideas I’ve constructed about myself."

Tamara Hale

War Stories LIVE! Tamara Hale

March 30, 2020

Joanna Vodopivec

"Customer obsession is actually one of our key values, which makes my job as a researcher a little bit easier."

Joanna Vodopivec Prabhas Pokharel

One Research Team for All - Influence Without Authority

March 9, 2022

Saara Kamppari-Miller

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

Saara Kamppari-Miller Nicole Bergstrom Shashi Jain

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

November 13, 2024

Nalini P. Kotamraju

"I fumed that my idea had been ignored, wondering if it was because I was new, a woman, brown, or had a PhD."

Nalini P. Kotamraju

Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"

March 10, 2021

Ian Johnson

"Downloading and playing with local open source models gives a different experience than using faceless APIs mediated through interfaces."

Ian Johnson

Latent Scope: Finding structure in unstructured data

June 11, 2025

Greg Petroff

"The tools are more important than ever even if the title UX becomes less central."

Greg Petroff

Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian

September 24, 2025

Frances Yllana

"AI can help us develop better alt text for images, which improves accessibility for unsighted users."

Frances Yllana Jorge Arango Maria Taylor Briana Thomas

The Big Question about Impact: A Panel Discussion

September 24, 2024

Maria Giudice

"We have to go from shipping a minimal viable product to creating a minimal lovable experience."

Maria Giudice

Remaking the Making Company: Moving from Product to Experience

June 9, 2016

Louis Rosenfeld

"Testing your book idea through workshops or talks gives a forcing function that helps prioritize content."

Louis Rosenfeld

Coffee with Lou: Should You Write a (UX) Book?

March 7, 2024