Rosenverse

Redefining Toolkits: Unbundling to Create a Perfect Match

Gold
Tuesday, March 11, 2025 • Advancing Research 2025
Share the love for this talk
Redefining Toolkits: Unbundling to Create a Perfect Match
Speakers: Benjamin Wiedmaier and Annie Mayfield
Link:

Summary

"Redefining the Research Toolkit" not only applies to methods, but tools, too. The marketplace for research platforms continues growing and evolving, offerings professionals the opportunity to find a nearly perfect research match. This means that more teams are opting to "unbundle" their research stack, choosing best-in-class options for critical phases instead of an all-in-one. This session has two goals: 1) Highlight some of the of tools researchers are prioritizing for their stacks 2) Present a case study of a best-in-class stack in action Join to learn about the freedom of choice researchers have for their toolkits and some of the best-in-class options available.

Key Insights

  • All-in-one research platforms average $100,000 per year and may not flexibly adapt as research needs evolve.

  • BYO toolkits enable teams to select best-in-class tools tailored to their specific research methods and questions.

  • User Interviews specializes in upstream participant recruitment, maintaining an exceptionally low fraud rate of 0.3%.

  • AI-driven participant matching improves recruitment quality and reduces panel churn in User Interviews’ platform.

  • Calendly uses a BYO stack combining Sprig for in-app surveys and User Interviews for recruiting to address diverse research needs.

  • Flexibility in tool choice allows teams with limited headcount and budgets to scale research effectively.

  • BYO stacks support democratizing research by enabling controlled access for non-researchers like PMs and designers.

  • User Interviews offers direct integrations and flexible API partnerships, plus options to use any tool via URL links.

  • Effective fraud detection combines automated monitoring with researcher-controlled flags to maintain data integrity.

  • Focusing on a core strength—in this case, participant recruitment—and partnering with specialized tools leads to more effective research workflows.

Notable Quotes

"Budgets are tight across research, design, and product; all-in-one platforms are costly and often don’t fit those budgets."

"When you get a best in class tool, you are able to execute more rigorous research and show clearer ROI."

"We found a greater variety and depth of questions that require richer methodologies, so BYO works better for us."

"Our fraudulent participant rate is about 0.3%, compared to the industry average of 20 to 22%."

"We built AI that suggests projects to participants that pre-match what you’re asking for instead of mass blasting everyone."

"The world is your oyster with whatever tool you want to use, even if we don’t directly integrate with it."

"The only way to do research wrong is to not do it at all."

"BYO gives researchers more control over who gets access to the panel and where you want guardrails versus freedom."

"My time is more expensive than my money; fraudulent data wastes valuable time."

"We focus on getting top quality participants and let others specialize in testing or analysis tools."

Ask the Rosenbot
Steve Portigal
The Future of Research: Bridging the Gaps
2021 • Advancing Research Community
Yunyan Li
UX Best Practices
2021 • Design at Scale 2021
Gold
Alicia D. Johnson
Disasters and the 21st Century
2021 • Civic Design 2021
Gold
Snehal Pendharkar
Conducting pre-research with AI agent personas: Pressure-testing concepts for expert workflows
2026 • Designing with AI 2026
Conference
Dawn Ressel
Full-Stack User Experiences: A Marriage of Design and Technology
2016 • Enterprise UX 2016
Gold
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Elizabeth Sklar
Co-creating research enablement with your tech org: a case study
2026 • Advancing Research 2026
Gold
Catt Small
Moving from Execution to Strategy as a Designer
2022 • Design in Product 2022
Gold
Peter Van Dijck
Hands-on AI #2: Understanding evals: LLM as a Judge
2025 • Rosenfeld Community
Taylor Klassman
The Evolution of UX Research Platforms
2026 • Advancing Research 2026
Gold
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Megan Nipe
Human-Centered Design for Engagement: Maturing from Newsletterville to Personalized, One-to-One Messaging
2021 • Civic Design 2021
Gold
George Hinchliffe
Delivering Amazing Experiences
2021 • Design at Scale 2021
Gold
John Maeda
About Design Organizations
2019 • DesignOps Community
Richard Buchanan
Creativity and Principles in the Flourishing Enterprise
2018 • Enterprise Experience 2018
Gold
Marieke McCloskey
User Science: Product Analytics & User Research
2021 • Advancing Research 2021
Gold

More Videos

Matt Bernius

"All processes are extractive; even if there’s an element of enrichment, reducing that extraction is crucial."

Matt Bernius Sarah Fathallah Hera Hussain Jessica Zéroual-Kara

Trauma-informed Research: A Panel Discussion

October 7, 2021

Jen Cardello

"Root cause analysis helps identify not just what is happening, but why it's happening, especially when more people have access to data."

Jen Cardello Dr. Shadi Janansefat Alex Wright

Curating insight: Strategies for integrating knowledge across research functions

March 11, 2025

Sha Hwang

"This project is actually 10 years in to a 50 year project."

Sha Hwang

The First Fifty Years of Civic Design

November 16, 2022

Kyle Godbey

"Emergence is properties of a whole system not expressed in properties of parts. Water is liquid but hydrogen and oxygen atoms are not."

Kyle Godbey

Non-linear service design for complex adaptive systems

December 10, 2025

Brigette Metzler

"Measuring ops impact by how much people are using the research and embedding feedback loops is much more valuable than just looking at money saved."

Brigette Metzler

Scaling ResearchOps: Helping Researchers do Their Best Work

March 30, 2020

Jennifer Kong

"People are the plot twist in this journey — despite promising AI, only human connection reveals new problems and solutions."

Jennifer Kong

Journeying toward AI-assisted documentation in healthcare

June 5, 2024

Magdalena Zadara

"Germany is a large federation with 16 federal states and an additional European Union layer, making digital service delivery quite complex."

Magdalena Zadara

Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late

November 16, 2022

Husani Oakley

"AI has eaten the world and will continue to eat the world forever."

Husani Oakley

Theme Three Intro

June 6, 2023

"Continuous feedback instead of just end-of-year reviews helps people understand their long-term goals and improve in the moment."

DesignOps and The Great Talent War of 2021

August 19, 2021