Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Scaling Research Via an Ops First Model at Clever
Gold
Monday, March 27, 2023 • Advancing Research 2023
Share the love for this talk
Scaling Research Via an Ops First Model at Clever
Speakers: Liza Pemstein and Jane Davis
Link:

Summary

In this Q&A session, we'll hear how Liza Pemstein, Research Operations Manager at Clever, built a robust research practice at one of the fastest-growing EdTech companies. Identifying the friction in the process experienced by design, product, engineering, and marketing, Liza has streamlined and scaled research. She'll share how she got started, what she's learned and what she's excited about in 2023!

Key Insights

  • Clever operates a research operations first model with no dedicated UX researchers, relying on designers and PMs to conduct research.

  • Liza Pennstein’s path to research operations included customer solutions roles emphasizing empathy, listening, and problem-solving.

  • Centralizing participant incentives through a shared credit card and Jira ticket system eliminated confusion and improved process efficiency.

  • Recruiting and scheduling user research participants was a major barrier that was addressed by creating shared participant databases.

  • The ROCKS program at Clever (Research Ops Knowledge Share) facilitates cross-team learning and un-silos research knowledge.

  • Initial focus was on increasing research confidence and comfort among practitioners before emphasizing quality and best practices.

  • Balancing standardization with flexibility is critical; intervention happens when research quality or participant screening falters.

  • Without dedicated UX researchers, Clever prioritizes enabling non-experts with training, templates, and external expert speakers.

  • Building a lightweight repository with Great Question software supports storing and sharing research insights effectively.

  • Future plans include broadening methods like unmoderated research, increasing visibility, and maintaining momentum through speaker series and workshops.

Notable Quotes

"Research is a sales role at its heart, delivering value to people to drive sustainable growth."

"I had a hunger for process and efficiency once I built a feedback bot to share customer insights."

"We don’t have dedicated user researchers, but designers and PMs conduct most research here."

"People were doing incentives in different ways, using their own cards, which was confusing, so I centralized it."

"The biggest struggle was recruiting and scheduling participants, so we started a shared Google sheet."

"Research is a group project and learning is a group project—you don’t have to be an expert but find those who are."

"I want to encourage excitement about research and gradually mold people into better researchers."

"Sometimes you have to make it messy before it can be clean again — disrupt to improve."

"We have the quantity of research work; now we’re focused on building up the quality."

"Unmoderated research is exciting because it lets us gather insights while freeing people up to be in two places at once."

Ask the Rosenbot
Sarah Fathallah
A Typology of Participation in Participatory Research
2023 • Advancing Research 2023
Gold
Amy Jiménez Márquez
The Atypical UX Manager Path
2020 • Enterprise Community
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Patrizia Bertini
Designing Within the Lines: How the EU AI Act Can Spark Better AI Innovation
2025 • DesignOps Community
Dan Hill
Designing for the infrastructures of everyday life
2024 • Designing with AI 2024
Gold
Jemma Ahmed
Theme Three Intro
2023 • Advancing Research 2023
Gold
Lija Hogan
Contexts of Use: A Framework for Connection
2021 • Civic Design 2021
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Ned Dwyer
Right horses for the right courses – how and when to democratize research
2025 • Advancing Service Design 2025
Conference
Gabriela Barneva
Operationalizing Inclusive Design in Design Ops
2025 • DesignOps Summit 2025
Gold
Michelle Bejian Lotia
Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
2023 • Advancing Research 2023
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Tricia Wang
SCALE: Discussion
2018 • Enterprise Experience 2018
Gold
Jill Fruchter
Inconvenient Insights: The Researcher's Role is to Stay Curious
2023 • Advancing Research 2023
Gold
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • DesignOps Summit 2022
Gold
Jorge Arango
[Demo] How to re-categorize content at scale using LLMs
2024 • Designing with AI 2024
Gold

More Videos

Nalini Kotamraju

"I believed we could do more — more for the company, customers, stakeholders, and ourselves if we had more agency."

Nalini Kotamraju

Research After UX

March 25, 2024

Dean Broadley

"Care less about being certain and more about being effective."

Dean Broadley

Not Black Enough to be White

January 8, 2024

Denise Jacobs

"It can be so tiring to be the only one constantly saying this has to be done differently."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Kim Fellman Cohen

"If you don’t have the support you need—budget, time, people—your program is at high risk of failure."

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

George Aye

"Why do designers decide how to design this? How do we wean design's addiction from whiteness?"

George Aye

That Quiet Little Voice: When Design and Ethics Collide

November 16, 2022

Nathan Curtis

"You have to treat the design system like a product that everyone in your organization will use."

Nathan Curtis

Beyond the Toolkit: Spreading a System Across People & Products

June 9, 2016

Greg Petroff

"Platforms need researchers and designers to make them meaningful and intentional for the people they are designed to serve."

Greg Petroff

The Compass Mission

March 10, 2021

Chloe Amos-Edkins

"Digital connectivity is necessary for participation in society, but half the world lacks internet access."

Chloe Amos-Edkins

A Cultural Approach: Research in the Context of Glocalisation

March 27, 2023

Mackenzie Cockram

"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022