Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Coach Enterprise Experimentation
Gold
Thursday, May 14, 2015 • Enterprise UX 2015
Share the love for this talk
How to Coach Enterprise Experimentation
Speakers: Alissa Briggs
Link:

Summary

In this talk, Alyssa Briggs shares her experience coaching enterprise experimentation, emphasizing that experimentation is a mindset for problem-solving by validating assumptions through small, fast tests. She highlights that executives often support experimentation in theory but struggle to embed it into daily work. Using Intuit as a case study, Briggs illustrates how simply training and embedding experiment coaches within teams transformed the company’s culture, enabling tens of thousands of experiments annually that generate millions in revenue. She describes three essential practices for experiment coaches: collaboratively planning experiments using an experiment grid, helping teams embrace and learn from failures by facing data honestly and conducting quick customer research, and catalyzing organizational change by spreading the coaching role and uncovering new opportunities for experimentation beyond product design. Briggs walks the audience through a live experiment about a name memory trick to demonstrate how to develop hypotheses and validate assumptions quickly. She also recounts a story where a small sales experiment disproved executives’ skepticism about a new product tier, leading to significant business impact and wider cultural adoption of experimentation. Briggs encourages everyone to adopt the experiment coaching mindset and start running small experiments to shift team behaviors and company culture.

Key Insights

  • Embedding experiment coaches within existing teams is key to sustaining a culture of experimentation.

  • Experimentation thrives on embracing small failures to avoid costly large ones.

  • Executives often support experimentation top-down, but real change happens when coaches integrate experiments into daily workflows.

  • The experiment grid is a powerful yet simple tool for collaboratively planning experiments by unpacking assumptions and defining testable hypotheses.

  • Most teams fail their first experiment, and coaches must help them face the data and learn rather than become discouraged.

  • Quick, informal customer research after an experiment helps uncover root causes and increases team engagement.

  • Consistently planning follow-up experiments builds momentum and prevents the 'one-and-done' pitfall.

  • Experiment coaching is not a full-time role but can be integrated as 10-20% of a team member’s responsibilities.

  • Experiments can test not just products but internal processes and business assumptions, unlocking broader innovation.

  • Demonstrating successful experiments with quantitative data can transform executive skepticism into enthusiastic support, driving significant business impact.

Notable Quotes

"If you’re afraid of failure, that’s okay. With experimentation, you’re going to fail—but in little tiny ways that don’t really matter."

"Executives get experimentation but often it doesn’t stick when people go back to their normal teams."

"The simple thing that worked at Intuit was training and embedding experiment coaches in teams."

"An experiment coach spends about 10 to 20 percent of their time helping the team change mindsets and actions."

"The most important tool is not for running experiments, but for planning them—the experiment grid."

"If we do X, then Y percent of people will do Z—that’s how you frame hypotheses to learn from both success and failure."

"It’s better to run lots of quick, cheap experiments to remove doubt than to wait for the perfect, statistically significant test."

"Nine out of ten teams fail their first experiment, which is normal but can be demoralizing."

"Helping your team face the data honestly and then diving into customer conversations helps uncover what went wrong."

"You know you’ve succeeded as a coach when you put yourself out of a job by giving away the role."

Ask the Rosenbot
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Alla Weinberg
Cross-Functional Relationship Design
2022 • Design in Product 2022
Gold
Chris Geison
What is Research Strategy?
2021 • Advancing Research 2021
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Rachael Dietkus, LCSW
Trauma-Responsive Design: Reimagining the Future of Design Now
2021 • Civic Design 2021
Gold
George Aye
That Quiet Little Voice: When Design and Ethics Collide
2022 • Civic Design 2022
Gold
Aras Bilgen
Research Democratization: A Debate
2023 • Advancing Research 2023
Gold
Noz Urbina
Rapid AI-powered UX (RAUX): A framework for empowering human designers
2025 • Rosenfeld Community
Jayne Engle
Civic Design for the Next Seven Generations—A Discussion on Sacred Civics
2022 • Civic Design Community
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!
2025 • Rosenfeld Community
Dave Malouf
The Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
2022 • DesignOps Community
Kate Towsey
ResearchOps AMA with Kate Towsey & Jake Burghardt
2025 • Advancing Research Community
Maria Taylor
Knowledge is Power: Managing the Lifeblood of the Design Org
2023 • DesignOps Summit 2023
Gold
Aletheia Delivre
New Shapes and Emerging Identities for Design Ops
2025 • DesignOps Summit 2025
Conference
Irina Tikhonova
Small Wins, Big Impact: Leveraging and Elevating User Engagement
2021 • Civic Design 2021
Gold
Laura Schaefer
DesignOps: A Conduit for Inclusion
2022 • DesignOps Summit 2022
Gold

More Videos

Sam Proulx

"No assistive technology user uses default settings; we spend years customizing everything to work for us."

Sam Proulx

SUS: A System Unusable for Twenty Percent of the Population

December 9, 2021

Michael Land

"In government, the org chart really matters – it influences who talks to whom and who gets to interface."

Michael Land

Establishing Design Operations in Government

February 18, 2021

Shipra Kayan

"Our one metric that mattered was maximizing the number of VOС-tagged tickets solved – showing customer feedback made it to the roadmap."

Shipra Kayan

How we Built a VoC (Voice of the Customer) Practice at Upwork from the Ground Up

September 30, 2021

Ian Swinson

"Why don’t we do what we do for users but apply it to our own careers? Absolutely nutty, right? So we did."

Ian Swinson

Designing and Driving UX Careers

June 8, 2016

Isaac Heyveld

"We went from a team of 80 to 170 overnight, and that was a wide open cheeks flushed moment for me."

Isaac Heyveld

Expand DesignOps Leadership as a Chief of Staff

September 8, 2022

Amy Evans

"Starting with one line of business and showing success creates a trickle effect for broader organizational adoption."

Amy Evans

How to Create Change

September 25, 2024

Kate Koch

"Super strength is the resilience and endurance to partner and drive strategic impact backed by customer obsession."

Kate Koch Prateek Kalli

Flex Your Super Powers: When a Design Ops Team Scales to Power CX

September 30, 2021

Dave Gray

"The higher up you are in an organization, the more distorted your reality is likely to be."

Dave Gray

Liminal Thinking: Sense-making for systems in large organizations

May 14, 2015

Matt Duignan

"You have to push against 200,000 years of human evolution to get researchers to create reusable knowledge."

Matt Duignan

Atomizing Research: Trend or Trap

March 30, 2020