Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Maria Rosala
AI-Assisted Qualitative Analysis: What to Automate and What to Own
2026 • Rosenfeld Community
Bob Baxley
Leading with Design Operations Past and Present
2019 • DesignOps Community
Christian Bason
Expand—Rethinking Design for Public Challenges
2022 • Civic Design Community
Kyle Godbey
Non-linear service design for complex adaptive systems
2025 • Rosenfeld Community
Nathan Shedroff
How Will Design be Taught When the Schools Shut Down?
2026 • Rosenfeld Community
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Lada Gorlenko
Theme 3: Introduction
2021 • Design at Scale 2021
Gold
Abby Covert
Stuck? Diagrams Help
2022 • DesignOps Community
Bas Raijmakers, PhD (RCA)
What Design Research can Learn from Documentary Filmmaking
2022 • Advancing Research 2022
Gold
Sahibzada Mayed
The Politics of Radical Research: A Manifesto
2023 • Advancing Research 2023
Gold
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Sam Yen
Driving Organizational Change Through Design? Do more of this and less of that
2017 • Enterprise Experience 2017
Gold
Lija Hogan
Doing more with more: Lessons from the Front Lines of Democratization
2022 • Advancing Research 2022
Gold
Savannah Carlin
Don't botch the bot: Designing interactions for AI
2024 • Designing with AI 2024
Gold
Patrizia Bertini
DesignOps + KPIs = Measure your Impact!
2024 • DesignOps Summit 2020
Gold

More Videos

Marissa Cui

"The key is leading with simplicity and educating users over time to create lasting value."

Marissa Cui Rachel He Michael Leggett Manos Saratsis

Climate Design Product Showcase

March 13, 2024

Snehal Pendharkar

"The biggest improvement was confidence — walking into a room with senior stakeholders prepared to ask sharper questions."

Snehal Pendharkar

Conducting pre-research with AI agent personas: Pressure-testing concepts for expert workflows

June 10, 2026

Alla Weinberg

"Toxic stress makes people dumb, reducing their cognitive ability by half."

Alla Weinberg

Healing Toxic Stress

September 23, 2024

Uday Gajendar

"These crafted artifacts are quick, dirty, and temporary—they last just long enough to fuel the conversation and decisions."

Uday Gajendar

The Wicked Craft of Enterprise UX

May 13, 2015

Jason Mesut

"Many leaders don’t have time to pause and reflect, but that reflection is critical for understanding the full picture of their leadership."

Jason Mesut Martina Hodges-Schell Jose Coronado

Unmasking Design Leadership: Navigating leadership without neglecting ourselves

October 30, 2025

Alfred Kahn

"If you don’t measure design’s impact, it’s like a tree falling in the forest with no one around to hear it."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Sam Proulx

"Involving people with disabilities from the start reveals impact and usability issues that automated testing simply cannot catch."

Sam Proulx

Online Shopping: Designing an Accessible Experience

October 3, 2023

Leisa Reichelt

"A developer pretending to be a product manager is not reliable participant data."

Leisa Reichelt

The Five Dysfunctions of Democratized Research at Scale

March 30, 2020

Dan Saffer

"AI struggles with context, taste, and common sense, and designers bring that to the table."

Dan Saffer

Why AI projects fail (and what we can do about it)

May 14, 2025