Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Adel Du Toit
Get Your CFO To Say: 'Our Strategic Goal is User Obsession'
2022 • Design at Scale 2022
Gold
Maria Giudice
Remaking the Making Company: Moving from Product to Experience
2016 • Enterprise UX 2016
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • DesignOps Summit 2025
Gold
Marc Rettig
Discussion
2015 • Enterprise UX 2015
Gold
Ben Davies
Expert Panel: The Principles of Research Repository Design
2022 • Advancing Research 2022
Gold
Cheryl Platz
Demystifying Multimodal Design: The Design Practice You Didn't Know You're Doing
2024 • Rosenfeld Community
Failure Friday #4: Invisible Work: How I Stalled My Career by Not Showing My Work
2025 • Rosenfeld Community
Mary-Lynne Williams
Exit Interview #4: From Product Design Leadership to Sound Healing
2026 • Rosenfeld Community
Kit Unger
Theme 2: Discussion
2024 • Enterprise Experience 2020
Gold
John Calhoun
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
2024 • DesignOps Summit 2024
Gold
Prayag Narula
How to Empower Your Designers to Do Good Research – And Why You Want To
2022 • Design at Scale 2022
Gold
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Amy Thibodeau
Opening Keynote: Process and Ambiguity
2019 • DesignOps Summit 2019
Gold
Tutti Taygerly
Videconference: How to Work with Difficult People with Tutti Taygerly
2020 • Enterprise Community
Marc Rettig
The Urge to Migrate
2026 • Rosenfeld Community
DesignOps and The Great Talent War of 2021
2021 • DesignOps Community

More Videos

Jess Greco

"Your partners need to trust that you are going to make their goals your goals."

Jess Greco

Claiming your power: Practical tools for amplifying your unique voice

March 13, 2025

Josh Clark

"AI is not just a tool for productivity or efficiency; it's a design material with unusual strengths and weaknesses."

Josh Clark Veronika Kindred

Sentient Scenes and Radically Adaptive Experiences

June 11, 2025

Aleksandra Korczynska

"Continuous feedback requires collecting responses tied to the same user across multiple channels with behavioral targeting."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Anat Fintzi

"Before, silos ruled. Now, teams share information and make cross-supply chain decisions focused on customers and associates."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Peter Morville

"The Library of Congress web presence was a findability nightmare, like the Winchester Mystery House."

Peter Morville

The Architecture of Understanding

May 13, 2015

Paul Ford

"The great gift of AI is not magic but giving access to parts of the computer that used to take weeks or months to fiddle with."

Paul Ford

New work, new words: A glossary for AI

June 9, 2026

Ariel Kennan

"We heard a bit about understanding the power and privilege that we have as designers, how to wield it, but also how to yield it to others."

Ariel Kennan

Theme Two Intro

November 17, 2022

James Rampton

"When you project Apple CarPlay, you are basically sending the device from your phone into the car, detaching from the native OS."

James Rampton

The Basics of Automotive UX & Why Phones Are a Part of That Future

July 25, 2024

Michaela Mora

"When you start concept testing, be clear about what parts of the concept describe the what, the why, and the how."

Michaela Mora

Advanced Concept Testing Approaches To Guide Product Development and Business Decisions

March 11, 2022