Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold
Ed Mullen
Designing the Unseen: Enabling Institutions to Build Public Trust
2022 • Civic Design 2022
Gold
Jose Coronado
From Zero to Hero
2022 • DesignOps Summit 2022
Gold
Sarah Brooks
Theme 3 Intro
2021 • Civic Design 2021
Gold
Kristin Skinner
Five Years of DesignOps
2021 • DesignOps Summit 2021
Gold
Emilia Åström
Unlock Your Team’s Intelligence with Collaboration Design
2022 • Design at Scale 2022
Gold
Dalia El-Shimy
So You've Got a Seat at the Table. Now What?
2020 • Advancing Research 2020
Gold
Leisa Reichelt
The Five Dysfunctions of Democratized Research at Scale
2020 • Advancing Research 2020
Gold
Dominique Ward
The Most Exciting Time for DesignOps is Now
2022 • DesignOps Summit 2022
Gold
Uday Gajendar
Theme One Intro
2023 • Enterprise UX 2023
Gold
Jacqui Frey
Panel Discussion: Integrating DesignOps
2018 • DesignOps Summit 2018
Gold
Abbey Smalley
Scaling UX Past the Size of Your Team
2024 • Enterprise Experience 2020
Gold
Dane DeSutter
What co-speech gestures reveal about users’ thinking during interviews
2023 • Rosenfeld Community
Jacqui Frey
Scale is Social Work
2020 • DesignOps Community
Marc Majers
Interrupted UX - Add A Dose of Reality To Usability Testing
2022 • Advancing Research 2022
Gold
Ashley Sewall
Exit Interview #5: Designing My Life After Tech
2026 • Rosenfeld Community

More Videos

Jess Greco

"If I had waited for someone else to give me permission, I'd still be waiting. We'd all be waiting."

Jess Greco

Claiming your power: Practical tools for amplifying your unique voice

March 13, 2025

Josh Clark

"AI is not just a tool for productivity or efficiency; it's a design material with unusual strengths and weaknesses."

Josh Clark Veronika Kindred

Sentient Scenes and Radically Adaptive Experiences

June 11, 2025

Aleksandra Korczynska

"Survey data should be treated as zero-party data that fuels personalized communications and lifecycle marketing."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Anat Fintzi

"Home Depot is the sixth largest private company in the US with over 500,000 employees, yet our deliveries experience was embarrassingly bad."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Peter Morville

"Most categories are fuzzy; there’s no clear boundary between what’s in or out."

Peter Morville

The Architecture of Understanding

May 13, 2015

Paul Ford

"Designers are being told to be less and less visual; your job involves more product discussion—you’re going from designer to describer."

Paul Ford

New work, new words: A glossary for AI

June 9, 2026

Ariel Kennan

"I’m regularly pleasantly surprised by not having to explain a lot of the foundational concepts of design when working with others inside government."

Ariel Kennan

Theme Two Intro

November 17, 2022

James Rampton

"The phone itself is almost the center of the automotive ecosystem on the way to work."

James Rampton

The Basics of Automotive UX & Why Phones Are a Part of That Future

July 25, 2024

Michaela Mora

"Showing several concepts side by side and forcing choices yields better discrimination than rating scales."

Michaela Mora

Advanced Concept Testing Approaches To Guide Product Development and Business Decisions

March 11, 2022