Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Rittika Basu
Age and Interfaces: Equipping Older Adults with Technological Tools
2023 • Advancing Research Community
Matt Bernius
Learnings from Applying Trauma-Informed Principles to the Research Process
2022 • Advancing Research 2022
Gold
Daniel J. Rosenberg
Designing with and for Artificial Intelligence
2022 • Enterprise Community
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • Civic Design 2021
Gold
Sarit Geertjes
People, not Petri Dishes: Stories from a Research Recruiter
2019 • DesignOps Community
Christopher Taylor Edwards
Design as a Team Practice, A Practical Guide to Cross-functional Collaboration
2021 • DesignOps Summit 2021
Gold
Megan Blocker
Panel: Excellence in Impact
2024 • Advancing Research 2024
Gold
Erika Flowers
Introduction to MURAL for UX
2021 • Design at Scale 2021
Gold
Nathan Curtis
Design Systems for Us: How Many One-Source(s)-of-Truth Are Enough?
2019 • DesignOps Community
Maria Rosala
Research Repositories
2026 • Advancing Research 2026
Conference
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Doug Powell
DesignOps and the Next Frontier: Leading Through Unpredictable Change
2025 • DesignOps Summit 2025
Gold
Susan Simon-Daniels
War Stories LIVE! Susan Simon-Daniels
2020 • Advancing Research 2020
Gold
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Yolanda Rankin
Black Feminist Epistemology as a Critical Framework for Equitable Design
2021 • Advancing Research 2021
Gold

More Videos

Louis Rosenfeld

"Newness isn’t necessarily a new methodology—it can be a new way of looking at something familiar."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024

Jilanna Wilson

"When you remove nonverbal cues from communication, messages get easily misinterpreted."

Jilanna Wilson

Distributed Design Operations Management

October 23, 2019

Matt Webb

"We can make our dreams real, bringing objects back from the other side of the bridge."

Matt Webb

Context Window: Five Futures for AI

June 11, 2025

Sam Proulx

"Accessibility requires flexible designs, not limited or simplified ones."

Sam Proulx

Accessibility: An Opportunity to Innovate

September 8, 2022

Theresa Neil

"Don’t bluff regulated healthcare UX experience; everyone will know, and it’s harmful to your team and clients."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Peter Van Dijck

"LLMs don’t have memory, so rating on a scale from one to five is pretty random. Better to have yes or no answers."

Peter Van Dijck

Hands-on AI #2: Understanding evals: LLM as a Judge

October 15, 2025

Rachel Posman

"Design ops isn’t just a support role; it is a strategic partner involved in decision making."

Rachel Posman John Calhoun

"Ask Me Anything" with Rachel Posman and John Calhoun, Authors of the Upcoming Rosenfeld Book, The Design Conductors

September 25, 2024

Tricia Wang

"Web3 offers a unique chance to get involved early before some of the ethical challenges of Web2 take root."

Tricia Wang

The most popular design thinking strategy is BS

January 27, 2022

Sarah Auslander

"Civic action as proof of concept to get government moving, to get the wheels turning was really interesting."

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022