Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment. In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • Traditional business metrics are often too narrow to capture the full value of design operations.

  • Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Farid Sabitov
Theme Four Intro
2022 • DesignOps Summit 2022
Gold
Sarah Flamion
Complex Problem? Add Clarity by Combining Research and Systems Thinking
2020 • Advancing Research 2020
Gold
Molly Fargotstein
Multipurpose Communication & UX Research Marketing
2019 • DesignOps Community
Paul Pangaro, PhD
Systems Disciplines: Table Stakes for 21st Century Organizations
2023 • Enterprise UX 2023
Gold
Deanna Mitchell
Designing with culture: Unlocking impactful insights for Product and UX
2025 • Advancing Research 2025
Gold
Ted Neward
Theme 4: Enterprise Organizational Journey
2019 • Enterprise Experience 2019
Gold
Sahibzada Mayed
Cultivating Design Ecologies of Care, Community, and Collaboration
2023 • DesignOps Summit 2023
Gold
Adam Cutler
People + Places + Practices = Outcomes
2016 • Enterprise UX 2016
Gold
Margot Bloomstein
Fostering Trust in Your Brand and Beyond
2020 • Enterprise Community
Dave Hora
Research in the Face of Complexity: New Sensibility for New Situations
2025 • Rosenfeld Community
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • Civic Design 2022
Gold
Nicole Aleong
Future Orientations to Everyday Life: Futures Anthropology as a Methodology
2024 • Advancing Research 2024
Gold
Deirdre Hirschtritt
Research is Only as Good as the Relationships You Build
2022 • Civic Design 2022
Gold
Christopher Taylor Edwards
Design as a Team Practice, A Practical Guide to Cross-functional Collaboration
2021 • DesignOps Summit 2021
Gold
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Charlotte Vorbeck
Pipeline to Civic Design
2021 • Civic Design 2021
Gold

More Videos

John Cutler

"These aren't linear; every circle or box has multiple lines into it—our work does not fit neatly into trees or pyramids."

John Cutler

The Alignment Trap

November 29, 2023

Monty Hammontree

"We changed the culture by changing the language, introducing common terms like hypotheses and experiments."

Monty Hammontree

The Future of UX Research

December 3, 2020

Matt Stone

"We over-emphasized technology and process information and forgot the people inside the organization."

Matt Stone

Scaling Empathy, A Case Study in Change Management

June 11, 2021

Gonzalo Goyanes

"Our aim is not just to demonstrate design’s value for growth, but to become an essential part of the company’s overall strategy."

Gonzalo Goyanes

Design ROI: Cover a Little, Get a Lot

September 8, 2022

Amy Parness

"Sustainability is a long haul; you have to stay stubborn and not take no for an answer in a bad way."

Amy Parness

Scaling Sustainability: Complementary strategies that drive long-term success

May 29, 2025

Alana Washington

"We heard from the community through surveys which helped us understand what you hope to see at this year's Design Ops Summit."

Alana Washington

Theme 3 Intro

October 1, 2021

Patrizia Bertini

"Most AI systems today cannot be retrofitted easily to comply with regulation."

Patrizia Bertini

Designing Within the Lines: How the EU AI Act Can Spark Better AI Innovation

December 11, 2025

Jane Reid

"Offering participants food or shelter crosses professional boundaries and can create unsafe expectations."

Jane Reid Janice Hannaway

Self-care in User Research

April 2, 2020

Nathan Curtis

"Getting exact endorsement from the top helps your cause immensely."

Nathan Curtis

Beyond the Toolkit: Spreading a System Across People & Products

June 9, 2016