Rosenverse
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
Meters, Miles, and Madness: New Frameworks to Measure the (Elusive) Value of DesignOps
Speakers: John Calhoun and Rachel Posman
Link:

Summary

Measuring DesignOps value is surprisingly complicated. Many practitioners would agree with the sentiment that measuring their impact feels aspirational at best, and theoretical at worst. Anecdotal evidence and praise from partners (“I don't know what we’d do without you!”) is nice, but doesn’t add up to proof that DesignOps is a worthy return on investment.

In this talk, we’ll share two novel approaches to measuring DesignOps success from our upcoming book, The Design Conductors: Your Essential Guide to Design Operations. These methods–the “Jobs to Be Done” and “HEROES” frameworks–can be used to uncover and define measures of value that more accurately capture the impact DesignOps (and design) has on a business and its stakeholders. We will also share some tested rubrics that DesignOps can use to prioritize and implement these new measures.

Key Insights

  • •

    Traditional business metrics are often too narrow to capture the full value of design operations.

  • •

    Only about 50% of design ops practitioners have clearly defined metrics; just 41% measure design ops impact.

  • •

    The Jobs to Be Done framework can be adapted to design ops to define measurable outcomes linked to stakeholder needs.

  • •

    The Hero's Framework categorizes design ops impact into six areas: Health, Effectiveness, Readiness, Outcomes, Ecosystem, and Sentiment.

  • •

    Design ops should measure metrics tailored to their organization's mission, operating model, and current workstreams for best results.

  • •

    Expansion and contraction business phases affect which design ops metrics and outcomes teams should prioritize.

  • •

    Outcome-driven prioritization factors (success, progress, problem) help ensure metrics focus on improving change, not just status quo.

  • •

    Visibility of design ops work is critical to securing investment and recognition from leadership, as noted by Jared Spool and Sarah Walker Bocher.

  • •

    Measurement frameworks should help design ops move away from measuring all things to measuring what truly matters for impact and sustainability.

  • •

    The example of a Gen AI Summer Camp showed how design ops used readiness metrics and qualitative feedback to demonstrate tangible value.

Notable Quotes

"Measuring wrong can cost you half a billion dollars, but also what you measure and how you measure matters a lot."

"Design ops isn’t rocket science, but between us it might actually be harder."

"Design ops is a flexible and adaptable practice, meaning our goals and outcomes are usually more fluid than stable."

"Applying traditional frameworks to design ops is like trying to fit a star in a box."

"Design teams don’t want a design ops drill, they need a quarter inch hole that requires a little bit of program management."

"Healthy teams are the foundation of a healthy organization."

"Every hiccup in a process and every outdated protocol accumulates a toll on the team as well as financial costs."

"Visibility is the most vital thing for UX leaders to be focused on."

"Endlessly justifying yourself rarely changes others’ opinions. It positions design ops as up for debate."

"We choose to measure design ops value not because it is easy, but because it is hard."

Ask the Rosenbot
Florence Okoye
AfroFuturism and UX Research
2023 • Advancing Research 2023
Gold
Dave Malouf
Theme 3: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Design at Scale 2022
Gold
Jeff Gothelf
The Intersection of Lean and Design
2019 • Enterprise Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Civic Design 2022
Gold
Sean Dolan
A Practical Look at Creating More Usable Enterprise Customer Journeys
2019 • Enterprise Community
Rebecca Topps
Planning and conducting remote usability studies for accessibility
2020 • Advancing Research Community
Saskia Liebenberg
Start Small for Big Impact
2019 • DesignOps Community
Lisa Gironda
Opener: Chief of Staff–An unexpected journey
2024 • DesignOps Summit 2020
Gold
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Andy Barraclough
From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme
2024 • DesignOps Summit 2024
Gold
Mansi Gupta
Drawing from Feminist Practice to Make Inclusive Design Operational
2022 • DesignOps Summit 2022
Gold
Peter Van Dijck
Coffee with Lou #4: Taking a Peek Under the Rosenbot's Hood
2024 • Rosenfeld Community
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Sarah Williams
A Framework for CX Transformation
2021 • Design at Scale 2021
Gold

More Videos

Nancy Douyon

"Google’s photo algorithm mistakenly tagged Black people as gorillas, which was called out by Jackie Elsene."

Nancy Douyon

We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex

June 15, 2018

Jack Behar

"UXPin is a full-stack platform that bridges the gap between design and development disciplines."

Jack Behar

How to Build Prototypes that Behave like an End-Product

December 6, 2022

Jorge Arango

"The whole should be greater than the sum of the parts, but achieving that requires intentional design and leadership."

Jorge Arango

Meeting of the Waters: Designing for Successful Inorganic Growth

August 12, 2021

Jon White

"Don’t try to tag all historical work at once — tag as you go to avoid re-platforming nightmares."

Jon White Erin May

Unsticking Research for Better Information Flow

March 11, 2026

Jessamyn Edwards

"Designing in enterprise means facilitating conversations and educating your teams because many stakeholders don’t speak design."

Jessamyn Edwards

Surviving Your UX Career in Enterprise Design

December 2, 2021

Stefanie Owens

"It can be hard for designers to immediately understand or advocate for the real value of design at deeper technical layers."

Stefanie Owens

Optimizing for Outcomes: Transformation Design in Systems at Scale

December 4, 2024

Ron Bronson

"Attention theft is when experiences are designed to siphon away your focus aggressively."

Ron Bronson

Design, Consequences & Everyday Life

November 18, 2022

Louis Rosenfeld

"Slack is growing steadily for us, but it’s hard to moderate and expensive at scale."

Louis Rosenfeld

The Rosenbot and the Rosenverse: An AMA with Lou Rosenfeld

June 5, 2024

Ash Brown

"Education is the foundation upon which we build a more sustainable future."

Ash Brown

Silver Linings: What DesignOps Learned in the Shift to WFH

October 23, 2020

Latest Books All books