Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
Mike Oren
Design Research Strategy & Strategic Design Research
2022 • Advancing Research Community
Megan Blocker
Theme 2 Intro
2025 • Advancing Research 2025
Gold
Amber Knabl
Empowering innovation: The critical role of inclusive product development in the AI era
2024 • Designing with AI 2024
Gold
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Sarah Flamion
Complex Problem? Add Clarity by Combining Research and Systems Thinking
2020 • Advancing Research 2020
Gold
Sam Proulx
Understanding Screen Readers on Mobile: How And Why to Learn from Native Users
2023 • Enterprise UX 2023
Gold
Jemma Ahmed
Theme 2 Intro
2024 • Enterprise Experience 2020
Gold
Nick Lewis
Designing and building low-carbon websites independently
2025 • Climate UX Interest Group
Ariba Jahan
Team Resiliency Through a Pandemic
2024 • DesignOps Summit 2020
Gold
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Sam Proulx
Everything You Ever Wanted to Know About Screen Readers
2021 • Design at Scale 2021
Gold
Benjamin Real
Maturity Models: A Core Tool for Creating a DesignOps Strategy
2021 • DesignOps Summit 2021
Gold
Alëna Iouguina
Designing Systems at Scale
2018 • DesignOps Summit 2018
Gold
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
Sam Ladner
Data Exhaust and Personal Data: Learning from Consumer Products to Enhance Enterprise UX
2016 • Enterprise UX 2016
Gold
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold

More Videos

Doug Powell

"Diversity is beyond just the right thing to do. Our teams must reflect the world we’re designing for."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"Middle schoolers might ask you tough questions about release forms and how their data will be used."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"Every parent we talked to had some variation of the question: this exists, can I use it right now? Where was this when I needed to apply?"

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"We never compromise on the business goals, but how we achieve them can be flexible and pragmatic."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"AI is here to stay, challenging our norms and shaping our future."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Mixed methods research, combining qualitative and quantitative, gives the fullest understanding of customer experience."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"When you’re talking, it’s easier to learn something new, but if you’re shy and don’t talk, you don’t learn anything."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"We neglected our power users during the responsive, one-size-fits-all strategy, and it really hurt us."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"Design is not just aesthetics; it’s about what the product feels like, sounds like, and how it differentiates in the market."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023