Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
Product and Design at Bloomberg: A 15-year Evolution
2022 • Design in Product 2022
Gold
Rachael Dietkus, LCSW
The power to heal and harm
2025 • Advancing Research 2025
Gold
Rachael Dietkus, LCSW
Leading through the long tail of trauma
2022 • Advancing Research Community
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Joel Branch
Humanizing AI: Filling the Gaps with Multi-faceted Research
2021 • Advancing Research 2021
Gold
Etienne Fang
Power of Insights: Why sharing is better than silos with Uber’s Insights Platform
2019 • Advancing Research Community
Onur Kocan
Understanding the Strategy for Civic Design in a Complex City: Istanbul
2022 • Civic Design 2022
Gold
Scott Plewes
Why Isn't Your UX Approach Going Viral?: A Mathematical Model
2023 • Advancing Research 2023
Gold
Megan Blocker
Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research
2024 • Advancing Research 2024
Gold
Luca Rager
Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch
2021 • DesignOps Summit 2021
Gold
Andrew Michael
Building a Product Insights Team
2022 • Advancing Research 2022
Gold
Kate Towsey
Ask Me Anything (AMA) with Kate Towsey
2025 • Rosenfeld Community
The AI + Design wave has arrived: Making sense of the 2026 AI + Design Report
2026 • Rosenfeld Community
Ana Ferreira
Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries
2024 • DesignOps Summit 2020
Gold
Liwei Dai
The Heart and Brain of the AI Research
2020 • Advancing Research 2020
Gold
Kevin Bethune
Gatekeepers and Servant Leadership
2020 • DesignOps Community

More Videos

Bob Baxley

"The theme today, 'better together,' focuses on partnering with others to transform."

Bob Baxley

Theme 4: Intro

January 8, 2024

Kurt McCulloch

"When you boil down all the research into actions someone might take, the thing that made our research valuable in the first place, understanding context, was all gone."

Kurt McCulloch

Faster alone, further together: Rebuilding collaboration in the age of AI research

March 10, 2026

Irina Tikhonova

"Digital transformation can be slowed down by leadership resistance and a desire for gradual change."

Irina Tikhonova Kari Dietrich

Small Wins, Big Impact: Leveraging and Elevating User Engagement

December 9, 2021

Renee Bouwens

"Every report should say this research is tied to this KPI, here’s the outcome we want to address."

Renee Bouwens

Landing Product Impact: Aligning Research as a Foundational Driver for Delivering the World’s Best Products

December 15, 2023

Nalini P. Kotamraju

"Making the design system open source put Salesforce and the UX team on the map."

Nalini P. Kotamraju

An Organizational Story: Salesforce Lightning Design System

June 9, 2016

Changying (Z) Zheng

"Operations work best when practical tools are put in peoples' hands, not just theory."

Changying (Z) Zheng

Practical DesignOps: From Ideas to Tools That Teams Actually Use

September 25, 2025

Rachael Dietkus, LCSW

"Our curation process is identity hidden to prioritize equity and thematic mapping over affiliation or geography."

Rachael Dietkus, LCSW Victor Udoewa Jennifer Strickland

Everything You Need to Know about the Civic Design 2022 Call for Presentations

May 17, 2022

Roberta Dombrowski

"Stakeholders might say no to research involvement if the context and relevance aren’t clear, or the ask seems overwhelming."

Roberta Dombrowski Sam Duong Woloszynski

Making Research a Team Sport

March 11, 2022

Megan Blocker

"We don’t just stop the research project when we collect data; analyzing, extracting insights, and shaping the narrative is part of it."

Megan Blocker Mujtaba Hameed Victor Udoewa

Panel: Excellence in Impact

March 25, 2024