Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”

Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
Louis Rosenfeld
GenAI for UXers: A Rosenbot Demo and Discussion
2025 • DesignOps Summit 2025
Gold
Amy Evans
How to Create Change
2024 • DesignOps Summit 2024
Gold
Aleksandra Korczynska
Survey Tools
2026 • Advancing Research 2026
Gold
Bria Alexander
Opening Remarks
2021 • Design at Scale 2021
Gold
Peter Van Dijck
Building the Rosenbot
2024 • Designing with AI 2024
Gold
Peter Merholz
Design at Scale is People!
2021 • Design at Scale 2021
Gold
Lada Gorlenko
Theme 2 Intro
2022 • Design at Scale 2022
Gold
Dave Gray
Liminal Thinking: Sense-making for systems in large organizations
2015 • Enterprise UX 2015
Gold
Sarah Brooks
Fireside chat with Sarah Brooks and Jen Pahlka
2021 • Civic Design Community
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Lori Muszynski
Keeping Design Weird
2023 • DesignOps Summit 2023
Gold
You’ve got Gold! A Rosenverse demo with Lou Rosenfeld
2026 • Rosenfeld Community
Brendan Jarvis
Framing Tomorrow by Questioning Today
2022 • Design at Scale 2022
Gold
Brennan Hartich
Communicating and Establishing DesignOps as a New Function
2018 • DesignOps Summit 2018
Gold
Maria Taylor
Knowledge is Power: Managing the Lifeblood of the Design Org
2023 • DesignOps Summit 2023
Gold

More Videos

Samuel Martin

"We have a responsibility to help heal communities and build trust so we can do design with intention."

Samuel Martin

Co-Design vs Faux-Design: Navigating the Complexities of Sharing Power in Co-Design

March 27, 2026

Kayla Farrell

"Working for large companies became frustrating because politics and procedures pulled me away from the research I love."

Kayla Farrell Chelsey Glasson Sean Fitzell Jared LeClerc

What It's Like To Be a User Researcher at Compass

March 12, 2021

Lada Gorlenko

"You can’t really advocate for UX or move an organization forward without interacting with your team and business colleagues."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Lily Aduana

"If I forgot a critical screener question, I just messaged all qualified participants to confirm they meet the criteria before approval."

Lily Aduana Savannah Hobbs Brittany Rutherford

5 Reasons to Bring Your Recruiting in-House (and How To Do It)

March 12, 2021

Tony Turner

"Bridging those silos within a company is really important because there’s so much wisdom locked in different groups."

Tony Turner

Capturing Deep Insights

September 30, 2021

Marjorie Stainback

"We flipped some onboarding traditions from physical notes to online kudos boards, keeping the welcoming spirit alive."

Marjorie Stainback Molly Fargotstein Stephanie Marsh

What Research Ops Professionals Have Learned from COVID-19

July 16, 2020

Theresa Marwah

"Building respect is the foundation to get insights that truly help us deliver to our customers' needs."

Theresa Marwah

How Atlassian is Operationalizing Respect in Research

February 27, 2020

Louis Rosenfeld

"Dave Nicholson is a true hero—he captures incredible detail and speed in his notes."

Louis Rosenfeld

Welcome / Housekeeping

June 7, 2023

Uday Gajendar

"Helen will bridge AI/ML towards engaging with data scientists to augment humanity with complex data."

Uday Gajendar

Theme Four Intro

June 6, 2023