Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
James Chudley
Decarbonising User Journeys: How minimising enables us to do more with less
2025 • Climate UX Interest Group
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Gold
Alla Weinberg
Workers Are Sick of Change: The Cure is Psychological Safety
2023 • Enterprise UX 2023
Gold
Anna Avrekh
Diversity In and For Design: Building Conscious Diversity in Design and Research
2021 • Design at Scale 2021
Gold
Marisa Bernstein
It Takes GRIT: Lessons from the Small, but Mighty World of Civic Usability Testing
2021 • Civic Design 2021
Gold
Laura Smith
Embedding Service Design and Agile Practice within UK Planning Teams to Create Services that Last
2024 • Advancing Service Design 2024
Gold
Sohit Karol
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
2020 • Advancing Research 2020
Gold
Neil Barrie
Widening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
2024 • Advancing Research 2024
Gold
Steve Krug
Don’t Make Me Think 3.0: What Endures and What Evolves in UX
2026 • Advancing Research 2026
Conference
Gordon Ross
12 Months of COVID-19 Design and Digital Response with the British Columbia Government
2021 • Civic Design 2021
Gold
Tricia Wang
The most popular design thinking strategy is BS
2022 • Enterprise Community
Saara Kamppari-Miller
DesignOps for Inclusive Design and Accessibility
2022 • DesignOps Community
Jen Cardello
Learning Velocity—The Insights Speedometer
2021 • Advancing Research Community
Rachel Posman
A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)
2020 • DesignOps Community
Gabriela Barneva
Operationalizing Inclusive Design in Design Ops
2025 • DesignOps Summit 2025
Gold
Llewyn Paine
Day 1 Using AI in UX with Impact
2025 • Designing with AI 2025
Gold

More Videos

Carol Scott

"If you don't have good usability to begin with, being trauma-informed is not going to work out."

Carol Scott Melissa Eggleston

Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices

February 5, 2025

Lija Hogan

"Regulations are design problems, not just technical ones, influencing how we create products and experiences."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Sam Proulx

"Mobile screen readers can confirm basic accessibility but desktop testing is still needed due to differences across tools."

Sam Proulx

Mobile Accessibility and You

June 9, 2022

Ana Ferreira

"Start with trust by default; most people won’t disappoint you and will deliver responsibly."

Ana Ferreira

Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries

January 8, 2024

Jorge Arango

"I made sure the LLM would only use terms from my predefined taxonomy, but it still introduced some of its own tags."

Jorge Arango

Scale Smart: AI-Powered Content Organization Strategies

September 24, 2024

Chris Moses

"We’re exploring if design ops should host a cross-product scrum team focused on visionary redesigns to have bigger impact."

Chris Moses

Stretching the Definition of DesignOps with Product Development

November 7, 2018

Trisha Causley

"The confident tone built into these tools may be one of the more problematic features of Gen AI."

Trisha Causley

[Demo] Complexity in disguise: Crafting experiences for generative AI features

June 5, 2024

Clara Kliman-Silver

"I really want to be able to sign off on decisions and make sure that I’m preserving my agency as a UX professional."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Michael Polivka

"Expressing gratitude publicly and privately gives people a sense of purpose and acknowledges their contributions."

Michael Polivka

Scaling Design through Relationship Maps

November 7, 2017