Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Wednesday, November 13, 2024 • DesignOps Community
Share the love for this talk
Key Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
Speakers: Saara Kamppari-Miller , Nicole Bergstrom and Shashi Jain
Link:

Summary

Why do so many metrics include the word “Key”? How do those metrics differ, and how do teams and organizations decide which key metric is best for them? Join this DesignOps community discussion to compare different types of three letter acronym metrics such as KPI, KEI, and KSP (Key Performance Indicator, Key Experience Indicator, and Key Strategic Priorities). And let's not forget OKR's (Objectives and Key Results)!

Key Insights

  • Nicole emphasizes linking accessibility initiatives to corporate Key Strategic Priorities to demonstrate business value beyond compliance.

  • Shashi's KPI Compass framework categorizes innovation metrics into quadrants balancing leading/lagging and thought leadership/impact.

  • Key Experience Indicators (KEIs) are distinct from KPIs and focus on user-centered measures that predict experience failure risks.

  • Measuring behavior change is crucial to predicting innovation success, not just tracking final business metrics.

  • Consistency in metrics terminology and templates across teams helps avoid sidetracking conversations with engineering partners.

  • Canary indicators identify critical experience fail points that can doom a product if not monitored carefully.

  • Strategic priorities, like Hilton’s KSPs, tend to remain stable over years to guide multi-year initiatives reliably.

  • Storytelling and communication training are vital for teams to convey the value of accessibility and design work aligned with business objectives.

  • In ambiguous or innovative contexts, metrics should remain a conversation starter rather than imposed fixed covenants.

  • Using user and market feedback can shift early metrics from vanity measures (like downloads) to meaningful adoption and usage data.

Notable Quotes

"If we are working on something that does not ladder up to our KSPs, then we should not be working on it."

"The canary in the coal mine—that’s how we identify the fail points in the user experience we absolutely can’t miss."

"Innovation is invention multiplied by adoption multiplied by inclusion."

"Metrics are always a conversation, they shouldn’t be a covenant."

"Adoption is the only thing that matters in innovation. If you’re not changing behavior, you haven’t innovated."

"Key is a reminder that these metrics reflect what is authoritative and important to the customer and the business."

"If you only measure vanity metrics like downloads, you risk tracking numbers that don’t translate to real usage or value."

"Our KSPs do not change year over year because changing them frequently is disruptive for the business direction."

"We use storytelling to turn accessibility from just compliance into a business differentiator that executives care about."

"When there’s dispute over what to measure, use that as an opportunity to discuss the behavior you want to change and the value capture."

Ask the Rosenbot
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Chris Geison
What is Research Strategy?
2021 • Advancing Research 2021
Gold
Mark Interrante
Collaboration Flows in Product Development
2017 • Enterprise Experience 2017
Gold
Molly Fargotstein
Multipurpose Communication & UX Research Marketing
2019 • DesignOps Community
Matt Bernius
Trauma-informed Research: A Panel Discussion
2021 • Advancing Research Community
Tracy McGoldrick
IBM User Experience Program—The What, Why and How
2021 • Advancing Research Community
Bilan Hashi
The Tension Between Story Collecting and Story Telling in Research
2021 • Advancing Research 2021
Gold
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Ryan Matthew
DesignOps without Boundaries: Building More with What You Have
2025 • DesignOps Summit 2025
Gold
Denise Jacobs
Interactive Keynote: Social Change by Design
2024 • Enterprise Experience 2020
Gold
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
Sarah Auslander
Insights Panel
2022 • Civic Design 2022
Gold
Marissa Cui
Climate Design Product Showcase
2024 • Climate UX Interest Group
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Jen Crim
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Shawna Hein
Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts
2022 • Civic Design 2022
Gold

More Videos

Chelsey Glasson

"Working in tech was a scenic detour on a long-term career marathon, with more detours ahead but now I have the tools to navigate them."

Chelsey Glasson

Exit Interview #3: Same as It Ever Was: What Leaving Tech Taught Me About Change

December 17, 2025

Laura Klein

"Using AI to scan large archives of research is a game changer for meta-analyses, even if it’s imperfect."

Laura Klein

Human vs. machine: Testing AI’s ability to synthesize and analyze research

March 11, 2026

Dr. Jamika D. Burge

"Imagine going to the hospital for an emergency and the doctor doesn’t believe you’re in pain because AI says so."

Dr. Jamika D. Burge

Broad Strokes: Connecting Design, Research, and AI to the World Around Us

June 7, 2023

Iram Shah

"Technology helps efficiency but cannot replace human connection—eye contact and handshake remain essential."

Iram Shah

Closing Keynote: The View from the Top

June 4, 2019

Alicia D. Johnson

"Emergency management is political—not partisan, but political—and that reality has transformed how we work in this field."

Alicia D. Johnson

Disasters and the 21st Century

December 10, 2021

Lija Hogan

"The era of 100-page reports is ending because we simply can’t produce them fast enough at scale."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Sofía Delsordo

"The design mindset was way more important than the bureaucratic mindset."

Sofía Delsordo Kassim Vera

Public Policy for Jalisco's Designers to Make Design Matter

December 8, 2021

Ron Bronson

"Attention theft is when experiences are designed to siphon away your focus aggressively."

Ron Bronson

Design, Consequences & Everyday Life

November 18, 2022

Richard Buchanan

"In fourth order design, the designer steps back and becomes a facilitator of discussions among others."

Richard Buchanan

Creativity and Principles in the Flourishing Enterprise

June 15, 2018