Rosenverse
Driving Change with CX Metrics

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • •

    MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • •

    They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • •

    Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • •

    Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • •

    Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • •

    Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • •

    The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • •

    Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • •

    Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • •

    Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Joe Elmendorf
Shitty Maps: A Trojan Horse for Sensemaking
2026 • Rosenfeld Community
Kritika Sony
Moving AI offscreen: Exploring failures, constraints, and recovery in physical game design
2026 • Designing with AI 2026
Gold
Amanda Kaleta-Kott
The Joys and Dilemmas of Conducting UX Research with Older Adults
2022 • Advancing Research 2022
Gold
Allan Lowson
Rehashing the Double Diamond: Collaborating across functions with AI-assisted prototyping
2026 • Designing with AI 2026
Gold
Feyikemi Akinwolemiwa
Uninvited, Not Unwelcome: What Researchers Bring When Nobody Asked
2026 • Rosenfeld Community
Connect with the Advancing Research Community
2026 • Advancing Research 2026
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Design in Product 2023
Gold
Tracy McGoldrick
IBM User Experience Program—The What, Why and How
2021 • Advancing Research Community
Melinda Belcher
Bridging the Gap: Making the Most of the Differences Between Agency and Enterprise
2024 • Enterprise Experience 2020
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • DesignOps Summit 2025
Gold
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Ned Gartside
Navigating accessibility and climate
2024 • Climate UX Interest Group
Nalini P. Kotamraju
Two Jobs in One: Being a “Leader who is a Researcher” and a “Researcher who is a Leader"
2021 • Advancing Research 2021
Gold
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold

More Videos

Tony Turner

"You can capture deep insights by looking at how users’ experiences change as they become familiar with a tool over time."

Tony Turner

Capturing Deep Insights

September 30, 2021

Frances Yllana

"Gatekeeping happens through language as much as through who’s in the room."

Frances Yllana Ann Buechner Jess Jones Betsy Ramaccia

D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)

November 16, 2022

Anna Poznyakov

"I’m drowning trying to create an understanding of our target audience for a new feature, but that leaves me no time for usability updates."

Anna Poznyakov Richa Prajapati

Get The Most Out Of Stakeholder Collaboration—and Maximize Your Research Impact

March 12, 2021

Sarah Williams

"We realized we didn’t have a well-articulated purpose or guiding principles for the types of customer experiences we aspired to deliver."

Sarah Williams

A Framework for CX Transformation

June 11, 2021

"Design needs to balance business strategy and technology limitations but always keep users centered."

Product and Design at Bloomberg: A 15-year Evolution

December 6, 2022

David Cronin

"Prototyping and co-creation workshops have been a huge accelerant to the design process."

David Cronin

The GE Design System and Thoughts about Craft at Scale

May 13, 2015

Liwei Dai

"After more than 10 years of doing UX research, I was still only working above the iceberg, focusing on the usability and feature level."

Liwei Dai

The Heart and Brain of the AI Research

March 31, 2020

Peter Van Dijck

"LLMs judging other LLMs might sound crazy, but if you do it right, it works."

Peter Van Dijck

Hands-on AI #1: Let’s write your first AI eval

October 8, 2025

Jon Fukuda

"Empathy means constantly going back to the people using the tools and hearing their feedback."

Jon Fukuda Amy Evans Ignacio Martinez Joe Meersman

The Big Question about Innovation: A Panel Discussion

September 25, 2024

Latest Books All books