Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Melinda Belcher
Insider preview of Enterprise Experience 2020
2020 • Enterprise Community
Uday Gajendar
Theme 1: Introduction
2021 • Design at Scale 2021
Gold
Kurdin Bazaz
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • Rosenfeld Community
Billy Carlson
Pro-level UI Tips for Beginners
2022 • DesignOps Summit 2022
Gold
Fisayo Osilaja
[Demo] The AI edge: From researcher to strategist
2024 • Designing with AI 2024
Gold
Corey Long
Hiring in DesignOps: A Critical Study on How to Hire and Get Hired
2024 • DesignOps Summit 2024
Gold
Milan Guenther
A Shared Language for Co-Creating Ambitious Endeavours
2023 • Enterprise UX 2023
Gold
Shazia Ali
Communication: Innovative techniques for making your voice heard [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Patrizia Bertini
Pushing DesignOps’ Influence into New Global Markets
2022 • DesignOps Summit 2022
Gold
Andrew Webster
Scaling Design Capability: How Involved Should You Be?
2021 • DesignOps Summit 2021
Gold
Abby Covert
Panel: Collaboration Tools
2017 • DesignOps Summit 2017
Gold
Noz Urbina
Rapid AI-powered UX (RAUX): A framework for empowering human designers
2025 • Rosenfeld Community
Mohammad Hossein Jarrahi
Contextuality problem: Exploring the Benefits of Qualitative and Quantitative Research
2023 • QuantQual Interest Group
Saara Kamppari-Miller
Theme Three Intro
2023 • DesignOps Summit 2023
Gold
Alberto Ferreira
Making it Count: Developing a custom digital metric framework that works
2021 • QuantQual Interest Group

More Videos

Randolph Duke II

"Flat Carol was a nearly life-size cardboard cutout of Carol."

Randolph Duke II

War Stories LIVE! Randy Duke II

March 30, 2020

Corey Nelson

"Don’t force yourself into a job you know isn’t right just because you’re unemployed."

Corey Nelson Amy Santee

Layoffs

November 15, 2022

Landon Barnes

"Research is not as easy as putting on a shoe and just doing it; it requires careful consideration and alignment with imperatives."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Amy Bucher

"Logic models are boring but powerful tools to align expectations that behavior change is complex and not immediate."

Amy Bucher

Harnessing behavioral science to uncover deeper truths

March 12, 2025

David Sternberg

"The interface you’ve designed is like a landscape shaping the flow of user behavior as water flows through a stream."

David Sternberg

Uncovering the hidden forces shaping user behavior

July 17, 2025

Deanna Smith

"More frequent change isn't always better; good communication and team pulse are key."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Jennifer Strickland

"Equity is equal outcomes, not just equal access to a bicycle everyone can’t necessarily ride."

Jennifer Strickland

Adopting a "Design By" Method

December 9, 2021

Rachel Posman

"Our team Ops Charter is about amplifying and celebrating the work and impact of design and design Ops."

Rachel Posman John Calhoun

A Closer Look at Team Ops and Product Ops (Two Sides of the DesignOps Coin)

November 19, 2020

Gina Mendolia

"Service design is 10% connecting the dots and 90% creating the conditions that bring those dots closer together."

Gina Mendolia

Therapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems

December 3, 2024