Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Prayag Narula
How to Empower Your Designers to Do Good Research – And Why You Want To
2022 • Design at Scale 2022
Gold
Dorelle Rabinowitz
The Magic Word is Trust
2018 • Enterprise Experience 2018
Gold
Kara Kane
Communities of Practice for Civic Design
2022 • Civic Design Community
Holly Cole
Understanding Experiences: When you have to do more than work
2018 • DesignOps Summit 2018
Gold
Luca Rager
Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch
2021 • DesignOps Summit 2021
Gold
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold
Adel Du Toit
Get Your CFO To Say: 'Our Strategic Goal is User Obsession'
2022 • Design at Scale 2022
Gold
Scher Foord
Turn the Ship Around: How to Apply Design Thinking Across Your Organization
2021 • Design at Scale 2021
Gold
Bria Alexander
Theme 1 Intro
2024 • DesignOps Summit 2024
Gold
Megan Blocker
Theme 2 Intro
2025 • Advancing Research 2025
Gold
Amy Jiménez Márquez
The Atypical UX Manager Path
2020 • Enterprise Community
Nidhi Singh Rathore
Embracing participation to unlock deeper truths in commercial research
2025 • Advancing Research 2025
Gold
Alla Weinberg
Workers Are Sick of Change: The Cure is Psychological Safety
2023 • Enterprise UX 2023
Gold
Leisa Reichelt
Opening Keynote: Operating in Context
2018 • DesignOps Summit 2018
Gold
Steve Portigal
War Stories LIVE! Steve Portigal
2020 • Advancing Research 2020
Gold
Taiye Akin-Akinyosoye
Amplifying voices and enhancing user research through group interviews
2025 • Advancing Research 2025
Gold

More Videos

Maria Giudice

"I treat everything in life as if it’s a design problem."

Maria Giudice

Remaking the Making Company: Moving from Product to Experience

June 9, 2016

Anat Fintzi

"Asking questions made our direct communication partners think I didn’t know what I was talking about; telling statements worked better."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

Sam Proulx

"If you have to learn a workaround, you want to learn it once and reuse it again and again."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Holly Cole

"I do this in public, put this on a wall where you can test it so people can come and say, that is not right."

Holly Cole

Understanding Experiences: When you have to do more than work

November 8, 2018

Frances Yllana

"Project pioneer organizations resist top-down control, needing design ops to provide tools allowing creative teams to pivot and adapt."

Frances Yllana

DesignOps–Leading the Path to Parity

April 27, 2023

Russ Unger

"The post-offer period isn’t HR’s alone; hiring teams must stay in touch and keep the candidate engaged."

Russ Unger

Onboarding: The Ecosystem, not the Afterthought

November 7, 2017

Dorelle Rabinowitz

"Feedback isn’t about liking. Helpful feedback needs to be, is this solving your problem or not."

Dorelle Rabinowitz

The Magic Word is Trust

June 15, 2018

Tatyana Mamut

"Don’t make up fictional personas; use real people’s names, photos, and stories to capture real customer complexity."

Tatyana Mamut

Opening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart

June 3, 2019

Tony Turner

"Knowledge repositories help you connect different research findings across studies, products, and even different companies."

Tony Turner

Capturing Deep Insights

September 30, 2021