Rosenverse
Driving Change with CX Metrics

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Roy Opata Olende
How Zapier Uses ‘All Hands Research’ to Increase Exposure to Users
2020 • Advancing Research Community
Husani Oakley
Bias Towards Action: Building Teams that Build Work
2018 • Enterprise Experience 2018
Gold
Robin Beers
Research as a Catalyst for Organizational Transformation
2021 • Advancing Research 2021
Gold
Caroline Jarrett
Have fun with statistics?
2024 • Rosenfeld Community
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold
Rittika Basu
Age and Interfaces: Equipping Older Adults with Technological Tools
2023 • Advancing Research Community
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
John Cutler
Prioritization for designers and product managers (1st of 3 seminars)
2024 • Rosenfeld Community
Barb Spanton
Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
2022 • Design at Scale 2022
Gold
Jon Fukuda
Theme One Intro
2023 • DesignOps Summit 2023
Gold
Alnie Figueroa
Teamwork: Strategies for Effective Collaboration with Other Program Management Teams
2022 • DesignOps Summit 2022
Gold
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold
George Abraham
Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)
2021 • DesignOps Summit 2021
Gold
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Joy KendiMwiti
Deciding when to automate: Integrating AI in high-stakes systems
2026 • Designing with AI 2026
Conference
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold

More Videos

Cennydd Bowles

"I love the things—technology, design—but I really dislike the industry."

Cennydd Bowles

Exit Interview #2: Rediscovering the ethical heart of design

November 6, 2025

Shawna Hein

"We’ve seen self-service tool usage grow from hundreds to millions of veterans served monthly after launching a unified VA platform."

Shawna Hein Kevin Hoffman

Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts

November 17, 2022

Alla Weinberg

"Psychological safety is the number one factor determining team effectiveness and performance."

Alla Weinberg

Design Teams Need Psychological Safety: Here’s How to Create It

September 9, 2022

Kit Unger

"Relationship is the new currency."

Kit Unger Lada Gorlenko

Theme 3 Intro

June 10, 2022

Joseph Meersman

"Design shares are more about presenting finished work and practice defending it, not about peer-to-peer feedback."

Joseph Meersman

Sweating the Pixel: Scaling Quality through Critique

June 10, 2021

Jason Mesut

"Just assume we haven’t read it and keep it quick."

Jason Mesut

Shaping design, designers and teams

November 8, 2018

Joshua Noble

"Difference-in-differences lets you study treatment effects in groups that start in very different places by looking at trends."

Joshua Noble

Casual Inference

October 6, 2023

Paul Ford

"Process and discipline got thrown out the window, but we really need new processes that incorporate these new AI tools."

Paul Ford

New work, new words: A glossary for AI

June 9, 2026

Jacqui Frey

"Say it until someone says it better back to you."

Jacqui Frey Alison Rand

Setting the Table for Dynamic Change

October 24, 2019