Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Crystal Philcox
The Many Faces of Operations
2017 • DesignOps Summit 2017
Gold
Ted Neward
Theme 4: Enterprise Organizational Journey
2019 • Enterprise Experience 2019
Gold
Jennifer Fraser
What would Emmy Noether Do? Math, Models and Mulling in UX Research
2023 • Advancing Research 2023
Gold
Patrizia Bertini
DesignOps + KPIs = Measure your Impact!
2024 • DesignOps Summit 2020
Gold
Nancy Douyon
We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex
2018 • Enterprise Experience 2018
Gold
Iram Shah
Closing Keynote: The View from the Top
2019 • Enterprise Experience 2019
Gold
Milan Guenther
The $212 billion ‘so what?’: unlocking impact in development cooperation
2025 • Advancing Service Design 2025
Conference
Tom Armitage
Day 2 Panel: Looking ahead: Designing with AI in 2026
2025 • Designing with AI 2025
Gold
Sam Ladner
How Research Can Drive Strategic Foresight
2022 • Advancing Research 2022
Gold
Dominique Ward
The Most Exciting Time for DesignOps is Now
2022 • DesignOps Summit 2022
Gold
Ian Johnson
Latent Scope: Finding structure in unstructured data
2025 • Designing with AI 2025
Gold
Smitha Papolu
Theme 3 Discussion
2024 • Enterprise Experience 2020
Gold
Victor Udoewa
Beyond Methods and Diversity: The Roots of Inclusion
2024 • Advancing Research 2024
Gold
Monty Hammontree
The Future of UX Research
2020 • Advancing Research Community
Sheryl Cababa
Living in the Clouds: Adopting a Systems Thinking Mindset
2023 • Enterprise UX 2023
Gold
Nicole Umphress
Delivering Design Education During a Global Pandemic: Lessons Learned
2022 • Design at Scale 2022
Gold

More Videos

Doug Powell

"Business leaders are super busy, data-driven, competitive, and have a finely tuned bullshit meter."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"If a group session with middle schoolers goes off the rails, one-on-one sessions can save the research."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"What if I said that this was actually the assignment: to write an essay using AI, helping students become familiar with translating their policy intent through AI?"

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"We never compromise on the business goals, but how we achieve them can be flexible and pragmatic."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"Our goal is to be AI fluent, not necessarily AI experts."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Research is not as easy as putting on a shoe and just doing it; it requires careful consideration and alignment with imperatives."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"It’s the worst thing when someone feels dumb in front of a customer because the tools don’t support them."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"Re-platforming is transformative not just for the product but also for the people, teams, and organization."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"Building rapport and rituals helped our remote team foster camaraderie and trust."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023