Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Andy Warr
Under My (Research) Umbrella: The Benefits and Challenges of Building a Unified Insights Function
2024 • Advancing Research 2024
Gold
Shipra Kayan
Emerging principles for using AI in Design: What the product design team at Miro has learned from deeply integrating AI in their workflow
2025 • Designing with AI 2025
Gold
Dana Bishop
2022: The Year UX Demonstrates its Business Impact
2022 • Advancing Research 2022
Gold
Sohit Karol
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
2020 • Advancing Research 2020
Gold
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Lija Hogan
Three Years Out: Perspectives on the Near-Term Future of User Research
2024 • Advancing Research Community
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Sarah Rink
Remote User Research: Dos and Don'ts from the Virtual Field
2020 • Advancing Research Community
Jamie Beck Alexander
How can you find your role in climate?
2024 • Climate UX Interest Group
Maria Taylor
Knowledge is Power: Managing the Lifeblood of the Design Org
2023 • DesignOps Summit 2023
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Nathan Reiff
Research, from Unimaginable to Presently Possible: A Future-Casting Sticky-Note Sprint
2026 • Advancing Research 2026
Conference
Peter Levin
Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility
2024 • Advancing Research 2024
Gold
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • Civic Design 2022
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold
Sheri Byrne-Haber
Accessibility at Scale
2021 • Design at Scale 2021
Gold

More Videos

Louis Rosenfeld

"Inclusion means not just who’s on the program but who you’re researching and collaborating with as you create your presentation."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024

Jilanna Wilson

"Human connection is a basic need that doesn’t get paused just because someone works remotely."

Jilanna Wilson

Distributed Design Operations Management

October 23, 2019

Matt Webb

"Computers got ten years better overnight with the arrival of GPT-3 in November 2022."

Matt Webb

Context Window: Five Futures for AI

June 11, 2025

Sam Proulx

"Accessibility is not a single project, it’s a journey that requires continuous iteration and improvement."

Sam Proulx

Accessibility: An Opportunity to Innovate

September 8, 2022

Theresa Neil

"Healthcare technology is absolutely a growing market with tremendous opportunity."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024

Peter Van Dijck

"One of the biggest problems in AI building is evolving your prompts and having a fast feedback loop."

Peter Van Dijck

Hands-on AI #2: Understanding evals: LLM as a Judge

October 15, 2025

Rachel Posman

"Why pick sides between product and design ops leadership? Why not both as user success leadership?"

Rachel Posman John Calhoun

"Ask Me Anything" with Rachel Posman and John Calhoun, Authors of the Upcoming Rosenfeld Book, The Design Conductors

September 25, 2024

Tricia Wang

"Web3 offers a unique chance to get involved early before some of the ethical challenges of Web2 take root."

Tricia Wang

The most popular design thinking strategy is BS

January 27, 2022

Sarah Auslander

"Always design a thing by considering it in this next largest temporal context — a day in a month, a decade in a century."

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022