Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Lena Shenkarenko
Collaborative Wireframing for Creating Team Alignment and Shipping Better Products
2020 • DesignOps Summit 2020
Gold
Abby Covert
Panel: Collaboration Tools
2017 • DesignOps Summit 2017
Gold
Marc Fonteijn
First Insights from the 2025 Service Design Salary(+) Report
2024 • Advancing Service Design 2024
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold
John Cutler
The Alignment Trap
2023 • Design in Product 2023
Gold
Bria Alexander
State of DesignOps Panel
2021 • DesignOps Summit 2021
Gold
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
Dan Willis
Filling the Void
2018 • DesignOps Summit 2018
Gold
Jodi Forlizzi
Design and AI innovation
2024 • Designing with AI 2024
Gold
Marieke McCloskey
User Science: Product Analytics & User Research
2021 • Advancing Research 2021
Gold
Rachael Dietkus, LCSW
Everything You Need to Know about the Civic Design 2022 Call for Presentations
2022 • Civic Design Community
Christian Crumlish
Afternoon Insights Panel
2022 • Design in Product 2022
Gold
Theresa Neil
Designing for Wellness: Specializing in Healthcare
2024 • Rosenfeld Community
Bud Caddell
Theme 2 Intro
2021 • DesignOps Summit 2021
Gold
Jemma Ahmed
Bringing together market and user research
2019 • Advancing Research Community
Alastair Simpson
Debunking the Myths of Cross-Disciplinary Collaboration
2019 • DesignOps Summit 2019
Gold

More Videos

Sam Proulx

"Accessibility is a process, not a project."

Sam Proulx

Accessibility: An Opportunity to Innovate

November 16, 2022

Sarah Brooks

"Anticipating and acting on risk turns uncertainty into inspiration for informed action."

Sarah Brooks

Theme 3 Intro

December 10, 2021

Louis Rosenfeld

"The social game at the end of the day is a great way to meet people and change your energy."

Louis Rosenfeld

Founder’s Welcome

December 6, 2022

Rachel Posman

"AI is not just a tool; it will be a worker, a reviewer, or even an end user in design processes."

Rachel Posman John Calhoun

"Ask Me Anything" with Rachel Posman and John Calhoun, Authors of the Upcoming Rosenfeld Book, The Design Conductors

September 25, 2024

Abbey Smalley

"The most salary ranges fell between $140,000 to $270,000 a year, with larger companies paying more, but there’s wide variability."

Abbey Smalley

Today’s Design Ops and Programs Landscape & Career Paths

October 4, 2023

Maverick Chan

"Feeding interview data into ChatGPT helped us develop realistic character profiles and dialogues."

Maverick Chan Claire Lin

From Doodle to Demo: AI as Our Storytelling Partner

October 23, 2025

Kristin Skinner

"This conference is a polished version of a conversation that's already happening in the world."

Kristin Skinner

Theme 1 Intro

September 29, 2021

Ruzanna Rozman

"Surgeons say during difficult operations the whole team feels like a single organism moved by the same purpose."

Ruzanna Rozman

Getting in Flow with Your Team

January 8, 2024

Patrizia Bertini

"If designers are spending time on non-design work, they are not doing design — and that’s the real problem."

Patrizia Bertini

DesignOps + KPIs = Measure your Impact!

January 8, 2024