Rosenverse
Driving Change with CX Metrics

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • •

    MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • •

    They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • •

    Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • •

    Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • •

    Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • •

    Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • •

    The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • •

    Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • •

    Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • •

    Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Audrey Crane
Shadow Design–Where Else is Design Happening in Your Organization?
2023 • Enterprise Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Advancing Research 2022
Gold
Alëna Iouguina
Designing Systems at Scale
2018 • DesignOps Summit 2018
Gold
Devon Powers
Imagining Better Futures
2022 • Advancing Research 2022
Gold
Christian Crumlish
Teaching My Agents to Disagree With Me
2026 • Rosenfeld Community
Jim Kalbach
Jobs To Be Done
2021 • Enterprise Community
Mansi Gupta
Advancing the Inclusion of Womxn in Research Practices
2022 • Advancing Research Community
Sam Ladner
Methodologies: Beyond the interview [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • DesignOps Summit 2021
Gold
Alnie Figueroa
Teamwork: Strategies for Effective Collaboration with Other Program Management Teams
2022 • DesignOps Summit 2022
Gold
Mike Oren
Design Research Strategy & Strategic Design Research
2022 • Advancing Research Community
Jen Cardello
Learning Velocity—The Insights Speedometer
2021 • Advancing Research Community
Louis Rosenfeld
How to use the Rosenbot
2026 • Rosenfeld Community
Matt Bernius
Learnings from Applying Trauma-Informed Principles to the Research Process
2022 • Advancing Research 2022
Gold
Ashley Cortez
Shifting Toward Community-Led Innovation in Local Government
2021 • Civic Design 2021
Gold
Sarah Sgarlato Pierini
From Passion to Execution: A Story of Evolving Research Maturity at LinkedIn
2022 • DesignOps Summit 2022
Gold

More Videos

Dr. Karl Jeffries

"If you said are you creative, I’m gonna say hell yeah."

Dr. Karl Jeffries

The Science of Creativity for DesignOps

January 8, 2024

Amy Marquez

"Find your senior executive ally who understands design and go talk to them first — it works."

Amy Marquez

INVEST: Discussion

June 15, 2018

Steve Portigal

"Building an internal research practice is critical for long-term value and integration of insights."

Steve Portigal Chris Chapo Kelly Goto Christian Rohrer

Discussion

May 13, 2015

Alicia Mooty

"Some needs proved full-time, so hybrid staffing models are often short term or mixed part-time."

Alicia Mooty

Design Staffing Models

September 30, 2021

Ana Maria Montero Barrantes

"I believe there's really no such thing as a boring place at the state of mind."

Ana Maria Montero Barrantes Aditi Dhar Michelle Kaplan Nate Osborne Matt Laurence

The Authentic UX Talent Show

January 8, 2024

Lada Gorlenko

"Operations tends to be something companies only think about when they are scaling rapidly, but investing early is essential."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Sofia Quintero

"Choosing a research repository is a big battle in your organization; the right decisions can really help you succeed."

Sofia Quintero

The Product Philosophy Behind EnjoyHQ

March 10, 2021

Laura Klein

"Please don’t chop off that engine until the new one is built because you are not the only one on this plane."

Laura Klein

Unique challenges of innovation in enterprises

April 23, 2020

Andrew Michael

"Without Jim, what we're going to see today would have been a lot worse."

Andrew Michael

Building a Product Insights Team

March 10, 2022

Latest Books All books