Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Charles Lee
Building a New Home for the Atlassian Design System
2020 • Enterprise Community
Sarah Coyle
Design and Analytics with Sarah Coyle
2020 • DesignOps Community
Wendy Johansson
Design at Scale: Behind the Scenes
2021 • Enterprise Community
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
Ned Gartside
Navigating accessibility and climate
2024 • Climate UX Interest Group
Mark Interrante
Collaboration Flows in Product Development
2017 • Enterprise Experience 2017
Gold
Ariel Kennan
Theme Two Intro
2022 • Civic Design 2022
Gold
Michelle Morrison
Practice What You Preach
2024 • DesignOps Summit 2020
Gold
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
Sofía Delsordo
Public Policy for Jalisco's Designers to Make Design Matter
2021 • Civic Design 2021
Gold
Megan Blocker
Panel: Excellence in Impact
2024 • Advancing Research 2024
Gold
Robert Fabricant
Industry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Randolph Duke II
War Stories LIVE! Randy Duke II
2020 • Advancing Research 2020
Gold
Nathan Curtis
Discussion
2016 • Enterprise UX 2016
Gold
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Nora Tejeda
Scaling Design Capabilities at BBVA Through a Self-service Design Model
2021 • Design at Scale 2021
Gold

More Videos

Nalini Kotamraju

"Our shift is structural, not because of personalities — it was needed to influence earlier, more business-centric decisions."

Nalini Kotamraju

Research After UX

March 25, 2024

Dean Broadley

"Society grows great when old people plant trees they know they shall never sit in."

Dean Broadley

Not Black Enough to be White

January 8, 2024

Denise Jacobs

"When we start changing our behavior, we start changing the voices and behavior around us."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Kim Fellman Cohen

"Our first problem statement was too big, so we refined it to something more doable and actionable."

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

George Aye

"We mostly have something called a BYOE scenario or bring your own ethics."

George Aye

That Quiet Little Voice: When Design and Ethics Collide

November 16, 2022

Nathan Curtis

"Sometimes alignment work isn’t enjoyable, but it’s part of the job for design system success."

Nathan Curtis

Beyond the Toolkit: Spreading a System Across People & Products

June 9, 2016

Greg Petroff

"Platforms need researchers and designers to make them meaningful and intentional for the people they are designed to serve."

Greg Petroff

The Compass Mission

March 10, 2021

Chloe Amos-Edkins

"Global and local cultures are closely entwined and always influencing one another."

Chloe Amos-Edkins

A Cultural Approach: Research in the Context of Glocalisation

March 27, 2023

Mackenzie Cockram

"We are not collecting vanity analytics, the data is doing real work on informing design."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022