Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Driving Change with CX Metrics
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
Driving Change with CX Metrics
Speakers: Todd Healy and Jess Greco
Link:

Summary

Mastercard is in the middle of a shift in how decisions are made throughout the product development lifecycle, with data-informed design at the center. A suite of measurements have been developed to quantify the quality of products and the resulting experiences at points across the product lifecycle, so teams can have customer-centric conversations about opportunities and risks. Hear more about these CX Metrics and how we’ve infused them into the product development lifecycle to exert influence across dozens of products across the company.

Key Insights

  • MasterCard shifted focus from product launch success to sustained adoption, retention, and partner implementation to drive revenue.

  • They developed three standardized CX metrics: Concept Desirability Score, Design Quality Score, and Partner Onboarding Gold Standards to measure product quality throughout development.

  • Concept Desirability Scores are rapid, low-cost tests of multiple early ideas to identify opportunity areas and course correct early.

  • Design Quality Scores dive deep on a single product’s user journey, onboarding, and emotional interactions to improve execution.

  • Partner Onboarding Gold Standards quantify implementation quality with cross-functional self-assessments tied to executive incentives.

  • Poor initial design quality can be revealed through these metrics—as with the subscription service example—leading to redesigned, higher-scoring solutions.

  • The CX metrics integrate with MasterCard’s stage gate product framework enabling targeted, timely decision making and reducing costly rework.

  • Their system actively discourages common anti-patterns like hippo-driven feature creep, mistaking pilot willingness for market demand, and misaligned incentives.

  • Human coaching remains vital to prevent gaming the system and to help product teams interpret metrics as tools for learning and iteration.

  • Leadership support and embedding these metrics into performance goals help create accountability and cultural adoption despite initial resistance.

Notable Quotes

"We’ve found a way to use data to speak the language of business while tying it back into our values as designers to build better products."

"Launching something small at MasterCard is inherently huge because of our global scale and complex partnerships."

"Hippo culture is building things based on the highest paid person's opinion, no insights, just vibes."

"Willingness to pilot is not the same as willingness to pay."

"Concept Desirability Scores take about eight days and can replace months of traditional research for early risk identification."

"Design Quality Scores look at all touchpoints, emotional interactions, and the cognitive journey to fine-tune product adoption."

"We don’t want product teams doing research just to check a box—they need to own and act on the insights."

"Partner Onboarding Gold Standards act like a carrot-shaped stick tied to executive bonuses to push speed to revenue without sacrificing quality."

"People think automation will replace authenticity, but right now, human intervention is still required to ensure quality."

"The scores are a means to an end to encourage behavior change, not to create fear or just produce numbers."

Ask the Rosenbot
Jen Cardello
Standardizing Product Merits for Leaders, Designers, and Everyone
2018 • Enterprise Experience 2018
Gold
Dan Mall
“Ask Me Anything” with Dan Mall, Author of Upcoming Rosenfeld Title, Design that Scales
2023 • DesignOps Summit 2023
Gold
Reginé Gilbert
Asking the Right Questions: Life, Hope and Moving Forward During the Pandemic
2022 • Design at Scale 2022
Gold
Sharon Bautista
Time to Make the Donuts: How User Research Helped Bridge Disparate Teams
2024 • Enterprise Experience 2020
Gold
Megan Blocker
A Selectively Scrappy Approach to ResearchOps
2018 • DesignOps Summit 2018
Gold
Bud Caddell
DesignOps Community Sensing Session
2021 • DesignOps Community
Paul Pangaro, PhD
Systems Disciplines: Table Stakes for 21st Century Organizations
2023 • Enterprise UX 2023
Gold
David Cronin
Discussion
2015 • Enterprise UX 2015
Gold
Bria Alexander
Opening Remarks
2023 • DesignOps Summit 2023
Gold
Kate Kalcevich
Integrating Accessibility in DesignOps
2024 • DesignOps Summit 2024
Gold
Carla Casariego
DesignOps in Wonderland
2019 • DesignOps Summit 2019
Gold
Jemma Ahmed
Bringing together market and user research
2019 • Advancing Research Community
Bria Alexander
Charting the future of DesignOps: A community workshop
2024 • DesignOps Community
Uday Gajendar
The Wicked Craft of Enterprise UX
2015 • Enterprise UX 2015
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Llewyn Paine
Coexisting with AI: A practical guide for researchers to navigate tools, ethics, and integration
2025 • Advancing Research 2025
Gold

More Videos

Carol Scott

"Trauma is a universal human experience."

Carol Scott Melissa Eggleston

Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices

February 5, 2025

Lija Hogan

"If you press the exact same button today and tomorrow you get two different results, that’s non-deterministic UX."

Lija Hogan Milan Mijatovic Sam Proulx Louis Rosenfeld

Three Years Out: Perspectives on the Near-Term Future of User Research

March 15, 2024

Sam Proulx

"The cognitive load saved by not having to manage security software on mobile phones is a huge accessibility win."

Sam Proulx

Mobile Accessibility and You

June 9, 2022

Ana Ferreira

"Meetings don’t exist for decision making but for brainstorming, catching up, and one-on-ones."

Ana Ferreira

Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries

January 8, 2024

Jorge Arango

"I made sure the LLM would only use terms from my predefined taxonomy, but it still introduced some of its own tags."

Jorge Arango

Scale Smart: AI-Powered Content Organization Strategies

September 24, 2024

Chris Moses

"The first year federal incentives ended, the market changed completely; buyers started listening to end users more."

Chris Moses

Stretching the Definition of DesignOps with Product Development

November 7, 2018

Trisha Causley

"The confident tone built into these tools may be one of the more problematic features of Gen AI."

Trisha Causley

[Demo] Complexity in disguise: Crafting experiences for generative AI features

June 5, 2024

Clara Kliman-Silver

"An AI-driven design tool should help me by lowering the redundant work, reducing tedious tasks and help me focus on the experience and strategy."

Clara Kliman-Silver

UX Futures: The Role of Artificial Intelligence in Design

June 7, 2023

Michael Polivka

"Design is frequently misunderstood; sometimes it’s just about explaining why it’s important to get others involved."

Michael Polivka

Scaling Design through Relationship Maps

November 7, 2017