Rosenverse
How to Lead With Data, and Without Data

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • •

    Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • •

    Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • •

    Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • •

    Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • •

    Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • •

    Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • •

    Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • •

    Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • •

    Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • •

    A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Jon Fukuda
Coordinating Human-Centered Organizations in the Post-AI Era
2026 • Shift UX 2026
Conference
Eric Shumake
Diagnosis UX: Building Influence in Healthcare Design
2026 • Rosenfeld Community
Teresa Swingler
Look, Up in the Sky! UX/UI for Aerospace
2022 • Enterprise Community
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Steve Turbek
Designing Interactive Graphics with AI Code Help
2026 • Rosenfeld Community
Suzan Bednarz
AccessibilityOps for All
2024 • DesignOps Summit 2020
Gold
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Eduardo Ortiz
Day 3 Theme Panel
2025 • Advancing Research 2025
Gold
Jeff Ephraim Bander
Eye Tracking Gamechanger: Why Smartphone Eye Tracking will Revolutionize Your UX Research
2022 • Advancing Research 2022
Gold
Sofia Quintero
Beyond Tools: The Messy Business of Implementing Research Repositories
2022 • Advancing Research 2022
Gold
Kaaren Hanson
Stop Talking, Start Doing
2017 • Enterprise Experience 2017
Gold
Chris Geison
Theme 1 Intro
2022 • Advancing Research 2022
Gold
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group
Alla Weinberg
Workers Are Sick of Change: The Cure is Psychological Safety
2023 • Enterprise UX 2023
Gold
Deanna Zandt
The Unspoken Complexity of “Self-Care” with Deanna Zandt
2022 • Civic Design Community
Dantley Davis
Leadership & Diversity—A Fireside Chat with Dantley Davis
2020 • Enterprise Community

More Videos

Denise Jacobs

"Silence is complicity. Inaction is support."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Dan Willis

"The unsung hero of Enterprise UX isn’t the user or the researcher or the designer — it’s the person drowning in emails, Excel sheets, and project plans."

Dan Willis

Enterprise Storytelling Sessions

June 8, 2016

Jim Kalbach

"Each first take was the only take, which got pressed on the album."

Jim Kalbach

Jazz Improvisation as a Model for Team Collaboration

June 4, 2019

Louis Rosenfeld

"Newness isn’t necessarily a new methodology—it can be a new way of looking at something familiar."

Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris Geison

Coffee with Lou #3: What Makes for a Successful UX Conference Presentation?

May 2, 2024

Ricardo Martins

"Cluster analysis will definitely replace personas, the fictional personas of course."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

Frank Duran

"A seat at the table versus building the table means setting routines where all the right partners come together regularly."

Frank Duran

Partnership Playbook: Lessons Learned in Effective Partnership

January 8, 2024

Alexandra Schmidt

"I’m totally freaked out by facial recognition and wonder why we don’t get together as consumers to push back on bad designs."

Alexandra Schmidt

Why Ethics Can't Save Tech

November 18, 2022

Davis Neable

"We formed a core task force empowered to make decisions quickly, avoiding weeks of delay waiting for formal approvals."

Davis Neable Guy Segal

How to Drive a Design Project When you Don’t Have a Design Team

June 10, 2021

Sam Proulx

"Errors in form fields should alert users immediately but also allow them to explore details at their own pace to avoid cognitive overload."

Sam Proulx

Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users

December 10, 2021