Rosenverse
How to Lead With Data, and Without Data

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • •

    Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • •

    Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • •

    Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • •

    Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • •

    Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • •

    Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • •

    Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • •

    Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • •

    Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • •

    A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Louis Rosenfeld
Opening Remarks
2023 • Advancing Research 2023
Gold
Maish Nichani
Sparking a Service Excellence Mindset at a Government Agency
2021 • Civic Design 2021
Gold
Peter Van Dijck
Hands on AI #3: Claude Code for UX people
2025 • Rosenfeld Community
Terry Buckman
Wargaming (An Introduction)
2023 • Enterprise Community
Robin Beers
Navigating organizational systems: Rethinking researcher’s role in driving change
2025 • Advancing Research 2025
Gold
James Wieselman Schulman
Research is a team sport: advancing the work when everyone does the research
2026 • Advancing Research 2026
Gold
Aletheia Delivre
New Shapes and Emerging Identities for Design Ops
2025 • DesignOps Summit 2025
Gold
Lona Moore
Scaling Design Beyond Designers
2021 • Design at Scale 2021
Gold
Dave Hora
A Research Skills Evolution
2021 • Advancing Research 2021
Gold
Megan Blocker
A Selectively Scrappy Approach to ResearchOps
2018 • DesignOps Summit 2018
Gold
Dan Willis
Enterprise Storytelling Sessions
2016 • Enterprise UX 2016
Gold
Sarah Coyle
Design and Analytics with Sarah Coyle
2020 • DesignOps Community
Rachael Greene
Building a Design Ops Practice that Really Works (Most of the Time)
2025 • DesignOps Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Design at Scale 2022
Gold
Russ Unger
Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started
2016 • Enterprise UX 2016
Gold
Chelsey Glasson
Exit Interview #3: Same as It Ever Was: What Leaving Tech Taught Me About Change
2025 • Rosenfeld Community

More Videos

Dr. Karl Jeffries

"An elite athlete doesn’t turn up at the Olympics and do their thing without coaches, physios, nutritionists."

Dr. Karl Jeffries

The Science of Creativity for DesignOps

January 8, 2024

Amy Marquez

"Investment is attention: If the business funds you, it means they believe in what you do."

Amy Marquez

INVEST: Discussion

June 15, 2018

Steve Portigal

"The core of it is really putting yourself in that other person’s shoes—it’s basic but essential empathy."

Steve Portigal Chris Chapo Kelly Goto Christian Rohrer

Discussion

May 13, 2015

Alicia Mooty

"Splitting people’s allocation across different teams created work-life balance issues requiring more oversight."

Alicia Mooty

Design Staffing Models

September 30, 2021

Ana Maria Montero Barrantes

"The only rule is to maintain your smile and just to feel the being that have fun."

Ana Maria Montero Barrantes Aditi Dhar Michelle Kaplan Nate Osborne Matt Laurence

The Authentic UX Talent Show

January 8, 2024

Lada Gorlenko

"If your users are happy, that anecdotal feedback often carries more weight early than quantitative metrics."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Sofia Quintero

"It’s very difficult to align researcher goals with business growth, but that’s exactly what we try to do."

Sofia Quintero

The Product Philosophy Behind EnjoyHQ

March 10, 2021

Laura Klein

"Lawyers, though nervous, can become some of the most reasonable people you deal with if you explain risks clearly."

Laura Klein

Unique challenges of innovation in enterprises

April 23, 2020

Andrew Michael

"Without Jim, what we're going to see today would have been a lot worse."

Andrew Michael

Building a Product Insights Team

March 10, 2022

Latest Books All books