Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Kevin Bethune
Reimagining Design: Unlocking Strategic Innovation
2022 • Design at Scale 2022
Gold
Zariah Cameron
ReDesigning Wellbeing for Equitable Care in the Workplace
2024 • DesignOps Summit 2024
Gold
Tony Turner
Capturing Deep Insights
2021 • DesignOps Summit 2021
Gold
Nicole Aleong
What UX research can learn from other research practices [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Soma Ghosh
What emerging methods are advancing UX research [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Robert Fabricant
Industry junctures: Paths forwards for UXR and the critical decisions that get us there [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Changying (Z) Zheng
Practical DesignOps: From Ideas to Tools That Teams Actually Use
2025 • Rosenfeld Community
Vanessa Varin
Tools are moments. Capabilities compound.
2026 • Designing with AI 2026
Conference
Frances Yllana
DesignOps–Leading the Path to Parity
2023 • DesignOps Community
Dane DeSutter
What co-speech gestures reveal about users’ thinking during interviews
2023 • Rosenfeld Community
Vicky Teinaki
Short Take #3: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Megan Blocker
Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research
2024 • Advancing Research 2024
Gold
Peter Van Dijck
Hands-on AI #1: Let’s write your first AI eval
2025 • Rosenfeld Community
George Abraham
Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)
2021 • DesignOps Summit 2021
Gold
Cara Maritz
The Art of Extrapolation
2023 • Advancing Research 2023
Gold
Dr. Jamika D. Burge
Broad Strokes: Connecting Design, Research, and AI to the World Around Us
2023 • Enterprise UX 2023
Gold

More Videos

Melissa Schmidt

"We created simple report templates to help people who have never done research before share their insights."

Melissa Schmidt Adam Menter

How UX Research Hit It Big in Las Vegas

June 4, 2019

Max Gadney

"Sometimes you need to view your career like an investor, where you invest your time for the kind of impact you want."

Max Gadney Andrea Petrucci Joshua Stehr Hannah Wickes

Assessing UX jobs for impact in climate

August 14, 2024

Maria Skaaden

"Positive deviants operate on the fringes and get the best results by doing things differently."

Maria Skaaden

Panel Discussion: Methodologies and Work Environments

November 8, 2018

George Abraham

"Our handoff process turns absolute layout into a flex layout structure, so you get responsiveness out of the box without copying screens for each size."

George Abraham Stefan Ivanov

Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)

September 30, 2021

Rachael Dietkus, LCSW

"Discomfort should not be dismissed; it’s a signpost for necessary change."

Rachael Dietkus, LCSW

The power to heal and harm

March 13, 2025

Peter Levin

"Organizations are much better at making something better than they are at doing something new."

Peter Levin

Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility

March 25, 2024

Louis Rosenfeld

"There is no need to take notes over the next few days. We have dedicated sketch notes and session resources."

Louis Rosenfeld Bria Alexander

Opening Remarks

March 27, 2023

Andrea Gallagher

"I suspect there are specific tasks we could support more effectively with dedicated tools."

Andrea Gallagher

The Problem Space

May 16, 2019

Steve Sanderson

"Just because you've been right in the past doesn't guarantee you'll be right in the future."

Steve Sanderson Alissa Briggs Jeff Gothelf Bill Scott

Discussion

May 14, 2015