Rosenverse
How to Lead With Data, and Without Data

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • •

    Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • •

    Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • •

    Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • •

    Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • •

    Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • •

    Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • •

    Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • •

    Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • •

    Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • •

    A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Ali Jeffery
How DesignOps Helped Enable Wall Street to Work Remotely
2020 • DesignOps Summit 2020
Gold
Rittika Basu
Age and Interfaces: Equipping Older Adults with Technological Tools
2023 • Advancing Research Community
Miles Orkin
Creativity and Culture
2018 • DesignOps Summit 2018
Gold
Jamie Beck Alexander
How can you find your role in climate?
2024 • Climate UX Interest Group
Lada Gorlenko
Theme 1: Intro
2024 • Enterprise Experience 2020
Gold
Audrey Crane
Shadow Design–Where Else is Design Happening in Your Organization?
2023 • Enterprise Community
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Uday Gajendar
You Are Not a Prompt Engineer: A talk on the future of designers
2026 • Rosenfeld Community
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
Jemma Ahmed
Bringing together market and user research
2019 • Advancing Research Community
Teresa Swingler
Look, Up in the Sky! UX/UI for Aerospace
2022 • Enterprise Community
Kate Towsey
Shaping the future of research ops: Expanding roles and strategies for a more integrated research ecosystem
2025 • Advancing Research 2025
Gold
Josh Clark
Sentient Design: Crafting Intelligent Interfaces with AI
2026 • Rosenfeld Community
Julia Cowing
From Researcher to Builder: Vibe Coding Tomer Sharon's Method Finder
2026 • Rosenfeld Community
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • DesignOps Summit 2022
Gold
Peter Merholz
The 2025 State of UX/Design Organizational Health
2025 • Rosenfeld Community

More Videos

Monty Hammontree

"Our mission is to empower everybody to learn; not the few learning on behalf of the many."

Monty Hammontree

The Future of UX Research

December 3, 2020

Deanna Zandt

"Self-care became a commodity where buying things was mistaken for healing and support."

Deanna Zandt

The Unspoken Complexity of “Self-Care” with Deanna Zandt

July 21, 2022

Shanti Mathew

"Designers hope to support the public in exercising agency and redistribute hegemonic power in public services."

Shanti Mathew Natalie Sims Natalia Radywyl

Civic Design at Scale: Introducing the Public Policy Layer Cake

December 9, 2021

Kathleen Asjes

"Allowing others to experience research firsthand can convert them into advocates for hiring more researchers."

Kathleen Asjes

Research Democratization: the Good, the Bad and the Ugly

March 10, 2022

Theresa Neil

"Our general public, even advanced enterprise users, often struggle to interpret even the most basic charts."

Theresa Neil

Just Build Me a Dashboard!

April 9, 2019

Aleksandra Korczynska

"AI changes everything in unstructured data analysis; before it was painful manual work, now it's automated theme detection and clustering."

Aleksandra Korczynska Caroline Jarrett Justyna Parmee

Survey Tools

March 12, 2026

Jessica Norris

"In women, ADHD can sometimes just be seen as being chatty or forgetful or clumsy, as opposed to restless and disruptive."

Jessica Norris

ADHD: A DesignOps Superpower

September 9, 2022

Gretchen Anderson

"I see designers fall back on being the defender of the user like a union boss, but that’s not your only responsibility."

Gretchen Anderson

Scaling the Human Center

June 8, 2017

Frances Yllana

"The impact of design ops isn’t as easy to define or predict like other business value measures."

Frances Yllana

Theme 2 Intro

September 24, 2024

Latest Books All books