Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data

Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Ask the Rosenbot
Eric Shumake
Diagnosis UX: Building Influence in Healthcare Design
2026 • Rosenfeld Community
Catherine Dubut
Bridging Physical and Digital Spaces: Approaches to Retail Service Design
2021 • Enterprise Community
Mary-Lynne Williams
Exit Interview #4: From Product Design Leadership to Sound Healing
2026 • Rosenfeld Community
Saara Kamppari-Miller
Cartography for Design Communities
2025 • DesignOps Summit 2025
Gold
Amy Bucher
Harnessing behavioral science to uncover deeper truths
2025 • Advancing Research 2025
Gold
Sohit Karol
Designing Delightful Listening Experiences: Mixed Methods Research in the Age of Machine Learning
2020 • Advancing Research 2020
Gold
Magdalena Zadara
Zero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late
2022 • Civic Design 2022
Gold
Katie Hansen
Experimental research: techniques for deep, psychology-driven insights
2025 • Advancing Research 2025
Gold
Sha Hwang
The Lost Year
2021 • Design at Scale 2021
Gold
Emilia Åström
Unlock Your Team’s Intelligence with Collaboration Design
2022 • Design at Scale 2022
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Joanna Vodopivec
One Research Team for All - Influence Without Authority
2022 • Advancing Research 2022
Gold
Bruce Gillespie
Learning from journalism: Balancing impactful communication with compassionate storytelling
2025 • Advancing Research 2025
Gold
Rich Mironov
How Can Product Managers and UXers Help Each Other (and Why are Product Folks so Annoying Sometimes)?
2022 • Design in Product 2022
Gold
Jaime Creixems
Best Practices when Creating and Maintaining a Design System
2023 • Enterprise UX 2023
Gold
Sofía Delsordo
Public Policy for Jalisco's Designers to Make Design Matter
2021 • Civic Design 2021
Gold

More Videos

Kevin Bethune

"Preciousness around ‘pure’ design language can shut down collaboration with curious stakeholders."

Kevin Bethune

Gatekeepers and Servant Leadership

January 30, 2020

Alfred Kahn

"Turning design from tactical to strategic is like turning a super tanker; it takes a long, wide arc."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Elena Naids

"It pains me to say, one of the tools they were using was MS Paint."

Elena Naids Liza McRuer

The Power of Difficult Conversations: A Case Study on How We Introduced Design Ops in the Federal Government Space

October 2, 2023

Amanda Kaleta-Kott

"Please get more precise than saying 65 plus when recruiting older adults."

Amanda Kaleta-Kott Lea Martin, PhD

The Joys and Dilemmas of Conducting UX Research with Older Adults

March 11, 2022

Fredrik Matheson

"Something’s going to go wrong. You know, something went wrong."

Fredrik Matheson

First-time users, longtime strategies: Why Parkinson’s Law is making you less effective at work – and how to design a fix.

June 8, 2016

Sam Proulx

"On mobile, there's only one screen reader per platform—no confusion across multiple software versions."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

September 8, 2022

Peter Levin

"Sometimes making a terrible first big change is a success because it enables the next person to improve upon it."

Peter Levin

Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility

March 25, 2024

Janelle Estes

"Don't just flip the switch and say go talk to customers; you have to set guardrails and train people."

Janelle Estes

UX Research Trends

January 28, 2021

Deanna Zandt

"Self-care became a commodity where buying things was mistaken for healing and support."

Deanna Zandt

The Unspoken Complexity of “Self-Care” with Deanna Zandt

July 21, 2022