Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Jemma Ahmed
Theme 2 Intro
2024 • Enterprise Experience 2020
Gold
Changying (Z) Zheng
Navigating Innovation with Integrity
2024 • DesignOps Summit 2024
Gold
Erin Hoffman-John
This Game is Never Done: Design Leadership Techniques from the Video Game World
2017 • DesignOps Summit 2017
Gold
Mark Boulton
Ops without Designers
2018 • DesignOps Summit 2018
Gold
Marieke McCloskey
User Science: Product Analytics & User Research
2021 • Advancing Research 2021
Gold
Anat Fintzi
Delivering at Scale: Making Traction with Resistant Partners
2022 • Design at Scale 2022
Gold
Michele Marut
Research Repositories Reconsidered
2019 • DesignOps Community
Peter Boersma
How to Define and Maintain a DesignOps Roadmap
2023 • DesignOps Summit 2023
Gold
Justin Entzminger
Risk and Reward: How to Diversify the Field of Civic Innovators and Designers
2022 • Civic Design 2022
Gold
David Sternberg
Uncovering the hidden forces shaping user behavior
2025 • Rosenfeld Community
Amy Gawronski Zuccaro
Advice for DesignOps Employee #1
2021 • DesignOps Summit 2021
Gold
Farid Sabitov
Automatization for Large Enterprise Teams
2024 • DesignOps Summit 2020
Gold
Charles Lee
Building a New Home for the Atlassian Design System
2020 • Enterprise Community
Dan Willis
Theme 3: Intro
2024 • Enterprise Experience 2020
Gold
Farid Sabitov
Theme Four Intro
2022 • DesignOps Summit 2022
Gold
Taiye Akin-Akinyosoye
Amplifying voices and enhancing user research through group interviews
2025 • Advancing Research 2025
Gold

More Videos

Yoel Sumitro

"Reflection in action involves noticing surprise, questioning assumptions, experimenting, and reflecting on the move."

Yoel Sumitro

Actions and Reflections: Bridging the Skills Gap among Researchers

March 9, 2022

Zariah Cameron

"A kickback is a free flowing and life giving gathering that transforms and electrifies throughout the night."

Zariah Cameron

ReDesigning Wellbeing for Equitable Care in the Workplace

September 23, 2024

Doug Powell

"Designers have a set of superpowers that give us a unique ability to envision a better future."

Doug Powell

DesignOps and the Next Frontier: Leading Through Unpredictable Change

September 11, 2025

Darian Davis

"If you’re generating buy-in, don’t forget what it will mean for your difficult stakeholders—they may have external pressures you don’t know about."

Darian Davis

Lessons from a Toxic Work Relationship

January 8, 2024

Mike Oren

"New ideas and changes to business trajectories rank pretty far down at only 18% of business leaders seeing it as a value of design."

Mike Oren

Why Pharmaceutical's Research Model Should Replace Design Thinking

March 28, 2023

Jacqui Frey

"It’s not about giving up power, it’s about gaining authority through partnership and trust."

Jacqui Frey

Scale is Social Work

March 19, 2020

Kim Holt

"Clear and consistent communication really keeps design leadership accountable and promotes transparency via knowledge sharing."

Kim Holt Emma Wylds Pearl Koppenhaver Maisee Xiong

A Salesforce Panel Discussion on Values-Driven DesignOps

September 8, 2022

Samuel Proulx

"It can cost up to 200 times more to remediate and retrofit a solution as opposed to designing it correctly in the first place."

Samuel Proulx

From Standards to Innovation: Why Inclusive Design Wins

September 10, 2025

Laine Riley Prokay

"Soft skills doesn’t always represent active listening, so we updated to be more specific."

Laine Riley Prokay

How DesignOps can Drive Inclusive Career Ladders for All

September 30, 2021