Summary
As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.
Key Insights
-
•
Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.
-
•
Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.
-
•
Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.
-
•
Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.
-
•
Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.
-
•
Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.
-
•
Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.
-
•
Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.
-
•
Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.
-
•
A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.
Notable Quotes
"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."
"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."
"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."
"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."
"Executives first want to know what metric you will impact next quarter before approving big initiatives."
"Data is a language you need to learn to speak and a mindset that must be continuously practiced."
"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."
"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."
"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."
"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."
Or choose a question:
More Videos
"Service design is not only supporting but leading – it’s about having opinion and data to lead change."
Ben Reason Aline Horta Majid Iqbal Fabiano LeoniMaking the system visible: The fastest path to better decisions
November 20, 2025
"Two-factor authentication accessibility improves when there are customizable options like text, email, or QR codes."
Sam ProulxMobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
March 10, 2022
"We realized that everyone kind of has a different definition of the voice of customer and that we needed to define our own to make it relevant for us."
Anna Nguyen Emily BroganWhy Our Voice of the Customer is Better Than Yours
March 10, 2022
"Design is not only limited to designers. Business analysts and engineers can design too, supported by designers."
Abby Covert Tomer SharonPanel: Collaboration Tools
November 6, 2017
"Accessibility really is a moving target; starting somewhere and iterating is better than waiting for perfection."
Phil HeskethDesigning Accessible Research Workflows
September 29, 2021
"Websites and digital spaces should be government, not just provide information about government."
Irina Tikhonova Kari DietrichSmall Wins, Big Impact: Leveraging and Elevating User Engagement
December 9, 2021
"Education and awareness had to be core parts of what we were doing in this shift."
Tim FrickThe journey of building a sustainable design practice
April 23, 2025
"Physical prototyping is a tool to explore interaction modalities and physically connected environments beyond all things digital."
Catherine DubutBridging Physical and Digital Spaces: Approaches to Retail Service Design
March 18, 2021
"Post-its give everyone an equal playing field, preventing dominant voices from overshadowing others in community conversations."
Justin Entzminger Terrance Smith Tracy M. Colunga Mai-Ling GarciaRisk and Reward: How to Diversify the Field of Civic Innovators and Designers
November 17, 2022
