Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

How to Lead With Data, and Without Data
Gold
Wednesday, June 7, 2023 • Enterprise UX 2023
Share the love for this talk
How to Lead With Data, and Without Data
Speakers: William Newton and Jenny Chang
Link:

Summary

As a product designer, do you trust data when it defies your intuition? What about those times when following data leads you astray? How can you balance the need for experimentation and iteration with the dangers of over-optimization? What are some of the ways that being "data driven" can go wrong? Follow along and learn about the Amplitude design team's journey blending qualitative and quantitative research to drive design strategy—while balancing quality and velocity.

Key Insights

  • Designers are increasingly using data not only to improve UX but to influence product strategy at higher business levels.

  • Too much focus on metrics can lead to over-optimization, harming user experience despite metric improvement.

  • Amplitude developed a framework categorizing projects by ambiguity: blue sky, horizontal, vertical, and paper cuts, guiding the choice of metrics and evaluation.

  • Blue sky projects require no gating short-term metrics because their impact unfolds over longer horizons.

  • Executive buy-in for major initiatives improves when qualitative feedback is segmented and linked to long-term business goals alongside metrics.

  • Redesigning Amplitude’s core chart controls increased user efficiency and conversion by 25%, demonstrating successful data-informed design.

  • Data literacy involves learning the language of metrics relevant to your business and connecting UX improvements to measurable business outcomes.

  • Data must be treated critically: the same data visualized differently can mislead if axes or contexts are adjusted to bias interpretation.

  • Building personal relationships with data scientists and aligning cross-functional goals encourages collaboration and better data use.

  • A continuous feedback loop combining qualitative research and quantitative data strengthens design hypotheses and strategic initiatives.

Notable Quotes

"It’s a great time to be a designer to make data-driven design decisions that impact beyond just pixels."

"We became addicted to our loop of over-optimization, driving traffic to certain features but frustrating users."

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

"The chart controls were circular and confusing; users looked like deer in headlights not knowing where to start."

"Executives first want to know what metric you will impact next quarter before approving big initiatives."

"Data is a language you need to learn to speak and a mindset that must be continuously practiced."

"You can draw a convincing dotted line between a UX improvement and a hard business outcome if you practice."

"We improved the core chart experience performance by 25%, faster time to save and to first edit have major impact."

"Data is expressive, but just because it’s a chart doesn’t mean the data is true or fair."

"Building personal relationships with data scientists and aligning team goals melts away silos and incentives."

Carol Massa
Designing Health: Integrating Service Design, Technology, and Strategy to Transform Patient and Clinician Experiences
2024 • Advancing Service Design 2024
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community
Luke Roberts
Panel Discussion
2024 • Advancing Service Design 2024
Gold
Kristin Wisnewski
Measuring What Matters
2019 • DesignOps Summit 2019
Gold
Crystal Yan
Building a Customer-Centric Culture
2020 • Advancing Research 2020
Gold
Scott Jensen
Short Take #2: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold
Audrey Crane
Shadow Design–Where Else is Design Happening in Your Organization?
2023 • Enterprise Community
Bria Alexander
Opening Remarks
2022 • DesignOps Summit 2022
Gold
Rob Mitzel
The Tale of Two Companies: Building a Successful UX Practice in a Century-Old Enterprise
2024 • Enterprise Experience 2020
Gold
Irina Tikhonova
Small Wins, Big Impact: Leveraging and Elevating User Engagement
2021 • Civic Design 2021
Gold
Erika Kincaid
Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture
2022 • Design at Scale 2022
Gold
Darian Davis
Lessons from a Toxic Work Relationship
2024 • Enterprise Experience 2020
Gold
Megan Blocker
A Selectively Scrappy Approach to ResearchOps
2018 • DesignOps Summit 2018
Gold
Noel Lamb
Cultivating Business Partnerships to Grow Research Ops
2022 • Advancing Research Community
Laura Schaefer
DesignOps: A Conduit for Inclusion
2022 • DesignOps Summit 2022
Gold

More Videos

John Mortimer

"If you want to be bringing dots closer together, you’re constantly looking at how things are changing in the moment."

John Mortimer Milan Guenther Lucy Ellis Patrick Quattlebaum

Panel Discussion

December 3, 2024

Sean Dolan

"If a journey map is living and used, revisit it every three months to check if it still reflects reality."

Sean Dolan

A Practical Look at Creating More Usable Enterprise Customer Journeys

October 31, 2019

Catt Small

"The difference between director and principal often lies in whether you’re inspired more by leading people or focused on outputs and craft."

Catt Small Micah Bennett Brian Carr Jessica Harllee

What's Next for ICs: Exploring Staff and Principal Designer Roles

February 22, 2024

Greg Petroff

"Stop trying to make your weaknesses a strength, just minimize weaknesses and maximize your strengths."

Greg Petroff

Design is the Differentiator: Bringing New Design Innovations to a Very Antiquated and Very Large Industry

June 9, 2021

Andy Warr

"A research review can increase awareness, improve quality, maintain high standards, provide learning, and promote creativity."

Andy Warr

Under My (Research) Umbrella: The Benefits and Challenges of Building a Unified Insights Function

March 25, 2024

Alexia Cohen

"The COP became the beating heart of our project, enabling quick credibility and community excitement."

Alexia Cohen Adriane Ackerman

Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona

December 4, 2024

Kevin Bethune

"Preciousness around ‘pure’ design language can shut down collaboration with curious stakeholders."

Kevin Bethune

Gatekeepers and Servant Leadership

January 30, 2020

Laura Weiss

"The ultimate goal in conflict is to engage with the tension while reducing the friction between humans."

Laura Weiss

Turn Down the Heat: 3 Ways to Handle Conflict in the Moment

November 20, 2024

Sheryl Cababa

"If you’re working at scale, many problems that seem like 'clock' problems actually have 'cloud' elements."

Sheryl Cababa

Living in the Clouds: Adopting a Systems Thinking Mindset

June 6, 2023