Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Sheri Byrne-Haber
Accessibility at Scale
2021 • Design at Scale 2021
Gold
Dana Chisnell
The Sensemaking Business
2026 • Advancing Research 2026
Conference
Sha Hwang
The First Fifty Years of Civic Design
2022 • Civic Design 2022
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Eduardo Ortiz
Theme 3 Intro
2025 • Advancing Research 2025
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Aiyana Bodi
Three Key Climate Initiatives and How You Can Help
2024 • Climate UX Interest Group
Uday Gajendar
Day 2 Welcome
2024 • Designing with AI 2024
Gold
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Caroline Vize
The State of UX: Five Lessons from 2021 to Accelerate Digital Experience in 2022
2022 • Advancing Research 2022
Gold
Doug Powell
Closing Keynote: Design at Scale
2018 • DesignOps Summit 2018
Gold
Kristin Skinner
Theme 2: Introduction and Provocation
2024 • DesignOps Summit 2020
Gold
Sarah Alvarado
How to make UX research leadership more effective [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Matt Bernius
Learnings from Applying Trauma-Informed Principles to the Research Process
2022 • Advancing Research 2022
Gold
Isaac Heyveld
Expand DesignOps Leadership as a Chief of Staff
2022 • DesignOps Summit 2022
Gold

More Videos

Brad Peters

"Having a well-thought-out data package really helps us see a fuller picture."

Brad Peters Anne Mamaghani

Short Take #1: UX/Product Lessons from Your Industry Peers

December 6, 2022

Lona Moore

"Rituals symbolize community identity and shared values, helping scale human-centered habits across boundaries."

Lona Moore

Scaling Design Beyond Designers

June 11, 2021

Josh Clark

"The future should not be self-driving. It’s up to us, not the technology, to figure out the right way to use AI."

Josh Clark Veronika Kindred

Sentient Design, AI, and the Radically Adaptive Experience (1st of 3 seminars)

January 15, 2025

Erin May

"It’s not a theme unless you can put your hand over every sticky note and the label explains the story."

Erin May Roberta Dombrowski Laura Oxenfeld Brooke Hinton

Distributed, Democratized, Decentralized: Finding a Research Model to Support Your Org

March 10, 2022

Tara Tressel

"I was interested in thinking of this method as a survey plus that you can use to get survey-like data but with more qualitative depth."

Tara Tressel

Investigating qualitative depth of AI-moderated interviews

March 10, 2026

Charles Lee

"We absolutely need feedback loops so engineers and designers can influence the design system and keep it relevant."

Charles Lee Jennie Yip

Building a New Home for the Atlassian Design System

October 22, 2020

Kristin Skinner

"We are not replacing teachers; we are enhancing their ability to teach effectively."

Kristin Skinner

Five Years of DesignOps

September 29, 2021

Megan Blocker

"Establish yourself as a trusted expert advisor whose work drives meaningful positive change."

Megan Blocker

Getting to the “So What?”: How Management Consulting Practices Can Transform Your Approach to Research

March 26, 2024

Shanti Mathew

"Designers hope to support the public in exercising agency and redistribute hegemonic power in public services."

Shanti Mathew Natalie Sims Natalia Radywyl

Civic Design at Scale: Introducing the Public Policy Layer Cake

December 9, 2021