Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Helen Armstrong
Augment the Human. Interrogate the System.
2023 • Enterprise UX 2023
Gold
Ellie Krysl
Planned Right. Managed Right. Designed Right.
2023 • Enterprise UX 2023
Gold
Jon Fukuda
Theme 3 Intro
2024 • DesignOps Summit 2024
Gold
Yalenka Mariën
Designing for Digital Inclusion in the Belgian Government
2021 • Civic Design 2021
Gold
Ellen Chisa
The Values of Design
2023 • Design in Product 2023
Gold
Kara Kane
Communities of Practice for Civic Design
2022 • Civic Design Community
Jim Kalbach
Jazz Improvisation as a Model for Team Collaboration
2017 • DesignOps Summit 2017
Gold
Barb Spanton
Doing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
2022 • Design at Scale 2022
Gold
Craig Brookes
"Just Make it Look Good" and Other Ways We're Misunderstood
2021 • Design at Scale 2021
Gold
Michelle Chin
The DesignOps Starter Kit
2021 • DesignOps Summit 2021
Gold
Shanti Mathew
Civic Design at Scale: Introducing the Public Policy Layer Cake
2021 • Civic Design 2021
Gold
Greg Nudelman
Designing Conversational Interfaces
2019 • Enterprise Community
Jen Briselli
Learning is the north star: service design for adaptive capacity
2025 • Advancing Service Design 2025
Gold
Steve Portigal
War Stories LIVE! Q&A-Discussion
2020 • Advancing Research 2020
Gold
Corey Nelson
Layoffs
2022 • Advancing Research Community
Brian T. O’Neill
Does Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
2022 • QuantQual Interest Group

More Videos

Christian Crumlish

"When something good happens, I want to name the person who did it and say yes, they passed through me. When something bad happens, I say let me and my team go look into it."

Christian Crumlish Wendy Johansson Rich Mironov Aditi Ruiz Adam Thomas

Afternoon Insights Panel

December 6, 2022

Michael Land

"Hiring specs for federal designers don’t exist as we expect—jobs are often classified as public affairs or other unrelated titles."

Michael Land

Establishing Design Operations in Government

February 18, 2021

Bria Alexander

"Please read our code of conduct. Every participant should feel safe, included, and welcome."

Bria Alexander Louis Rosenfeld

Welcome

September 8, 2022

Surya Vanka

"When groups started swarming on problems, they exhibited autonomy, creativity, experimentation, and hidden leadership."

Surya Vanka

Unleashing Swarm Creativity to Solve Enterprise Challenges

June 10, 2021

Cassandra Piester

"Being tactical means updating Kanban tickets for one team; being strategic means updating annual Gantt charts across the portfolio."

Cassandra Piester

Developing and Deploying Your Design Operations Strategy

September 24, 2024

Ariel Kennan

"Our presenters have iterated on their talks to provide rich resources and new knowledge."

Ariel Kennan

Theme 2 Intro

December 9, 2021

Joshua Graves

"Anger doesn’t mean you have to yell—it’s about speaking clearly to the problem you see."

Joshua Graves

We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)

April 14, 2025

Peter Van Dijck

"Context design and context engineering mean figuring out and building what needs to go into that text file sent to the model."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!

July 30, 2025

Bria Alexander

"Please put your questions inside the thread specific to the talk so we can keep everything organized."

Bria Alexander

Opening Remarks

November 17, 2022