Rosenverse
10k Screens Later: How We Became a Data-Driven Design Organization

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Jemma Ahmed
Take Back those Words! Reclaiming and Reenergizing the Vocabulary of UX
2026 • Rosenfeld Community
Daniel Korczynski
From generic to contextual research insights with AI | Live Q&A
2026 • Advancing Research 2026
Gold
Cennydd Bowles
Responsible Design in Reality
2021 • Design at Scale 2021
Gold
Alla Weinberg
Healing Toxic Stress
2024 • DesignOps Summit 2024
Gold
Yulya Besplemennova
[Demo] Stress-testing GenAI in user research synthesis
2024 • Designing with AI 2024
Gold
Sheryl Cababa
Expanding Your Design Lens with Systems Thinking
2023 • Enterprise Community
Bethany Brown
Rewiring operations with service design and AI
2025 • Advancing Service Design 2025
Gold
Jemma Ahmed
Theme 1 Intro
2026 • Advancing Research 2026
Gold
Nova Wehman-Brown
We've Never Done This Before
2019 • Enterprise Experience 2019
Gold
Wendy Johansson
Be a Product Boss!
2022 • Design in Product 2022
Gold
Adam Cutler
People + Places + Practices = Outcomes
2016 • Enterprise UX 2016
Gold
Janelle Estes
UX Research Trends
2021 • Advancing Research Community
Chloe Amos-Edkins
A Cultural Approach: Research in the Context of Glocalisation
2023 • Advancing Research 2023
Gold
Nicole Wright
Democratizing Research at HoneyBook
2022 • Advancing Research 2022
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • Civic Design 2021
Gold
Aurobinda Pradhan
Introduction to Collaborative DesignOps using Cubyts
2022 • DesignOps Summit 2022
Gold

More Videos

Wyatt Hayman

"We send out surveys weekly, but respondents have two weeks to answer, accommodating different time zones and bandwidth."

Wyatt Hayman

Global Research Panels

August 8, 2020

Michal Anne Rogondino

"These applications were designed by engineers for engineers without interviewing the operators."

Michal Anne Rogondino

Saving Outer Space: The First UX Design System for Our Nation’s Satellites

January 8, 2024

Sara Asche Anderson

"We have to slow down to speed up — understanding the why before chasing the outcome."

Sara Asche Anderson Jamie Kaspszak

Not Your Ordinary Re-Brand: Design's Path to Driving Customer Obsession at Best Buy

January 8, 2024

Patrizia Bertini

"Applying systems thinking means changing one part affects the whole system — there’s no linearity in relationships."

Patrizia Bertini

DesignOps + KPIs = Measure your Impact!

January 8, 2024

Amahra Spence

"I often feel trapped inside someone else’s imagination and I must engage my own imagination to break free."

Amahra Spence

Designing for Liberation, Rehearsing Freedom

November 18, 2022

Dan Willis

"Anytime you have to justify something by saying just this one time, you know you’re going down a bad road."

Dan Willis

Enterprise Storytelling Sessions

May 13, 2015

Erika Flowers

"Bring your own AI and keep control over the process rather than being locked into vendor solutions."

Erika Flowers

AI-Readiness: Preparing NASA for a Data-Driven, Agile Future

June 10, 2025

Dalia El-Shimy

"VPs want to know what you really think. They want to see your insights distilled into a clear position of what needs to be done next."

Dalia El-Shimy

So You've Got a Seat at the Table. Now What?

March 31, 2020

Ben Davies

"The hyperbolic discounting bias means researchers prefer immediate gratification over deferred rewards from reusable knowledge."

Ben Davies Matt Duignan Andrew Michael Dr. Emily DiLeo

Expert Panel: The Principles of Research Repository Design

March 11, 2022