Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization

Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Christian Bason
Expand—Rethinking Design for Public Challenges
2022 • Civic Design Community
Bria Alexander
Opening Remarks
2022 • Civic Design 2022
Gold
Randolph Duke II
War Stories LIVE! Randy Duke II
2020 • Advancing Research 2020
Gold
Sarah Gallimore
Inspire Progress with Artifacts from the Future
2022 • Civic Design 2022
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Luca Rager
Empowering Gaming at Scale: How Xbox Builds Powerful, Automated, and Distributed Design Systems with Sketch
2021 • DesignOps Summit 2021
Gold
Dane DeSutter
Beyond the Console: The rise of the Gamer Experience and how gaming will impact UX Research across industries
2024 • QuantQual Interest Group
Ryan Matthew
Bridging Design and Code: AI-Powered Design System Integration
2025 • DesignOps Summit 2025
Gold
Chris Govias
Perspectives on Civic Design
2021 • Civic Design Community
Daniel Gloyd
Designing Warmth
2025 • Rosenfeld Community
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Sara Conklin
Exit Interview: 20 Years of Tech, One Very Big Bet, and a Lot of Heat Pumps
2026 • Rosenfeld Community
Jilanna Wilson
Distributed Design Operations Management
2019 • DesignOps Summit 2019
Gold
Dan Donald
Design Systems as a Vehicle for Systemic Change
2023 • DesignOps Community
Sabrina Mach
How to Design Your Design Operating Model
2021 • DesignOps Summit 2021
Gold

More Videos

Christian Madsbjerg

"The story of the immunotherapy patient and his daughters became a riff that executives remember to understand complex life impacts."

Christian Madsbjerg

Influencing Strategy

March 31, 2020

Mansi Gupta

"Trust is about perceptions versus realities shaped by higher expectations and harsher consequences for women."

Mansi Gupta

Women-Centric Research: What, Why, How

March 29, 2023

Nora Tejeda

"When things get more complicated than expected, we re-negotiate scope or assign a full designer to keep quality."

Nora Tejeda Giovanna Alonso

Scaling Design Capabilities at BBVA Through a Self-service Design Model

June 10, 2021

Sarah Fathallah

"We formed a lived expert review board to support framing conclusions and decisions, rebalancing power from academic peer reviewers."

Sarah Fathallah

A Typology of Participation in Participatory Research

March 28, 2023

Alison Rand

"Design operations is really about how we’re scaling the work, thinking our practices, and serving cross-functional teams."

Alison Rand Sarah Brooks

Scaling Impact with Service Design

March 25, 2021

Sam Proulx

"On mobile, there's only one screen reader per platform—no confusion across multiple software versions."

Sam Proulx

Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World

September 8, 2022

Sydney Lawson

"Genesys includes a fake tool in the software list to stop incentive-driven participants from checking all boxes."

Sydney Lawson

Anatomy of a Strong User Panel

March 10, 2026

Kristen Guth, Ph.D.

"The FOG method helps teams separate facts from fiction and move past hangups toward alignment."

Kristen Guth, Ph.D.

Out of the FOG: A Non-traditional Research Approach to Alignment

March 28, 2023

Sam Ladner

"Asking for help and humbling yourself is one of the most powerful ways to grow as a researcher."

Sam Ladner Megan Blocker Tanvi Dhond Katie Hansen Ricardo Martins

Methodologies: Beyond the interview [Advancing Research Community Workshop Series]

September 4, 2024