Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Shawna Hein
Create a Cohesive Civic Design Practice Across Agency, Vendors, and Contracts
2022 • Civic Design 2022
Gold
Jackie Ho
Lead Effectively While Preserving Team Autonomy with Growth Boards
2024 • Enterprise Experience 2020
Gold
Allison Sanders
Operating with Purpose
2024 • DesignOps Summit 2020
Gold
Sarah Fathallah
A Typology of Participation in Participatory Research
2023 • Advancing Research 2023
Gold
Matt Duignan
Atomizing Research: Trend or Trap
2020 • Advancing Research 2020
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Dave Hoffer
UX Job Search AMA #3 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Lori Muszynski
Keeping Design Weird
2023 • DesignOps Summit 2023
Gold
Carl Turner
You Can Do This: Understand and Solve Organizational Problems to Jumpstart a Dead Project
2023 • Advancing Research 2023
Gold
Dr. Jamika D. Burge
A Genuine Conversation about the Future of UX Research
2024 • Advancing Research Community
Yoel Sumitro
Actions and Reflections: Bridging the Skills Gap among Researchers
2022 • Advancing Research 2022
Gold
Louis Rosenfeld
Coffee with Lou: Should You Write a (UX) Book?
2024 • Rosenfeld Community
Jose Coronado
People First - Design at JP Morgan
2021 • Design at Scale 2021
Gold
Lavy Kumar
Future of Work
2021 • Design at Scale 2021
Gold
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!
2025 • Rosenfeld Community

More Videos

Sam Proulx

"If you have to learn a workaround, you want to learn it once and reuse it again and again."

Sam Proulx

Online Shopping: Designing an Accessible Experience

June 7, 2023

Ignacio Martinez

"The system encourages growth not just in craft but also in client engagement and team collaboration."

Ignacio Martinez

Fair and Effective Designer Evaluation

September 25, 2024

Sarah Kinkade

"Using program dollars to hire freelancers helped me avoid difficult conversations with HR and finance early on."

Sarah Kinkade Mariana Ortiz-Reyes

Design Management Models in the Face of Transformation

June 8, 2022

Daniel Gloyd

"The Shakers’ principled approach to design was a precursor to Bauhaus’s form follows function and today’s user-centered values."

Daniel Gloyd

Warming the User Experience: Lessons from America's first and most radical human-centered designers

May 9, 2024

Patrick Boehler

"Structure beats free form interaction—systematic query frameworks are essential to avoid pitfalls."

Patrick Boehler

Fishing for Real Needs: Reimagining Journalism Needs with AI

June 10, 2025

Andy Barraclough

"Anyone can do a basic AI summary with zero control, but we offer robust, evidence-backed, scalable chat analysis."

Andy Barraclough Betsy Nelson

From Costly Complexity to Efficient Insights: Why UX Teams Are Switching To Voxpopme

September 23, 2024

Alexandra Schmidt

"Standard design research looks for pain points, not harms, and harms often cannot be identified in typical user research."

Alexandra Schmidt

Why Ethics Can't Save Tech

November 18, 2022

Louis Rosenfeld

"In government, the motivation is reduction of misery: why are we up at 3 AM fixing something avoidable?"

Louis Rosenfeld

Discussion: What Operations can teach DesignOps

November 6, 2017

Mitchell Bernstein

"Every decision in a design system is like a puzzle piece; together they form a bigger picture that evolves over time."

Mitchell Bernstein

Organizing Chaos: How IBM is Defining Design Systems with Sketch for an Ever-Changing AI Landscape

September 29, 2021