Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

10k Screens Later: How We Became a Data-Driven Design Organization
Gold
Tuesday, September 24, 2024 • DesignOps Summit 2024
Share the love for this talk
10k Screens Later: How We Became a Data-Driven Design Organization
Speakers: John Paul de Guzman
Link:

Summary

In this session, we'll explore how a small UI design firm (30 headcount) developed a data-driven DesignOps practice. I'll demonstrate how we use automation and robust data pipelines to excel in managing and delivering large volumes of work. This talk also shows how we transform Figma’s comment feature into a powerful tool that integrates seamlessly with Trello, enabling our team to achieve high efficiency and precision, and to track everything in real-time without the usual administrative and logistical challenges. Automation and Custom Tooling: We leverage automation to ensure every process, from capturing to moving work, is streamlined and efficient. This approach helps us tackle the scalability of thousands of work cards without overwhelming our team. Data-Driven with a Soul: This talk addresses concerns around micromanagement and surveillance, illustrating how our methods support consistent delivery (with 95% confidence) without sacrificing the human aspect of our work.

Key Insights

  • Frost Design Group achieved 95% accuracy in forecasting delivery dates using team performance data despite volatile project conditions.

  • They replaced traditional two-week sprints with a Kanban continuous delivery approach, better suiting their workflow.

  • Figma comment boxes were transformed into a tracking system with AI that automatically creates and categorizes work cards, reducing manual admin time drastically.

  • Monte Carlo simulations are used to forecast project delivery probabilities, enabling data-driven decision making and avoiding guesswork in estimations.

  • Tracking cycle time by production stage helps identify bottlenecks and informs resource allocation, improving workload balance.

  • Data-driven insights justified front-loading resources and demonstrated a 49% increase in delivery volume in one financial year segment.

  • They measure designer productivity by combining output and time data, avoiding both quantity over quality bias and purely hours-based assessment.

  • Interventions based on performance data helped junior designer Leia improve output by 364% year-over-year, leading to promotion and mentoring responsibilities.

  • Automations reduced unnecessary manual work by 2 hours daily for lead designer Dawn, showing the importance of streamlining non-design tasks.

  • Their design operations team expanded from one to three roles, separating responsibilities across work pipeline, team quality, and client commitments.

Notable Quotes

"We can forecast delivery dates despite shifting and volatile project conditions with 95% efficiency."

"Our file is a visual representation of our entire workflow and mirrors the structure of our board."

"Tracking and capturing the work can be a whole new work in itself. People will naturally abandon complicated process."

"We replaced the exhausting time estimation game with data-driven models that give us more reliable results."

"Spending more time doesn’t automatically make you productive, it just means you spent more time doing things."

"Data is supposed to be neutral. It’s the way one uses it that can bring good or bad outcomes."

"We use this data not to dehumanize, but to elevate each person’s uniqueness and fairly incentivize performance."

"Design is really a team sport. Everyone is responsible to keep the workspace up to date."

"Being design-driven is a means to an end. We aim to be 1% better every day with data-driven obsession."

"Our mission is to raise the overall value of design in the Philippines where it’s often seen as an expense item."

Ask the Rosenbot
Mackenzie Guinon
M.C. Escher’s UX Research Career Ladder
2022 • Advancing Research 2022
Gold
Erin Weigel
UX Lessons from running more than 1,200 A/B Tests
2024 • Rosenfeld Community
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Design at Scale 2021
Gold
Carol Scott
Avoid Harming Your Team and Users: Promoting Care and Brand Reputation with Trauma-Informed UX Practices
2025 • Rosenfeld Community
Jon Fukuda
The Big Question about Innovation: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Abby Covert
Panel: Collaboration Tools
2017 • DesignOps Summit 2017
Gold
Yoel Sumitro
Actions and Reflections: Bridging the Skills Gap among Researchers
2022 • Advancing Research 2022
Gold
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Jamika Burge
Embracing change: Navigating shifting landscapes with compassion and agency
2025 • Advancing Research 2025
Gold
Tim Frick
The journey of building a sustainable design practice
2025 • Climate UX Interest Group
Jemma Ahmed
Theme Panel
2025 • Advancing Research 2025
Gold
Robin Beers
Navigating organizational systems: Rethinking researcher’s role in driving change
2025 • Advancing Research 2025
Gold
Amy Brana Stuart
Rest in Peace Fly-in-fly-out Design
2022 • Design at Scale 2022
Gold
Samuel Proulx
From Standards to Innovation: Why Inclusive Design Wins
2025 • Advancing Service Design 2025
Conference
Katie Hansen
Experimental research: techniques for deep, psychology-driven insights
2025 • Advancing Research 2025
Gold
Michael Land
Establishing Design Operations in Government
2021 • DesignOps Community

More Videos

Doug Powell

"There’s no faking it with this crowd."

Doug Powell

Closing Keynote: Design at Scale

November 8, 2018

Mila Kuznetsova

"Recruiting kids really means recruiting adults as their proxies and gatekeepers."

Mila Kuznetsova Lucy Denton

How Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices

March 9, 2022

Sarah Gallimore

"We created a Wizard of Oz smoke and mirrors prototype—a simple text message thread between a parent and this service—just to get enough buy-in."

Sarah Gallimore

Inspire Progress with Artifacts from the Future

November 18, 2022

Lada Gorlenko

"You can’t really advocate for UX or move an organization forward without interacting with your team and business colleagues."

Lada Gorlenko Sharbani Dhar Sébastien Malo Rob Mitzel Ivana Ng Michal Anne Rogondino

Theme 1: Discussion

January 8, 2024

Alnie Figueroa

"Who deals with chaos, ambiguity, and disruption best? It’s us, design operations humans."

Alnie Figueroa

The Future of Design Operations: Transforming Our Craft

September 10, 2025

Landon Barnes

"Driver modeling tells you which research initiatives matter most based on their impact on key outcome metrics."

Landon Barnes

Are My Research Findings Actually Meaningful?

March 10, 2022

Emily Eagle

"Every workaround is a mini story of somebody overcoming an obstacle."

Emily Eagle

Can't Rewind: Radio and Retail

June 3, 2019

Malini Rao

"Managing resistance is mental; customers might resist because they've learned workarounds with legacy systems."

Malini Rao

Lessons Learned from a 4-year Product Re-platforming Journey

June 9, 2021

Asia Hoe

"Collaborate and involve design early and often to better align on user goals and reduce friction."

Asia Hoe

Partnering with Product: A Journey from Junior to Senior Design

November 29, 2023