Designing Systems at Scale
Summary
To be alive in the twenty-first century is to rely on countless complex systems that profoundly affect our lives, where small mistakes can turn into massive inefficiencies and failures. These failures seem to stem from very different problems, but their underlying causes turn out to be surprisingly similar. This talk will cover about the common threads of these failures, how they can be detected and ultimately prevented using various research disciplines – from market research to anthropology to data science. We need to get good at designing and building for systems to tackle multi-dimensional problems at the intersection of people and technology.
Key Insights
-
•
Judy Ellis taught that the key to good design is combining aesthetics, manufacturing knowledge, business understanding, and an appreciation of play.
-
•
Successful processes must be transparent yet become invisible in their application to empower creativity and decision-making.
-
•
Shopify divides work into product lines addressing audiences and service lines ensuring knowledge flow and organizational alignment.
-
•
Data types fall into two complementary streams: object-oriented 'what' data and context-oriented 'where' data.
-
•
The brain processes 'what' and 'where' data in different hippocampus areas before combining them for decision-making.
-
•
Qualitative, context-rich data (the where stream) reaches decision centers earlier and is critical for effective decisions.
-
•
Cross-disciplinary collaboration among UX researchers, data scientists, anthropologists, and market researchers maximizes insight quality.
-
•
Knowledge distribution is best managed by shaping the environment and workflows rather than micromanaging the process itself.
-
•
Leadership support and clear executive ownership are vital to sustaining centralized research and knowledge cycles.
-
•
Tools like Shopify’s merchant frustrations database improve asset visibility and continuous improvement for customer-centric decisions.
Notable Quotes
"Toy design is not about the toy itself but a tool that unlocks unlimited possibilities before a child."
"Successful operations must have well laid out and transparent process documentation but that process must become invisible in its application."
"Service lines bridge gaps between product lines by facilitating knowledge flow and alignment."
"Our brains are terrible at operating on only one type of information—object-oriented or context-oriented."
"It’s the context-related data gathered through qualitative research that reaches our decision-making centers first."
"UX researchers at Shopify travel across product lines to share expertise and knowledge continuously."
"Knowledge creation, distribution, application, and evaluation form a reusable circular process, not a linear one."
"Leadership is really important—having an executive who understands the value of the knowledge cycle is key."
"Shopify’s merchant frustrations tool tracks customer issues over time, making that knowledge available to all teams."
"Properly maintained, knowledge becomes a vehicle that probes the most hidden spaces of what’s possible."
Or choose a question:
More Videos
"We’re seeing a bifurcated experience between teams ready for virtual work and those struggling with it."
Bud Caddell Kristin Skinner Alana WashingtonDesignOps Community Sensing Session
May 13, 2021
"We pivoted big parts of our weekly all-hands to just personal connection because work stuff can happen anytime."
Shipra Kayan Tess DixonHow Tess Dixon Facilitates Team Engagement and Collaboration at Condé Nast Using Miro
October 1, 2021
"Our first day’s theme is around establishing and growing design operations."
Bria AlexanderOpening Remarks
September 29, 2021
"There is no magical tool like GitHub that lets you scale designers the way you can scale developers."
John Maeda Alison RandAbout Design Organizations
May 13, 2019
"If the code doesn’t match the label on screen, voice controls might not recognize the command."
Kate KalcevichDesigning inclusively with AI
June 5, 2024
"A seat at the table versus building the table means setting routines where all the right partners come together regularly."
Frank DuranPartnership Playbook: Lessons Learned in Effective Partnership
January 8, 2024
"Give teams permission to think about the whole service and solve the problem holistically."
Kristin TaylorBuilding Bridges Across Organizational Silos
November 18, 2022
"Life has a way of working itself out."
Maria GiudiceEmpowering change: Reigniting purpose, passion and impact in research
March 13, 2025
"The primary goal of a hackathon is the community building aspect and getting people to talk to each other and learn from each other."
Shelby SwitzerMaking Space for Community Knowledge-sharing in a Distributed World
December 10, 2021
Latest Books All books
Dig deeper with the Rosenbot
What impact does AI have on the speed and demand for user research according to Basil Faqiri?
How have historical paradigm shifts affected the availability of midwifery care in the US?
How do research operations teams manage the tension between consolidating tools and needing specialized capabilities?