Summary
In this session, we’ll unlock the transformative power of experimental research methods, inspired by best practices in the social sciences. We’ll explore what these techniques are and how they can be used to reveal powerful insights, drive innovation. You’ll also learn how to get started, how to apply them in ethical and rigorous ways, and why these methods are essential for deep, psychology-driven insights.
Key Insights
-
•
Experimental research reveals deeper psychological insights beyond basic AB testing.
-
•
It helps establish causal relationships, not just correlations, enhancing understanding of human behavior.
-
•
Effective experiments require clear independent and dependent variables to isolate impacts.
-
•
Randomization is crucial for eliminating bias in experiments.
-
•
Control groups provide necessary comparisons to assess effects accurately.
-
•
Measurement of outcomes must be consistent and reliable for valid conclusions.
-
•
Katie's experiment tested 20 different trust signals to see which impacted user trust the most.
-
•
Research ethics, including participant consent and transparency, are vital in experimental design.
-
•
Quality of results is dependent on adequate sample sizes and robust statistical analyses.
-
•
Operationalizing research outcomes into actionable insights is essential for real-world application.
Notable Quotes
"Experimental research is a method where variables are manipulated in a controlled environment to establish causal relationships."
"AB tests are valuable, but they often stop at the what and don't uncover the why."
"Surveys measure self-reported attitudes but can't confirm what caused those attitudes."
"Experimental research bridges the gap; it explores why changes work by manipulating variables and measuring their effects."
"Running an experiment might seem daunting at first, but it's about identifying your hypothesis and setting your variables."
"Trust is critical for marketplaces like Thumbtack, where customers invite pros into their homes."
"Half of the trust signals tested had a statistically significant positive effect on overall trust in Thumbtack."
"Global attitudinal measures like trust are notoriously difficult to shift, but targeted efforts can create a real difference."
"Correlation does not equal causation - to say X causes Y, you really need to do an experiment."
"Start small with one clear testable hypothesis, focusing on answering the questions that matter most."
More Videos
"We don't promise that feedback will show up in the product, but we value every piece of input."
Tracy McGoldrickIBM User Experience Program—The What, Why and How
October 15, 2021
"Take this conversation or these sets of conversations that you participate in over the next few days and bring it back wherever you're going."
Bria Alexander Louis RosenfeldOpening Remarks Day 1
March 25, 2024
"A growth board is an advanced tool that requires significant internal alignment."
Kit Unger Jackie Ho Veevi Rosenstein Vasileios XanthopoulosTheme 2: Discussion
January 8, 2024
"I kind of liked it because if I am difficult, then I am in good company."
Jemma AhmedTheme 2 Intro
March 26, 2024
"Our UX work guides the entire team, but it only does so when it’s written and distributed."
Chris HodowanecAgile + User Experience: How to navigate the Agile landscape as an UX Practitioner
November 16, 2022
"Customers are the North Star, and we need to live by that, not just state it."
Tatyana MamutOpening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart
June 3, 2019
"At Compass, you can promote your ideas even if you're not a boss."
Caitlyn Hampton Monica Lee Jina YoonCompass 101: Growing Your Career In A Startup World
June 11, 2021
"The best teams regularly revisit and reflect on their priorities to stay aligned with changing business needs."
John Cutler Harry MaxPrioritization for designers and product managers (1st of 3 seminars)
June 13, 2024
"Flexibility allows you to use whatever tool you want, integrating easily with existing software."
Benjamin Wiedmaier Annie MayfieldRedefining Toolkits: Unbundling to Create a Perfect Match
March 11, 2025
