This video is only accessible to conference ticket holders.
Log in Create account
For 90 days after a conference, only paid ticket holders can watch conference videos. After that, all Gold members have access.
Contact Support if you are having any issues.
Summary
In this session, we’ll unlock the transformative power of experimental research methods, inspired by best practices in the social sciences. We’ll explore what these techniques are and how they can be used to reveal powerful insights, drive innovation. You’ll also learn how to get started, how to apply them in ethical and rigorous ways, and why these methods are essential for deep, psychology-driven insights.
Key Insights
-
•
Experimental research reveals deeper psychological insights beyond basic AB testing.
-
•
It helps establish causal relationships, not just correlations, enhancing understanding of human behavior.
-
•
Effective experiments require clear independent and dependent variables to isolate impacts.
-
•
Randomization is crucial for eliminating bias in experiments.
-
•
Control groups provide necessary comparisons to assess effects accurately.
-
•
Measurement of outcomes must be consistent and reliable for valid conclusions.
-
•
Katie's experiment tested 20 different trust signals to see which impacted user trust the most.
-
•
Research ethics, including participant consent and transparency, are vital in experimental design.
-
•
Quality of results is dependent on adequate sample sizes and robust statistical analyses.
-
•
Operationalizing research outcomes into actionable insights is essential for real-world application.
Notable Quotes
"Experimental research is a method where variables are manipulated in a controlled environment to establish causal relationships."
"AB tests are valuable, but they often stop at the what and don't uncover the why."
"Surveys measure self-reported attitudes but can't confirm what caused those attitudes."
"Experimental research bridges the gap; it explores why changes work by manipulating variables and measuring their effects."
"Running an experiment might seem daunting at first, but it's about identifying your hypothesis and setting your variables."
"Trust is critical for marketplaces like Thumbtack, where customers invite pros into their homes."
"Half of the trust signals tested had a statistically significant positive effect on overall trust in Thumbtack."
"Global attitudinal measures like trust are notoriously difficult to shift, but targeted efforts can create a real difference."
"Correlation does not equal causation - to say X causes Y, you really need to do an experiment."
"Start small with one clear testable hypothesis, focusing on answering the questions that matter most."
More Videos
"Language is what shapes us, and stories can clarify understanding."
Andrea GallagherThe Problem Space
May 16, 2019
"You need support from the top down and grassroots will and interest and community building."
Alison Rand Sarah BrooksScaling Impact with Service Design
March 25, 2021
"Design operations teams help free up designers to focus on impactful work."
Jose Coronado Julie Gitlin Lawrence LipkinPeople First - Design at JP Morgan
June 10, 2021
"Some projects don't translate well from private to public—let go of delight."
Louis Rosenfeld Lashanda Hodge Senongo Akpem Chris HodowanecBecoming a Civic Designer: Making the Move from Private to Public Sector
November 17, 2022
"There was a lot of research around, but nobody was using it."
Edgar Anzaldua MorenoUsing Research to Determine Unique Value Proposition
March 11, 2021
"Enterprises are adapting agile and lean practices to keep up with smaller, faster competitors."
Jackie HoLead Effectively While Preserving Team Autonomy with Growth Boards
January 8, 2024
"We need to harness the power of citizens to make real change."
Ali Jeffery Sheri ChudowHow DesignOps Helped Enable Wall Street to Work Remotely
October 22, 2020
"Impact and influence have lost their meaning; let's talk about making a difference instead."
Julie Norvaisas Pert Eilers Mina JonssonBack to basics, or start from scratch?
March 12, 2025
"From the medical profession, we learn principles like respect for autonomy and the idea of ‘first do no harm.’"
Mariah HayEthics in Tech Education: Designing to Provide Opportunity for All
June 14, 2018
