Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Casual Inference

Friday, October 6, 2023 • QuantQual Interest Group
Share the love for this talk
Casual Inference
Speakers: Joshua Noble
Link:

Summary

You've probably heard the old adage "correlation does not imply causation" but at some point we've got to say that drinking boiling hot tea and burning our tongues are more than just "strongly correlated". Enter Causal Inference, a collection of techniques for reasoning about the relationship between effects that can be measured and causes that can be identified. It helps us bridge between what we hear when we talk directly to users and what we observe about their behavior at scale and over time. This talk will introduce some of the key techniques in causal inference and how they can be used by UX Researchers and Designers to understand potential users, current users, and the products we might build.

Key Insights

  • Causal inference bridges qualitative reasoning about context with quantitative analysis of data.

  • Directed acyclic graphs help visualize causal pathways and common pitfalls like confounders and colliders.

  • Randomized controlled experiments require random assignment, blinding, and pre-specified hypotheses to measure causal effects reliably.

  • Average treatment effect summarizes impact on an entire population, whereas intent-to-treat reveals experiment adherence issues.

  • Quasi-experimental methods like regression discontinuity and difference-in-differences allow causal inference when randomization is not possible.

  • Matching is an art, relying on context-aware ranges rather than exact numeric equivalence to equate comparison groups.

  • Synthetic control methods build counterfactual scenarios when no good control group exists.

  • Understanding and designing for interpretability strengthens collaboration between designers and analytics or marketing teams.

  • Qualitative research poses valuable questions; quantitative causal inference sharpens and tests these questions.

  • Behavioral causes revealed by causal inference may differ from user self-reported reasons, offering deeper insights.

Notable Quotes

"Correlation doesn’t imply causation, but sometimes two things happening together do imply a causal mechanism."

"Qualitative reasoning about causal mechanisms usually comes from lived experience, either the researcher or participants."

"Designers have a hard time interpreting econometrics-driven quantitative research without bridging approaches."

"Random assignment to treatment or control is essential—if it’s not random, it’s not a true experiment."

"If you cannot remove a treatment, it may not be a true cause."

"Matching is very much an art, not a science; there are no hard and fast rules for what counts as a good match."

"Difference-in-differences lets you study treatment effects in groups that start in very different places by looking at trends."

"Synthetic controls create counterfactuals using historical data when no suitable control group exists."

"Qualitative research really poses questions; quantitative methods let you investigate those questions rigorously."

"You don’t want to overdesign just to make something more data interpretable because that can lead you down a dark path."

Ask the Rosenbot
Ashley Sewall
Exit Interview #5: Designing My Life After Tech
2026 • Rosenfeld Community
Theresa Marwah
How Atlassian is Operationalizing Respect in Research
2020 • Advancing Research Community
Abby Covert
Panel: Collaboration Tools
2017 • DesignOps Summit 2017
Gold
Kate Towsey
Participant Recruitment and Management Tools
2026 • Advancing Research 2026
Gold
Maverick Chan
From Doodle to Demo: AI as Our Storytelling Partner
2025 • Rosenfeld Community
Jen Cardello
Standardizing Product Merits for Leaders, Designers, and Everyone
2018 • Enterprise Experience 2018
Gold
Sam Proulx
To Boldly Go: The New Frontiers of Accessibility
2022 • Advancing Research 2022
Gold
Kate Koch
Flex Your Super Powers: When a Design Ops Team Scales to Power CX
2021 • DesignOps Summit 2021
Gold
Angy Peterson
More Than Technology: Personalized Public Sector Experiences
2021 • Civic Design 2021
Gold
Chris Engledowl
A Mixed Method Approach to Validity to Help Build Trust
2023 • QuantQual Interest Group
Shelby Switzer
Making Space for Community Knowledge-sharing in a Distributed World
2021 • Civic Design 2021
Gold
Prabhas Pokharel
Order and Chaos: New Ways of Collaborating on Synthesis and Storytelling
2022 • Advancing Research 2022
Gold
Erin Weigel
Real-world lessons to improve your conversion rates
2024 • Rosenfeld Community
Elana Chapman
Getting started with accessibility research
2025 • Rosenfeld Community
Rachael Dietkus, LCSW
The power to heal and harm
2025 • Advancing Research 2025
Gold
Peter Van Dijck
Hands on AI #3: Claude Code for UX people
2025 • Rosenfeld Community

More Videos

Maria Giudice

"Design is not a noun. Design is an active verb."

Maria Giudice

Remaking the Making Company: Moving from Product to Experience

June 9, 2016

Christian Crumlish

"Critique sessions work best when the topic is shaped so answers are relevant and help the process, especially by including business goals."

Christian Crumlish

AMA with Christian Crumlish, author of Product Management for UX People

March 24, 2022

Rima Campbell

"Why do we struggle to have impact even when we know we should?"

Rima Campbell Amrit S Bhachu

Increase Productivity and Drive Business Impact

September 24, 2024

Sam Proulx

"Apple unlocked so much innovation by including accessibility in the phone with VoiceOver built-in."

Sam Proulx

To Boldly Go: The New Frontiers of Accessibility

June 10, 2022

Victor Udoewa

"Land acknowledgments don’t actually change the allocation of power or resources; they work at the mythological level."

Victor Udoewa

Radical Participatory Design: Decolonizing Participatory Design Processes

December 10, 2021

Jon Fukuda

"Empathy means constantly going back to the people using the tools and hearing their feedback."

Jon Fukuda Amy Evans Ignacio Martinez Joe Meersman

The Big Question about Innovation: A Panel Discussion

September 25, 2024

Rachael Dietkus, LCSW

"Trauma is a rupture in meaning making; the way you see yourself, the world and others is shocked and overturned by an event."

Rachael Dietkus, LCSW

Trauma-Responsive Design: Reimagining the Future of Design Now

December 10, 2021

Kyria Stephens

"Buffalo is the city of good neighbors—people immediately asked how they could help after the tragedy."

Kyria Stephens Marlon Kerner

Power to Heal: Civic Design in the Aftermath of Tragedy

November 17, 2022

Niko Laitinen

"The red buttons are forbidden to reduce excessive engagement — that runs counter to typical design goals."

Niko Laitinen

Adaptable Org Design for Resilient Times

June 10, 2021