Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews

Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Russ Unger
Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started
2016 • Enterprise UX 2016
Gold
Dantley Davis
Leadership & Diversity—A Fireside Chat with Dantley Davis
2020 • Enterprise Community
Patrick Boehler
The service shift: transforming media organizations to create real value through design
2025 • Advancing Service Design 2025
Gold
Tatyana Mamut
Opening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart
2019 • Enterprise Experience 2019
Gold
Erin Hoffman-John
This Game is Never Done: Design Leadership Techniques from the Video Game World
2017 • DesignOps Summit 2017
Gold
Onur Kocan
Understanding the Strategy for Civic Design in a Complex City: Istanbul
2022 • Civic Design 2022
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Sara Logel
Your Colleagues are Your Users Too
2023 • Advancing Research 2023
Gold
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold
Emily Lessard
RFPs Without Tears: Writing Inclusive RFPS that Don't Scare Away Talent
2021 • Civic Design 2021
Gold
Alexandra Schmidt
Why Ethics Can't Save Tech
2022 • Civic Design 2022
Gold
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Lona Moore
Scaling Design Beyond Designers
2021 • Design at Scale 2021
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Fredrik Matheson
First-time users, longtime strategies: Why Parkinson’s Law is making you less effective at work – and how to design a fix.
2016 • Enterprise UX 2016
Gold

More Videos

Isaac Heyveld

"The chief of staff is the bridge between our executive leadership team and the design ops practitioners."

Isaac Heyveld

Expand DesignOps Leadership as a Chief of Staff

September 8, 2022

Jim Kalbach

"That’s not who you are. You’re better than that."

Jim Kalbach

Peace is waged with sticky notes: Mapping Real-World Experiences

June 14, 2018

Alfred Kahn

"If you don’t measure design’s impact, it’s like a tree falling in the forest with no one around to hear it."

Alfred Kahn

A Seat at the Table: Making Your Team a Strategic Partner

November 29, 2023

Peter Van Dijck

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

Peter Van Dijck

Building impactful AI products for design and product leaders, Part 1: The new product journey

July 16, 2025

Iulia Cornigeanu

"We need to be mindful and sniff out whether the small data we’re using is still fresh or already expired."

Iulia Cornigeanu

QuantQual Book Club: Small Data

March 8, 2024

Nick Cochran

"Treat every conversation as an opportunity to make connections and learn meaningfully."

Nick Cochran

Growing in Enterprise Design through Making Connections

June 3, 2019

Roberta Dombrowski

"Begin every research project with the end in mind by framing what impact you hope to drive."

Roberta Dombrowski Sam Duong Woloszynski

Making Research a Team Sport

March 11, 2022

April Reagan

"Foresight is a team sport—the more perspectives you have, the better your future scenarios become."

April Reagan

Look, Think, Act: The Futures-Smart Design Organization

October 1, 2021

Bria Alexander

"Sponsors make this conference possible, and we genuinely thank you from the bottom of our hearts."

Bria Alexander

Opening Remarks

October 4, 2023