Rosenverse
What co-speech gestures reveal about users’ thinking during interviews

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews

Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Deanna Smith
Leading Change with Confidence: Strategies for Optimizing Your Process
2024 • DesignOps Summit 2024
Gold
Noah Bond
Redefining truth and inclusivity: Navigating data ownership and ethical research in the age of disinformation
2025 • Advancing Research 2025
Gold
Charlotte Lee
Theme 1 Intro
2021 • Civic Design 2021
Gold
Briana Thomas
When Design Ops Comes in H.O.T. : A Tale of a Transformed Design Org
2021 • DesignOps Summit 2021
Gold
Dan Willis
Enterprise Storytelling Sessions
2019 • Enterprise Experience 2019
Gold
Tamara Kartoziia
Think global, adapt local: how service design accelerated B2B market entry by 6 months
2025 • Advancing Service Design 2025
Gold
Maggie Dieringer
Creating Consistency Through Constant Change
2024 • DesignOps Summit 2020
Gold
Mike Brzozowski
UX in everyday products: Empowering climate conscious choices
2024 • Climate UX Interest Group
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Roy Opata Olende
How Zapier Uses ‘All Hands Research’ to Increase Exposure to Users
2020 • Advancing Research Community
Jaime Creixems
Best Practices when Creating and Maintaining a Design System
2023 • Enterprise UX 2023
Gold
Frances Yllana
The Big Question about Impact: A Panel Discussion
2024 • DesignOps Summit 2024
Gold
Dave Hoffer
UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer
2025 • Rosenfeld Community
Jacqui Frey
Scale is Social Work
2020 • DesignOps Community
Dave Malouf
Closing Keynote: Amplify. Not Optimize.
2019 • DesignOps Summit 2019
Gold
Eric Shumake
An AMA on UX's Role in Healthcare
2026 • Rosenfeld Community

More Videos

Sarah Auslander

"Civic action as proof of concept to get government moving, to get the wheels turning was really interesting."

Sarah Auslander Betsy Ramaccia Gordon Ross

Insights Panel

November 18, 2022

Cheryl Platz

"High status is taking up more space, holding eye contact, and purposeful gestures."

Cheryl Platz

Collaborative Creativity through Improv

November 7, 2018

Alexandra Schmidt

"We should reconsider the savior complex designers sometimes have — we’re not all powerful."

Alexandra Schmidt

Why Ethics Can't Save Tech

November 18, 2022

Maggie Dieringer

"Being a predominantly female team brings stereotypes and challenges, especially in male-dominated product organizations."

Maggie Dieringer

Cutting through the Noise

September 24, 2020

Uday Gajendar

"Boone’s talk reveals methods for senior ICs to make sense of enterprise complexities by staging interventions."

Uday Gajendar

Theme One Intro

June 6, 2023

Deanna Mitchell

"Translating complex qualitative findings into simple, visual frameworks is critical for making your work stick."

Deanna Mitchell

Designing with culture: Unlocking impactful insights for Product and UX

March 12, 2025

Erin Weigel

"We are the shopkeepers of today; it just looks a little bit different."

Erin Weigel

Real-world lessons to improve your conversion rates

June 26, 2024

Uday Gajendar

"Enterprise is sexy – that phrase helped me get at least one job."

Uday Gajendar Lada Gorlenko Dave Malouf Louis Rosenfeld Dan Willis

10 Years of Enterprise UX: Reflecting on the community and the practice

June 18, 2025

Brigette Metzler

"Maintaining a research repository often requires 100% manual participation from researchers, which can be hard to enforce."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community

August 27, 2020