Rosenverse
What co-speech gestures reveal about users’ thinking during interviews

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews

Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Deanna Smith
Leading Change with Confidence: Strategies for Optimizing Your Process
2024 • DesignOps Summit 2024
Gold
Charlotte Vorbeck
Pipeline to Civic Design
2021 • Civic Design 2021
Gold
Ovetta Sampson
Turning UX Passion into Real Product Influence
2023 • Enterprise UX 2023
Gold
James Lang
If you can design an app, you can design a community
2025 • Rosenfeld Community
Nick Fine
How UX researchers can partner with (and not be replaced by) AI [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Mujtaba Hameed
The new horizon of ethnography: using AI to unlock the full potential of in-person research
2026 • Advancing Research 2026
Gold
Lais de Almeida
Designing Data Services
2024 • Advancing Service Design 2024
Gold
Vasileios Xanthopoulos
A Top-Down and Bottom-Up Approach to User-Centric Maturity at Scale
2024 • Enterprise Experience 2020
Gold
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Civic Design 2021
Gold
Claire Dhoosche
Coordinating chaos: Preventing workflow fragmentation when everyone accelerates with AI
2026 • Designing with AI 2026
Gold
Dianne Que
Real Talk: Proving Value through a Scrappy Playbook
2019 • DesignOps Summit 2019
Gold
Jon Fukuda
Theme 3 Intro
2024 • DesignOps Summit 2024
Gold
Erin Hauber
Design is Not the Frosting on the Scaled Agile Layer Cake
2019 • DesignOps Summit 2019
Gold
Fatimah Richmond
The Future of ReOps as a Strategic Function: A Roadmap for Getting There
2024 • Advancing Research 2024
Gold
Michaela Mora
Advanced Concept Testing Approaches To Guide Product Development and Business Decisions
2022 • Advancing Research 2022
Gold
Angelos Arnis
Our Fragmented Identity
2026 • Rosenfeld Community

More Videos

Stefanie Owens

"An agency developing shared technical infrastructure must market to itself why change is preferred, not just rely on top-down mandate."

Stefanie Owens

Optimizing for Outcomes: Transformation Design in Systems at Scale

December 4, 2024

Gonzalo Goyanes

"The success of ROI measurement depends mainly on the maturity stage and context of your design team."

Gonzalo Goyanes

Design ROI: Cover a Little, Get a Lot

September 8, 2022

Alla Weinberg

"Psychological safety is the felt permission to be honest, to speak up, and admit mistakes without fear."

Alla Weinberg

People Are Sick of Change: Psychological Safety is the Cure

July 20, 2023

Brigette Metzler

"Research repositories and libraries are social things — many teams look for best practices beyond just building a library."

Brigette Metzler Dana Chrisfield

Research Repositories: A global project by the ResearchOps Community

August 27, 2020

Lada Gorlenko

"Those three retailers pivoted literally overnight under intense change management pressure."

Lada Gorlenko

Theme 2 Intro

June 9, 2022

Sheryl Cababa

"Systems thinking requires constant engagement with stakeholders; working in isolation will not lead to successful outcomes."

Sheryl Cababa

Expanding Your Design Lens with Systems Thinking

February 23, 2023

Benjamin Real

"Understaffing is consistent in design ops no matter where in the world you are."

Benjamin Real

Maturity Models: A Core Tool for Creating a DesignOps Strategy

October 1, 2021

Erin Weigel

"Conversion means change, not just sales or profit."

Erin Weigel

Get Your Whole Team Testing to Design for Impact

July 24, 2024

James Lang

"Scaling communities is about propagation, not just growing as big as possible."

James Lang

If you can design an app, you can design a community

May 22, 2025