Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
April Reagan
Look, Think, Act: The Futures-Smart Design Organization
2021 • DesignOps Summit 2021
Gold
Yalenka Mariën
Designing for Digital Inclusion in the Belgian Government
2021 • Civic Design 2021
Gold
Caroline Jarrett
Garbage in, garbage out? Measuring error rates to get ready for AI
2026 • Rosenfeld Community
Shazia Ali
Communication: Innovative techniques for making your voice heard [Advancing Research Community Workshop Series]
2024 • Advancing Research Community
Niko Laitinen
Adaptable Org Design for Resilient Times
2021 • Design at Scale 2021
Gold
Doug Powell
Closing Keynote: Design at Scale
2018 • DesignOps Summit 2018
Gold
Charlotte Lee
Theme 1 Intro
2021 • Civic Design 2021
Gold
Nathan Curtis
Discussion
2016 • Enterprise UX 2016
Gold
Jason Mesut
Unmasking Design Leadership: Navigating leadership without neglecting ourselves
2025 • Rosenfeld Community
Stefanie Owens
Optimizing for Outcomes: Transformation Design in Systems at Scale
2024 • Advancing Service Design 2024
Gold
Ruzanna Rozman
Getting in Flow with Your Team
2024 • DesignOps Summit 2020
Gold
Billy Carlson
Principles of Team Wireframing
2023 • DesignOps Summit 2023
Gold
Louis Rosenfeld
Opening Remarks
2022 • Civic Design 2022
Gold
Maria Giudice
Empowering change: Reigniting purpose, passion and impact in research
2025 • Advancing Research 2025
Gold
Jess Greco
Creating a Basis for Change: Scaling Design Maturity
2022 • Design at Scale 2022
Gold
Dan Willis
Enterprise Storytelling Sessions
2017 • Enterprise Experience 2017
Gold

More Videos

Sam Proulx

"Blockchain apps aren’t located in one country or jurisdiction, making legal enforcement of accessibility laws difficult."

Sam Proulx

To Boldly Go: The New Frontiers of Accessibility

November 18, 2022

Melinda Belcher

"Enterprise work is never done — you ship something, then come back to fix and improve it."

Melinda Belcher

Insider preview of Enterprise Experience 2020

May 28, 2020

Billy Carlson

"If you had to visualize what you thought you agreed to in a meeting, a lot of times there’s misalignment."

Billy Carlson

Principles of Team Wireframing

October 2, 2023

Hana Nagel

"Rituals embody our values by translating those values into repeated actions that sustain behavior change."

Hana Nagel

Turning Research Ripples into Waves

November 8, 2018

Ari Zelmanow

"Researchers are like detectives. We turn data into stories, stories into strategy, strategy into outcome."

Ari Zelmanow

Dark Side of Democratization

October 13, 2023

Scott Plewes

"A model is a simplified representation of a system, whether in math or UX."

Scott Plewes

Why Isn't Your UX Approach Going Viral?: A Mathematical Model

March 28, 2023

Dominique Ward

"Now design ops teams must be fit for purpose. We should meet the needs of the work we have to do."

Dominique Ward

The Most Exciting Time for DesignOps is Now

September 8, 2022

Sarah Coyle

"Data is just single points; information is making that data meaningful through context."

Sarah Coyle

Design and Analytics with Sarah Coyle

July 30, 2020

Jemma Ahmed

"Insight practices are just getting more established and are asking questions of higher strategic value, bringing disciplines closer together."

Jemma Ahmed

Convergent Research Techniques in Customer Journey Mapping

March 31, 2020