Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews

Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Laura Gatewood
Beyond Buzzwords: Adding Heart to Effective Slack Communication
2024 • DesignOps Summit 2024
Gold
Christian Rohrer
Research Operations at Scale
2017 • DesignOps Summit 2017
Gold
Bria Alexander
Opening Remarks
2023 • Advancing Research 2023
Gold
Tara Tressel
Investigating qualitative depth of AI-moderated interviews
2026 • Advancing Research 2026
Conference
Mansi Gupta
Drawing from Feminist Practice to Make Inclusive Design Operational
2022 • DesignOps Summit 2022
Gold
Catt Small
What's Next for ICs: Exploring Staff and Principal Designer Roles
2024 • Rosenfeld Community
Sam Proulx
SUS: A System Unusable for Twenty Percent of the Population
2021 • Design at Scale 2021
Gold
Jack Behar
How to Build Prototypes that Behave like an End-Product
2022 • Design in Product 2022
Gold
Tim Parmee
Changing Our Design Pressure Points
2023 • DesignOps Summit 2023
Gold
Peter Van Dijck
Hands on AI #3: Claude Code for UX people
2025 • Rosenfeld Community
Mujtaba Hameed
The new horizon of ethnography: using AI to unlock the full potential of in-person research
2026 • Advancing Research 2026
Conference
Kara Kane
Communities of Practice for Civic Design
2022 • Civic Design Community
Uday Gajendar
10 Years of Enterprise UX: Reflecting on the community and the practice
2025 • Enterprise Community
Dave Malouf
Panel: Design Systems and Documentation
2017 • DesignOps Summit 2017
Gold
Jen Cardello
Curating insight: Strategies for integrating knowledge across research functions
2025 • Advancing Research 2025
Gold
Louis Rosenfeld
Founder’s Welcome
2022 • Design in Product 2022
Gold

More Videos

"There’s this kind of Collective unconscious within teams that formed over the pandemic which is deeper than any other team experience I’ve had."

DesignOps and The Great Talent War of 2021

August 19, 2021

George Zhang

"To have a strategy you need to think about how to prioritize, because you can’t do all the research requests that come in."

George Zhang Molly Stevens

UX Research Excellence Framework

March 11, 2021

Fatimah Richmond

"It’s impossible for any single researcher or manager to identify all cross-functional impact opportunities alone."

Fatimah Richmond

The Future of ReOps as a Strategic Function: A Roadmap for Getting There

March 25, 2024

Alana Washington

"Demand is ultimately outstripping supply, and carrier rates are rising, pushing up prices for shippers."

Alana Washington

(Remote) Service Design: A Transformation Case Study

June 8, 2022

Wendy Johansson

"I would let my teams have those hard experiences instead of holding the umbrella because resilience comes from facing challenges directly."

Wendy Johansson

Be a Product Boss!

December 6, 2022

Corey Nelson

"This is a moment to rethink identity beyond UX and get creative with income streams and career paths."

Corey Nelson Amy Santee

Layoffs

November 15, 2022

Onur Kocan

"Testing and implementation processes require sustainable, acceptable approaches using prototype iterations and user feedback."

Onur Kocan Ayhan Ensici

Understanding the Strategy for Civic Design in a Complex City: Istanbul

November 16, 2022

Jorge Arango

"Graph RAG uses knowledge graphs instead of just plain text snippets, which really improves precision."

Jorge Arango

Scale Smart: AI-Powered Content Organization Strategies

September 24, 2024

Melinda Belcher

"Enterprise work is never done — you ship something, then come back to fix and improve it."

Melinda Belcher

Insider preview of Enterprise Experience 2020

May 28, 2020