Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
What co-speech gestures reveal about users’ thinking during interviews
Summary
Key Insights
-
•
Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.
-
•
Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.
-
•
Users commonly hold multiple, context-dependent mental models rather than a single static one.
-
•
In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.
-
•
Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.
-
•
Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.
-
•
Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.
-
•
Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.
-
•
Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.
-
•
Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.
Notable Quotes
"Gestures are a window to the mind."
"Gesture and speech form an integrated system; they reinforce one another."
"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."
"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."
"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."
"When gestures and speech mismatch, it often means the speaker is still refining their mental model."
"Without looking at the gesture, we would have come to a less complete mental model."
"Users have more than one mental model; they can be constructed on the fly depending on context."
"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."
"The more you gesture, the more your interviewee will gesture."
Or choose a question:
More Videos
"I still kind of pause in my tracks when I see our corporate vision: a world without fear of cancer."
Barb SpantonDoing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
June 10, 2022
"FEMA is not going to stand up a UX department; it’s our responsibility to witness and find ways to help."
Emily Danielson“I mean, I can lift a shovel”: Design Skills in Disaster Response
June 9, 2022
"Text-based campaigns have been very successful for reaching senior citizens."
Amanda Kaleta-Kott Lea Martin, PhDThe Joys and Dilemmas of Conducting UX Research with Older Adults
March 11, 2022
"We categorized all our work into may-do, must-do, and desire-to-do buckets to better allocate our efforts."
Amy EvansHow to Create Change
September 25, 2024
"Having tight feedback loops and directly controlling the end-to-end experience completely changed how excited I was to do design work."
John DonmoyerShipping your code generation experiments to production
June 11, 2025
"I got my PhD so I could get a seat at the table."
Kristen Honey"Let’s Talk About Data and Crisis”: Public Digital Service Delivery = Open Data + Human Centered Design
November 18, 2021
"Power dynamics exist in every session. People don’t want to be embarrassed or feel put on the spot."
Mila Kuznetsova Lucy DentonHow Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices
March 9, 2022
"Trend forecasting is a culture industry—it produces culture as much as it predicts it."
Devon PowersImagining Better Futures
March 9, 2022
"Giving people control over their work is just as important as validating their experiences to reduce burnout."
Rachael Dietkus, LCSW Uday Gajendar Dr. Dawn Emerick Dawn E. Shedrick, LCSWLeading through the long tail of trauma
July 13, 2022
