Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
What co-speech gestures reveal about users’ thinking during interviews
Summary
Key Insights
-
•
Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.
-
•
Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.
-
•
Users commonly hold multiple, context-dependent mental models rather than a single static one.
-
•
In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.
-
•
Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.
-
•
Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.
-
•
Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.
-
•
Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.
-
•
Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.
-
•
Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.
Notable Quotes
"Gestures are a window to the mind."
"Gesture and speech form an integrated system; they reinforce one another."
"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."
"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."
"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."
"When gestures and speech mismatch, it often means the speaker is still refining their mental model."
"Without looking at the gesture, we would have come to a less complete mental model."
"Users have more than one mental model; they can be constructed on the fly depending on context."
"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."
"The more you gesture, the more your interviewee will gesture."
Or choose a question:
More Videos
"OKRs can sometimes suck the oxygen out of the room if they focus only on bottom-line results, missing process and experience."
Bria Alexander Benson Low Natalya Pemberton Stephanie GoldthorpeOKRs—Helpful or Harmful?
January 20, 2022
"Research is evolving. It’s not about ownership, it’s about impact."
Sean McKayCoexisting with non-researchers: Practical strategies for a democratized research future
March 11, 2025
"Payment is an important part of the research relationship for many users."
Emily Williams Nora FioreWhen UX Research and Institutional Racism Collide: A Case Study
March 12, 2021
"What would a personal cheer team that wants you to grow and blossom look and feel like? Is that the future of design ops?"
Sahibzada Mayed Lauren LinCultivating Design Ecologies of Care, Community, and Collaboration
October 4, 2023
"Middle schoolers might ask you tough questions about release forms and how their data will be used."
Mila Kuznetsova Lucy DentonHow Lessons Learned from Our Youngest Users Can Help Us Evolve our Practices
March 9, 2022
"Miles Davis gave the musicians the music as they entered the studio, and most first takes were the final ones."
Jim KalbachJazz Improvisation as a Model for Team Collaboration
November 6, 2017
"The drag-and-drop UI builder frees engineers to focus on backend data integration while designers create and tweak layouts."
George Hinchliffe Joy LiuDelivering Amazing Experiences
June 10, 2021
"Pay transparency within organizations is critical for enabling good work and healthier working conditions."
Sarah Auslander Betsy Ramaccia Gordon RossInsights Panel
November 18, 2022
"DesignOps needs to work more horizontally to bridge silos between product, marketing, engineering, and research."
Dave Malouf Patrizia Bertini Jon FukudaThe Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 2)
April 28, 2022
Latest Books All books
Dig deeper with the Rosenbot
How do current pressures like AI tools and offshoring affect designers’ sense of purpose and warmth in design?
Why might there be growing demand for AI-free or human-authentic creative services?
How can product teams ensure they prioritize meaningful user outcomes over rapid AI-powered development speed?