Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
What co-speech gestures reveal about users’ thinking during interviews
Summary
Key Insights
-
•
Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.
-
•
Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.
-
•
Users commonly hold multiple, context-dependent mental models rather than a single static one.
-
•
In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.
-
•
Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.
-
•
Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.
-
•
Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.
-
•
Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.
-
•
Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.
-
•
Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.
Notable Quotes
"Gestures are a window to the mind."
"Gesture and speech form an integrated system; they reinforce one another."
"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."
"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."
"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."
"When gestures and speech mismatch, it often means the speaker is still refining their mental model."
"Without looking at the gesture, we would have come to a less complete mental model."
"Users have more than one mental model; they can be constructed on the fly depending on context."
"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."
"The more you gesture, the more your interviewee will gesture."
Or choose a question:
More Videos
"Synthetic data is useful because it’s easier to generate more examples of something you already have than to create entirely new data."
Peter Van DijckBuilding impactful AI products for design and product leaders, Part 2: Evals are your moat
July 23, 2025
"If you need to cross a dangerous river, you don’t paddle straight into it; you use the opposing force of the river to move forward."
John CutlerOxbows, Rivers, and Estuaries: How to navigate the currents of change (without burning out)
December 3, 2024
"Celebrate customer-driven successes so the whole organization learns what’s valued and aspires to do the same."
Ross SmithBreaking Barriers with Empathy
June 9, 2017
"Most business tools focus on quantitative value and are terrible at capturing qualitative value, where design lives."
Nathan ShedroffRedefining Value: Bridging the Innovation Culture Divide
May 14, 2015
"We discovered shared ideals and differences of opinion — debating and working hard together is part of the process."
Husani OakleyTheme Two Intro
June 6, 2023
"I get bored super easily; that’s why I’m super into the AI stuff right now."
Greg PetroffExit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian
September 24, 2025
"The fly on the wall approach is a myth; interaction between filmmaker and people is what makes documentary film interesting."
Bas Raijmakers, PhD (RCA) Charley Scull Prabhas PokharelWhat Design Research can Learn from Documentary Filmmaking
March 11, 2022
"Crisis is an opportunity, and user research must claim a louder, prouder voice on brand leadership."
Neil BarrieWidening the Aperture: The Case for Taking a Broader Lens to the Dialogue between Products and Culture
March 25, 2024
"Mapping is not just for presentation; it’s a practice to evolve your understanding of a complex problem space."
Sheryl Cababa Alexis OhThinking in systems to address climate with Sheryl Cababa
June 12, 2024
