Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
What co-speech gestures reveal about users’ thinking during interviews
Summary
Key Insights
-
•
Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.
-
•
Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.
-
•
Users commonly hold multiple, context-dependent mental models rather than a single static one.
-
•
In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.
-
•
Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.
-
•
Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.
-
•
Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.
-
•
Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.
-
•
Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.
-
•
Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.
Notable Quotes
"Gestures are a window to the mind."
"Gesture and speech form an integrated system; they reinforce one another."
"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."
"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."
"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."
"When gestures and speech mismatch, it often means the speaker is still refining their mental model."
"Without looking at the gesture, we would have come to a less complete mental model."
"Users have more than one mental model; they can be constructed on the fly depending on context."
"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."
"The more you gesture, the more your interviewee will gesture."
Or choose a question:
More Videos
"Design should offer tangible co-benefits like saving money or time to motivate sustainable choices."
Mike Brzozowski Laura Palotie Steve Isley Nancy TsangUX in everyday products: Empowering climate conscious choices
July 17, 2024
"Qualitative synthesis is like watching a movie in a cinema; AI synthesis feels like watching a recap on YouTube—fast but missing the immersive experience."
Weidan LiQualitative synthesis with ChatGPT: Better or worse than human intelligence?
June 4, 2024
"There is always more work to be done than we will ever be able to do, so enabling others to participate is beneficial."
Robert Fabricant Peter Merholz Noam Segal Teresa TorresA Genuine Conversation about the Future of UX Research
March 20, 2024
"Mobile users gave Amazon a higher Net Promoter Score than desktop users, indicating a better mobile experience."
Daniela Magaña Flores Ariane Rahn Jeff Ephraim BanderAhead of Competition: Learn What UX Benchmarking Can Do for Your Business Today
March 10, 2022
"Browser autofill isn’t just convenience; it removes barriers to purchase for people with physical disabilities."
Sam ProulxOnline Shopping: Designing an Accessible Experience
March 28, 2023
"Being humble enough to ask a competitor for feedback changed everything and led directly to job offers."
Corey LongHiring in DesignOps: A Critical Study on How to Hire and Get Hired
September 23, 2024
"The key to good diagrams is accessibly rendered, content driven, and visually supported."
Abby CovertStuck? Diagrams Help
October 27, 2022
"Management consultants sell solutions to problems, not just research data, which is why they command high fees."
Nicole Aleong Michaela Mora Prayag Narula Brianna SylverWhat UX research can learn from other research practices [Advancing Research Community Workshop Series]
September 14, 2023
"Designers who say they are system thinkers tend to build confidence and comfort with ambiguity faster than others."
Catt SmallMoving from Execution to Strategy as a Designer
December 6, 2022
Latest Books All books
Dig deeper with the Rosenbot
What are effective use cases for AI in prototyping UI interactions according to senior designers?
How can organizations prevent design system knowledge from being an undocumented API surface reliant on specific individuals?
How should design systems evolve to support both human and AI consumers effectively?