Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Cornelius Rachieru
Handling Complexity: Framing a Scale of Design
2021 • Design at Scale 2021
Gold
Marissa Cui
Climate Design Product Showcase
2024 • Climate UX Interest Group
Ariba Jahan
Team Resiliency Through a Pandemic
2024 • DesignOps Summit 2020
Gold
Scher Foord
Turn the Ship Around: How to Apply Design Thinking Across Your Organization
2021 • Design at Scale 2021
Gold
Lavy Kumar
Future of Work
2021 • Design at Scale 2021
Gold
Jemma Ahmed
Theme 2 Intro
2022 • Advancing Research 2022
Gold
Margot Bloomstein
Fostering Trust in Your Brand and Beyond
2020 • Enterprise Community
Samuel Proulx
Invisible barriers: Why accessible service design can’t be an afterthought
2024 • Advancing Service Design 2024
Gold
Peter Van Dijck
Hands-on AI #1: Let’s write your first AI eval
2025 • Rosenfeld Community
Aditi Ruiz
Pulse Check: Empathy Mapping Your Product Manager, Pt. 2
2022 • Design in Product 2022
Gold
Catt Small
Moving from Execution to Strategy as a Designer
2022 • Design in Product 2022
Gold
Sydney Lawson
Anatomy of a Strong User Panel
2026 • Advancing Research 2026
Conference
Florence Okoye
AfroFuturism and UX Research
2023 • Advancing Research 2023
Gold
Dan Mall
“Ask Me Anything” with Dan Mall, Author of Upcoming Rosenfeld Title, Design that Scales
2023 • DesignOps Summit 2023
Gold
Jorge Arango
Design as an Antidote to VUCA
2019 • Enterprise Community
Trisha Causley
[Demo] Complexity in disguise: Crafting experiences for generative AI features
2024 • Designing with AI 2024
Gold

More Videos

Victor Udoewa

"I hope that today you are transformed."

Victor Udoewa

Theme One Intro

March 27, 2023

Ben Reason

"Designers act as mediators and neutral parties to get stuck projects unstuck through design diplomacy."

Ben Reason Aline Horta Majid Iqbal Fabiano Leoni

Making the system visible: The fastest path to better decisions

November 20, 2025

Deanna Washington

"Evangelism means crossing departments to show how design thinking can improve many parts of the organization."

Deanna Washington Tim Allen Jeff Courcelle John Maeda Matt Raw Erica Tjader

Scaling Success: Paving the Path from DesignOps to VP

October 4, 2023

Taylor Klassman

"Researchers are moving away from viewing AI as automation and toward a sophisticated collaborator or soundboard."

Taylor Klassman

The Evolution of UX Research Platforms

March 10, 2026

Sean Dolan

"Too often journeys are these beautiful artifacts that no one actually understands or uses."

Sean Dolan

A Practical Look at Creating More Usable Enterprise Customer Journeys

October 31, 2019

George Hinchliffe

"Our design system and components are public on developer.servicenow.com to encourage customer reuse and consistency."

George Hinchliffe Joy Liu

Delivering Amazing Experiences

June 10, 2021

Dr. Jamika D. Burge

"Organizational change management requires as much care and effort as executing good research."

Dr. Jamika D. Burge Robert Fabricant Peter Merholz Noam Segal Teresa Torres

A Genuine Conversation about the Future of UX Research

March 20, 2024

Aletheia Delivre

"Our edge isn’t in managing what exists; it’s in architecting new meaning at scale where process becomes infrastructure."

Aletheia Delivre

New Shapes and Emerging Identities for Design Ops

September 11, 2025

Sarah Auslander

"The mayor and CEO were concerned that if the city stepped in on early childhood education, they'd own it forever."

Sarah Auslander

Incremental Steps to Drive Radical Innovation in Policy Design

November 18, 2022