Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide a non-verbal window into users' mental models, often revealing thoughts not expressed in speech.

  • Representational gestures, especially those made in personal gesture space, indicate cognitive processes and implicit imagery.

  • Users commonly hold multiple, context-dependent mental models rather than a single static one.

  • In interviews, interviewer gestures increase participant gesturing and improve conversational rapport.

  • Video interviews pose challenges for capturing gestures fully; positioning and prompting can mitigate this.

  • Speech-gesture mismatches often signal ongoing mental model construction or word searching by users.

  • Gestures can reveal emotional attachment or disengagement with technology, influencing adoption and retention.

  • Mental models can be anchored by recent technology prototypes, such as chat GPT for AI understanding.

  • Structured interview protocols that elicit gesturing and separate talking from tool use optimize gesture data collection.

  • Open source motion tracking and gesture analysis tools can aid qualitative research by quantifying gesture patterns.

Notable Quotes

"Gestures are a window to the mind."

"Gesture and speech form an integrated system; they reinforce one another."

"We’re really leaving half of our data on the table by not attending to gesture when eliciting mental models."

"Gesture is not computer and smartphone gestures, but spontaneous movements people make with hands and arms."

"Four me gestures happen in that personal gesture space and serve as thinking tools for the speaker."

"When gestures and speech mismatch, it often means the speaker is still refining their mental model."

"Without looking at the gesture, we would have come to a less complete mental model."

"Users have more than one mental model; they can be constructed on the fly depending on context."

"The degree to which the user feels in control with an intelligent agent brings up conversational mental models."

"The more you gesture, the more your interviewee will gesture."

Ask the Rosenbot
Mike Oren
Why Pharmaceutical's Research Model Should Replace Design Thinking
2023 • Advancing Research 2023
Gold
How to Identify and Increase your "Experience Quotient"
2018 • Enterprise Experience 2018
Gold
Robin Beers
Panel: Excellence in Communicating Insights
2024 • Advancing Research 2024
Gold
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Chris Chapo
Data Science and Design: A Tale of Two Tribes
2015 • Enterprise UX 2015
Gold
George Abraham
Design Systems To-Go: Indigo.Design Overview and Exploring the Developer Workflow (Part 3)
2021 • DesignOps Summit 2021
Gold
Patrick Boehler
Fishing for Real Needs: Reimagining Journalism Needs with AI
2025 • Designing with AI 2025
Gold
Gabrielle Verderber
Documentation Your Team Will Actually Use
2023 • DesignOps Summit 2023
Gold
Kayla Farrell
What It's Like To Be a User Researcher at Compass
2021 • Advancing Research 2021
Gold
Jen Cardello
Standardizing Product Merits for Leaders, Designers, and Everyone
2018 • Enterprise Experience 2018
Gold
James Rampton
The Basics of Automotive UX & Why Phones Are a Part of That Future
2024 • Rosenfeld Community
Michaela Mora
Advanced Concept Testing Approaches To Guide Product Development and Business Decisions
2022 • Advancing Research 2022
Gold
Saara Kamppari-Miller
Inclusive Design is DesignOps
2021 • DesignOps Summit 2021
Gold
Aras Bilgen
Who does the math: A designer’s journey in building an AI-based tutoring app
2025 • Designing with AI 2025
Gold
Joshua Graves
We Need To Talk: Navigating Conversations with Your Boss (Part 1 of 3)
2025 • Rosenfeld Community
John Devanney
The Design Management Office
2017 • DesignOps Summit 2017
Gold

More Videos

Greg Petroff

"One super small step of value is better than proposing the big picture if the client isn’t ready."

Greg Petroff

Exit Interview #1: Greg Petroff: From Silicon Valley Executive to Sonoma County Possibilitarian

September 24, 2025

Kelly Dern

"Previously, creating user journeys for millions took hours, now generative AI can do it in minutes."

Kelly Dern

AI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process

October 3, 2023

Lada Gorlenko

"Mastercard designers grew meaningfully for both people and business despite uncertainty."

Lada Gorlenko

Theme 2 Intro

June 9, 2022

Prayag Narula

"The best products come from marrying analytics with human stories."

Prayag Narula Hannah Hudson

Empowering Designers to do Good Research

March 11, 2022

Kritika Yadav

"LLMs don't just match keywords or labels. They infer meaning, extract subtle nuances, and understand intent behind words."

Kritika Yadav

Optimizing AI Conversations: A Case Study on Personalized Shopping Assistance Frameworks

June 10, 2025

Mark Templeton

"The hardest thing for any tech company to do is move from technology-out thinking to customer and business outcome thinking."

Mark Templeton

Creating a Legacy: the ultimate experience

June 9, 2017

Amy Gawronski Zuccaro

"Nobody has all the answers and we're human. That's okay."

Amy Gawronski Zuccaro

Advice for DesignOps Employee #1

September 29, 2021

Bria Alexander

"Please put your questions inside the thread specific to the talk so we can keep everything organized."

Bria Alexander

Opening Remarks

November 17, 2022

Adrian Howard

"90% of everything is crud."

Adrian Howard

Sturgeon’s Biases

September 25, 2024