Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

What co-speech gestures reveal about users’ thinking during interviews
Friday, June 30, 2023 • Rosenfeld Community
Share the love for this talk
What co-speech gestures reveal about users’ thinking during interviews
Speakers: Dane DeSutter and Stephanie Scopelitis
Link:

Summary

Drs. DeSutter and Scopelitis discussed how User Experience (UX) researchers can triangulate and enrich information from one-on-one interviews by attending to users’ co-speech gestures—the spontaneous movements that humans make with their hands and body when communicating. Gestures are a “window to the mind” and can reveal unspoken information about users’ emotional states as well as the structure and composition of their mental models. They concluded with a practical guide for efficiently implementing gesture research.

Key Insights

  • Gestures provide additional layers of meaning beyond spoken words, revealing users' mental imagery.

  • Gesture research has roots in psycholinguistics and can enhance user experience research by providing deeper insights.

  • Recognizing gestures during interviews can lead to richer data and better understanding of user mental models.

  • The integration of gesture and speech is vital; they form a co-expressive system that communicates meaning together.

  • Users may have multiple mental models that can shift depending on context and technology, as demonstrated in user interviews.

  • The spacing and context of gestures matter; gestures in the interactional space are often more telling than those close to the body.

  • Gesture analysis can identify implicit user relationships to systems, helping refine user experience design.

  • Techniques such as motion tracking can augment analysis, providing quantitative insights into gesture frequency and type.

  • Gestures can indicate a user's affective states and attachment to particular features or tools.

  • By understanding gesture, researchers can craft better follow-up questions that lead to more insightful interviews.

Notable Quotes

"Gestures are a window to the mind."

"Gesture serves as a window into the mind, revealing images that users often think are concealed."

"We're leaving half of our data on the table by not attending to gesture during interviews."

"Gestures hold great value in communication; they facilitate and negotiate meaning in interaction."

"Pay attention to those representational gestures that will bring forward mental imagery."

"Developing a better understanding of our users requires attention to both gesture and speech as integrated data."

"Users have more than one mental model; mental models are not monolithic but can shift based on context."

"Without looking at gesture, we would come to a less complete understanding of the user’s mental model."

"Users may be more wary of engaging with systems they don’t feel in control of."

"Gesture analysis can be applied in generative, usability, and design phases of the process."

More Videos

Saara Kamppari-Miller

"Visibility is key to driving accessibility initiatives."

Saara Kamppari-Miller

DesignOps for Inclusive Design and Accessibility

May 26, 2022

Rusha Sopariwala

"Our work here is not done. This is a journey."

Rusha Sopariwala

Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+

June 9, 2022

Tess Dixon

"The notion that a team’s happiness hinges on constant laughter and joy is misleading."

Tess Dixon

C'mon Get Happy

September 29, 2021

Greg Petroff

"What percentage of what agents do comes from their mind having to work versus the system telling them how much to earn?"

Greg Petroff

Design is the Differentiator: Bringing New Design Innovations to a Very Antiquated and Very Large Industry

June 9, 2021

Kim Fellman Cohen

"Does your question give you enough room to explore new ideas?"

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

Catt Small

"Titles can mean really different things at different places, especially depending on the company's size."

Catt Small Micah Bennett Brian Carr Jessica Harllee

What's Next for ICs: Exploring Staff and Principal Designer Roles

February 22, 2024

Jodi Forlizzi

"User-centered design must adapt when working with AI, often starting with data sets instead of users."

Jodi Forlizzi

Design and AI innovation

June 5, 2024

Samuel Proulx

"The mindset I'm jumping into with these sessions is to assess usability and accessibility."

Samuel Proulx Laur Baek

Inclusive Research: Debunking Myths and Getting Started

March 12, 2025

Séamus Byrne

"We need to be more prepared to dance together."

Séamus Byrne

Aligning Teams with Choreography

January 8, 2024