Accessible only to conference ticket holders.
Log in Create account Buy conference recordings
For 90 days after a conference, only paid ticket holders can watch conference videos. After that, all Gold members have access.
If you have purchased recording access and cannot see the video, please contact support.
Summary
When research teams are small, the hardest tradeoff is often between depth and scale. Live interviews surface rich, contextual insights, but they’re also time-consuming, resource-heavy, thus often deprioritized when bandwidth is low. In this session, I’ll share how I experimented with AI-moderated interviews to bridge that gap, using technology to recover depth and empathy without requiring live facilitation. Faced with the need to understand our customers’ decision-making (those who purchased our platform, and those who didn’t), I initially relied on surveys. However, I found they lacked the nuance that real conversations reveal. By introducing AI moderation, I created a way for participants to engage in adaptive, conversational interviews that went beyond the limits of static forms. I’ll walk through how I set up these sessions, what prompts worked (and didn’t), and how I analyzed the results. I’ll also share how I’ve used other AI tools like ChatGPT and Perplexity to assist with synthesis and bias-checking, creating a workflow that both expands my analytical reach and strengthens the rigor of my findings. As AI tools continue to enter the researcher’s toolkit, this case study illustrates how we can thoughtfully integrate them to expand—not erode—the human depth of qualitative work. It offers a model for how lean teams can maintain research quality while navigating the realities of limited time, budget, and capacity. This talk will explore the emerging space between automation and augmentation, finding opportunity for depth when time and resources are tight.”
Key Insights
-
•
AI moderation enables more qualitative depth than surveys due to participants speaking freely rather than typing.
-
•
AI moderation platforms require detailed and explicit research plans to guide effective questioning.
-
•
AI moderation often struggles with natural conversational flow and can produce awkward pauses or interruptions.
-
•
Follow-up questions from AI moderators can be vague, repetitive, or miss opportunities to dive deeper.
-
•
AI moderation is best suited for validation or deepening understanding of known problem spaces, not foundational discovery.
-
•
Participants may prefer AI moderation over surveys due to ease and flexibility, supporting higher and quicker recruitment.
-
•
AI moderation platforms currently do not replace human analysis or provide strong quantitative outputs.
-
•
High emotion or sensitive topics may work better with AI moderation if participants prefer anonymity, but human interviews excel at rapport.
-
•
Researchers need to invest time troubleshooting and iterating prompts in AI moderated studies.
-
•
AI moderated studies allow teams of one to scale qualitative research when live interviews are infeasible.
Notable Quotes
"Having one-on-calls with people is one of my favorite parts of conducting research, and it felt like this was being taken away from me just for the sake of saving business resources."
"AI moderation, in short, is a large language model that runs a moderated interview session without a human researcher present."
"I was interested in thinking of this method as a survey plus that you can use to get survey-like data but with more qualitative depth."
"When I gave the model my list of interview questions, it actually made for a really bizarre participant experience because the model just went down the line of the different questions."
"People were more likely to explain their thought process and just more context around the particular situation of their org."
"There were moments like long kind of awkward pauses where the model was processing, and sometimes it actually spoke over participants."
"Follow-up questions missed opportunities to probe deeper or were confusing, and participants had to repeat themselves."
"If I really just needed only quantitative data from a survey, then I would not go the AI moderated route."
"AI moderation did not make the research easier. It made it possible under real world constraints."
"From setup to analysis, AI moderation still requires a good understanding of what quality research looks like and how to interpret qualitative data."
Or choose a question:
More Videos
"Creating and holding space is like putting bumpers up in bowling so people know they won’t fail if they engage."
Gina MendoliaTherapists, Coaches, and Grandmas: Techniques for Service Design in Complex Systems
December 3, 2024
"What would happen if you do think about validity? Would it change your process or research plans?"
Chris EngledowlA Mixed Method Approach to Validity to Help Build Trust
April 28, 2023
"Inclusive research should not be a feel-good activity but treated as the norm on every project."
Etienne FangThe Power of Care: From Human-Centered Research to Humanity-Centered Leadership
March 10, 2021
"Accessibility isn’t just for developers or designers—everyone has a role to play."
Samuel ProulxDesigning beyond caricatures: Embracing real, diverse user needs
December 4, 2024
"Not everyone using machine learning is your user; think also about the people labeling the data feeding these models."
Brian T. O’Neill Maria Cipollone Luis Colin Manuel Dahm Mike OrenDoes Designing and Researching Data Products Powered by ML/AI and Analytics Call for New UX Methods?
February 18, 2022
"Don’t kill grandma is our mantra reminding us the stakes of the tiny design details we face every day."
Barb SpantonDoing Work That Matters: A Look Beyond The Idealistic Notion of 'Doing Meaningful Work'
June 10, 2022
"What if flying was as easy as driving a car? That was our main question designing user interfaces."
Teresa SwinglerLook, Up in the Sky! UX/UI for Aerospace
October 27, 2022
"We created a Slack bot that automatically created tickets from UX bug reports to help design managers discover issues faster and track recovery time."
Peter BoersmaHow to Define and Maintain a DesignOps Roadmap
October 3, 2023
"If we want to do the best work, we cannot keep looking in the same places and hope the right people will just show up."
Mackenzie GuinonM.C. Escher’s UX Research Career Ladder
March 9, 2022
Latest Books All books
Dig deeper with the Rosenbot
What role do research projects with customizable data inputs play in Survicate's new product?
How can research and operations teams collaboratively evaluate new UXR tools for usability and operational fit?
How does involving more senior researchers in the field improve research outcomes when AI handles transcript processing?