Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Demo] Stress-testing GenAI in user research synthesis
Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
[Demo] Stress-testing GenAI in user research synthesis
Speakers: Yulya Besplemennova
Link:

Summary

In this high-pressure scenario, the challenge was to conduct 17 user interviews in three days and synthesize a comprehensive report in just one additional day. I’ll explore how we used AI to streamline the research process, from transcription to synthesis, and how tools like ChatGPT contributed to efficient data processing and insight generation. We’ll reflect on the potential and pitfalls of using AI in accelerated user research, from practical aspects to more philosophical considerations on potential changes to the research process. Takeaways Practical insights into integrating AI with traditional research methodologies to expedite the research process An overview of the effectiveness of AI transcription and synthesis tools in real-world research scenarios Critical examination of AI's role in data processing and how it compares with human analysis Strategic considerations for service designers when employing AI to support rapid user research Reflection on the ethical implications and potential impact on the quality of insights and researcher well-being when relying on AI to speed up research processes

Key Insights

  • Generative AI speeds textual content generation and navigation but needs human guidance for meaningful interpretation.

  • AI struggles to track and assign specific interviewees to user archetypes without explicit human instruction.

  • A semi-automated workflow combining AI brainstorming and human refinement delivers faster synthesis under tight deadlines.

  • ChatGPT excels at brainstorming clustering parameters and generating detailed archetype descriptions when properly guided.

  • AI can retrieve relevant interview quotes but often requires multiple prompt iterations to produce longer, contextually fitting excerpts.

  • Visual generation of archetypes using DALL·E currently yields poor results and often contradicts instructions.

  • Token limits can constrain transcript processing, but newer models like GPT-4 improve handling of larger inputs.

  • AI behaves like an 'alien intern'—powerful but unaware of design research processes and simple context details unless taught.

  • Using AI as a creative partner helps overcome cognitive overload, especially in rapid, solo research scenarios.

  • Less experienced researchers require caution when using AI tools, as expert evaluation is crucial for output quality.

Notable Quotes

"Gene AI can really help to speed up synthesis moments connected to textual content generation and navigation."

"AI behaves like an alien intern because it’s not capable of making sense of some things easy for us."

"I approached the GPT as if it were my human colleague, asking very open questions."

"It couldn’t find specific interviewees for user archetypes, so I had to adapt and guide the process."

"This is a hero moment because it accelerates and simplifies exploration of clustering options."

"Text generation is the biggest superpower of ChatGPT in research synthesis right now."

"Visuals from DALL·E didn’t work well; I asked multiple times to remove cables but they still appeared."

"AI can help manage cognitive overload and speed up work when you have short timeframes and no team."

"You need a lot of annoying handholding with the AI: reminding it of how many people you have or simple things."

"I wouldn’t leave AI tools to juniors because it’s crucial to have your own experience to evaluate outputs."

Ask the Rosenbot
Edward Cupps
The Principal Path: Journeying from Management to Individual Contributor
2021 • Design at Scale 2021
Gold
Steve Portigal
Discussion
2015 • Enterprise UX 2015
Gold
Francesca Barrientos, PhD
You Need Your Own Definition of Design Maturity
2022 • Design at Scale 2022
Gold
Sean Dolan
A Practical Look at Creating More Usable Enterprise Customer Journeys
2019 • Enterprise Community
Aletheia Delivre
New Shapes and Emerging Identities for Design Ops
2025 • DesignOps Summit 2025
Gold
Sarah Gallimore
Inspire Progress with Artifacts from the Future
2022 • Civic Design 2022
Gold
Anna Avrekh
Expert Panel: Leading in and with Research
2022 • Advancing Research 2022
Gold
Anupama Dhareshwar
From blueprint to bot: Designing resilient AI-powered services
2025 • Advancing Service Design 2025
Gold
Michele Wong
Helping Them Help Us
2024 • Enterprise Experience 2020
Gold
Sam Proulx
Accessibility: An Opportunity to Innovate
2022 • Advancing Research 2022
Gold
Ian Johnson
Latent Scope: Finding structure in unstructured data
2025 • Designing with AI 2025
Gold
Liam Thurston
Why Your Design Team Is Quitting, And How To Fix It
2022 • Design at Scale 2022
Gold
Iram Shah
Closing Keynote: The View from the Top
2019 • Enterprise Experience 2019
Gold
Jeff Gothelf
Who does what by how much?
2025 • Advancing Service Design 2025
Gold
Amy Marquez
INVEST: Discussion
2018 • Enterprise Experience 2018
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold

More Videos

Louis Rosenfeld

"Affordable, powerful AI and data-driven design have become shockingly practical for enterprise settings."

Louis Rosenfeld

Welcome / Housekeeping

June 6, 2023

Paul Pangaro, PhD

"Organizations succeed when they have a language that works today but must evolve it to survive tomorrow."

Paul Pangaro, PhD

Systems Disciplines: Table Stakes for 21st Century Organizations

June 6, 2023

Dharani Perera

"De-risking sessions gave disgruntled stakeholders a forum to voice fears before larger workshops."

Dharani Perera

The mandala of service design: unlocking alignment and action through service design

November 20, 2025

Tiffany Cheng

"Digital products aren’t like buildings; they can keep changing and moving as you need."

Tiffany Cheng

Designing in a Pandemic: Integrating Speed and Rigor

June 9, 2022

Ricardo Martins

"Conjoint analysis was used by Apple and Samsung to determine the financial value of specific product features."

Ricardo Martins

Unlocking the power of advanced quantitative methods

March 12, 2025

William Newton

"Blue sky projects should have no gating metrics and no short-term metrics because their value is long-term."

William Newton Jenny Chang

How to Lead With Data, and Without Data

June 7, 2023

Sahibzada Mayed

"Are you able to say I caused harm, and can you repair the relationships and impacts your work has created?"

Sahibzada Mayed

The Politics of Radical Research: A Manifesto

March 27, 2023

Prayag Narula

"There is overlap between design and research when it comes to different types of research; it’s not a strict divide."

Prayag Narula

How to Empower Your Designers to Do Good Research – And Why You Want To

June 10, 2022

Jack Behar

"Merge translates code from Storybook, npm, or third-party libraries into interactive, visual components in the editor."

Jack Behar

How to Build Prototypes that Behave like an End-Product

December 6, 2022