[Demo] Stress-testing GenAI in user research synthesis

Key Insights

• •

  Generative AI speeds textual content generation and navigation but needs human guidance for meaningful interpretation.

• •

  AI struggles to track and assign specific interviewees to user archetypes without explicit human instruction.

• •

  A semi-automated workflow combining AI brainstorming and human refinement delivers faster synthesis under tight deadlines.

• •

  ChatGPT excels at brainstorming clustering parameters and generating detailed archetype descriptions when properly guided.

• •

  AI can retrieve relevant interview quotes but often requires multiple prompt iterations to produce longer, contextually fitting excerpts.

• •

  Visual generation of archetypes using DALL·E currently yields poor results and often contradicts instructions.

• •

  Token limits can constrain transcript processing, but newer models like GPT-4 improve handling of larger inputs.

• •

  AI behaves like an 'alien intern'—powerful but unaware of design research processes and simple context details unless taught.

• •

  Using AI as a creative partner helps overcome cognitive overload, especially in rapid, solo research scenarios.

• •

  Less experienced researchers require caution when using AI tools, as expert evaluation is crucial for output quality.

Notable Quotes