Disrupting generative AI products with just-in-time consumer insights
Summary
Key Insights
-
•
Building AI products requires giving up control over user experience as outputs vary uniquely per user.
-
•
Traditional usability testing is insufficient for AI products; longitudinal testing with repeated sessions is critical.
-
•
Human-in-the-loop 'Wizard of Oz' testing effectively simulates AI to rapidly test assumptions before building.
-
•
Model drift in LLMs can cause unpredictable or inappropriate responses, requiring mechanisms like turn capping.
-
•
Users form evolving relationships with AI assistants, which need careful design and testing over time.
-
•
Many product orthodoxies based on fixed experiences expire with generative AI and must be reinvented.
-
•
Failure early and often is essential in AI product development due to its inherent unpredictability.
-
•
Users may lack mental models for AI assistants, making onboarding and user education a bigger challenge.
-
•
Asynchronous, multi-threaded conversations are natural for humans but difficult for bots to handle accurately.
-
•
AI can automate synthesis tasks, enabling humans to focus on creativity, accountability, and authenticity.
Notable Quotes
"Building on AI and generative AI means giving up control."
"If the folks that design this tech don’t know how it works, it’s okay if you don’t know either."
"Alignment inevitably means changing minds."
"Everyone’s gonna be wrong. If you’re right all the time, you’re probably not taking big enough swings."
"With LLMs, the product itself is no longer a dependent variable; experiences diverge per user."
"We are studying experiences in ecosystems, not just products anymore."
"Failing early is critical because you can’t control the model."
"The promise of an LLM assistant requires users to build new mental models while using it."
"Turn capping helps reset the model to prevent drift but introduces new problems around memory."
"Creativity, accountability, and authenticity are going to be the new markers of humanity."
Or choose a question:
More Videos
"Service design isn’t about the tools but the way of thinking to approach any problem with a humanist approach."
Laura Smith Tom GaylerEmbedding Service Design and Agile Practice within UK Planning Teams to Create Services that Last
December 3, 2024
"Tones need to be distinct enough so that merchants can spot the tone that best fits their brand."
Trisha Causley[Demo] Complexity in disguise: Crafting experiences for generative AI features
June 5, 2024
"My goal for today is to showcase how generative AI can go beyond just speeding up our processes to actually catapult us in our career."
Fisayo Osilaja[Demo] The AI edge: From researcher to strategist
June 4, 2024
"I am usually the only designer on the team, and it gets difficult to get the information needed for design decisions."
Ruzanna RozmanGetting in Flow with Your Team
January 8, 2024
"AI-powered enterprise tools sit on top of the classic IA infrastructure; without it, AI can’t scale effectively."
Karen McGrane Jeff EatonAI for Information Architects: Are the robots coming for our jobs?
November 21, 2024
"Balancing how often you signal someone and how many questions you ask is key to reduce participant fatigue."
Amelia ColeData-Prompted Interviews
December 17, 2021
"Aim for a healthy balance of safeguards and inspiration to protect customers but keep non-researchers motivated."
Trisha TerharEmpathizing with the Empowered: Non-Researcher Responses to Democratization
March 10, 2022
"In Indigenous outcomes, payments are released only when medical and community metrics are met and validated."
Jen van der MeerService design performs value
November 19, 2025
"I relied very largely on just qualitative information and looking for patterns from talking to over 100 people in a 9,000-employee company."
Steve Baty Richard Dalton Maria Giudice Harry MaxDiscussion
June 9, 2016
