Rosenverse
Don't botch the bot: Designing interactions for AI

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Don't botch the bot: Designing interactions for AI

Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
Don't botch the bot: Designing interactions for AI
Speakers: Savannah Carlin
Link:

Summary

It seems like every company is adding a conversational AI chatbot to their website lately, but how do you actually go about making these experiences valuable and intuitive? Savannah Carlin will present a case study on a conversational AI chatbot—Marqeta Docs AI—that she designed for a developer documentation site in the fintech industry. She will share her insights, mistakes, and perspectives on how to use AI in a meaningful, seamless way, especially for companies like Marqeta that operate in highly regulated industries with strict compliance standards. The talk will use specific examples and visuals to show what makes conversational AI interactions uniquely challenging and the design patterns that can address those challenges. These include managing user expectations, handling errors or misunderstandings within the conversation, and ensuring that users can quickly judge the quality of a bot’s response. You’ll gain a deeper understanding of the intricacies involved in designing interactions for AI, along with practical advice you can apply in your own design processes. Take-aways What to consider before you add AI to your product to ensure it will be valuable, usable, and safe for its intended workflows The interactions that are unique to conversational AI experiences and the design patterns that work for them Common challenges in designing conversational AI experiences and how to overcome them

Key Insights

  • Defining a clear and specific primary use case is crucial before starting any generative AI chatbot project.

  • High-quality, thoroughly reviewed training data is foundational to delivering accurate and useful AI outputs.

  • Initial state messaging must clearly frame what the chatbot can and cannot help with to reduce irrelevant or off-topic queries.

  • Loading indicators for AI text responses should be subtle, with progress reflected by text appearing rather than distracting animations.

  • Supporting efficient scrolling and prompt review is vital since users frequently check and refine their inputs against often long answers.

  • Error states in AI chatbots shift from traditional fixed errors to helping users write better prompts to get more relevant results.

  • Transparency about accuracy, AI limitations, and source citations builds user trust, especially in regulated domains like FinTech.

  • Providing users with prompt engineering guidance via documentation significantly improves the quality of chatbot interactions.

  • Accessibility considerations, like keyboard navigation and screen reader compatibility, must be integrated from the start, especially given the large text outputs.

  • Chatbots can reduce customer support friction and encourage users to ask questions they might not have otherwise, enhancing user engagement with the product.

Notable Quotes

"If you have any doubts about the quality of the training data, do not proceed."

"You want to assist people in framing the interaction and setting their expectations correctly so they know how to be successful."

"The biggest difference with error states in AI bots is helping people write prompts effectively, not just recovering from simple failures."

"Loading text itself is a loading indicator; the letters appearing show progress better than jumpy animations."

"People often forget what they wrote and then want to check their prompt again before refining it."

"Every output should have at least three source links, almost like citations in a research paper."

"We had to be very careful about accuracy because we're in FinTech and compliance is critical."

"People started asking questions to the bot that they wouldn’t have taken the time to email about."

"It’s really important to be clear and transparent about what the tool is good at and what it’s not good at."

"Accessibility testing included making sure everyone could navigate it using a keyboard alone."

Ask the Rosenbot
Rebecca Buck
Mission: Keep Talent in Research Roles!
2021 • Advancing Research 2021
Gold
Silke Bochat
5 Antifragile Strategies for a DesignOps 2.0
2024 • DesignOps Summit 2024
Gold
Jemma Ahmed
Theme 1 Intro
2026 • Advancing Research 2026
Gold
Leah Buley
Ask Me Anything with Leah Buley and Joe Natoli, co-authors of The User Experience Team of One (2nd edition)
2024 • Rosenfeld Community
Sydney Lawson
Anatomy of a Strong User Panel
2026 • Advancing Research 2026
Gold
Billy Carlson
Principles of Team Wireframing
2023 • DesignOps Summit 2023
Gold
Sarah Williams
A Framework for CX Transformation
2021 • Design at Scale 2021
Gold
Rebecca Topps
Planning and conducting remote usability studies for accessibility
2020 • Advancing Research Community
Alexia Cohen
Increasing Health Equity and Improving the Service Experience for Under-Served Latine Communities in Arizona
2024 • Advancing Service Design 2024
Gold
Bria Alexander
Opening Remarks
2024 • Advancing Research 2021
Gold
Raven Veal
Dark Metrics: Illuminating the Negative Impact of Digital Health Design
2021 • Advancing Research 2021
Gold
Ivana Ng
Level Up Your Program with ProductOps
2024 • Enterprise Experience 2020
Gold
Chris Geison
Theme 1 Intro
2022 • Advancing Research 2022
Gold
Jorge Arango
Design as an Antidote to VUCA
2019 • Enterprise Community
Nova Wehman-Brown
We've Never Done This Before
2019 • Enterprise Experience 2019
Gold
Dave Malouf
The Future of DesignOps
2024 • DesignOps Summit 2020
Gold

More Videos

Victor M. Gonzalez

"Doing work in one-week cycles fits the rhythms of companies working with agile and Scrum frameworks."

Victor M. Gonzalez

Practicing Learners and Learning Practitioners

March 10, 2021

Alla Weinberg

"Toxic stress is experiencing too much too fast for too long without healing."

Alla Weinberg

Healing Toxic Stress

September 23, 2024

Libby Maurer

"Maybe I too had joined another tech company that really wasn’t walking the walk when it came to diversity."

Libby Maurer

Treating Diversity & Inclusion in Hiring as a Design Problem

December 5, 2019

Briana Thomas

"When design ops comes in hot, it means honest, orchestrated, and timely."

Briana Thomas Christina Rodriguez

When Design Ops Comes in H.O.T. : A Tale of a Transformed Design Org

September 30, 2021

Mitchell Bernstein

"Every decision in a design system is like a puzzle piece; together they form a bigger picture that evolves over time."

Mitchell Bernstein

Organizing Chaos: How IBM is Defining Design Systems with Sketch for an Ever-Changing AI Landscape

September 29, 2021

Jon Temple

"The best sources for tool info are other researchers in communities like LinkedIn and Slack, not really any independent compendiums."

Jon Temple Kalee Dankner Bruce Falk Lauren Galanter

Panel: Stacks, Security, and Stakeholders: The Hidden Work of UXR Tool Procurement

March 12, 2026

Sheri Byrne-Haber

"Auto captions on YouTube struggle with accents and technical terms, so manual or human-reviewed captioning is essential for accuracy."

Sheri Byrne-Haber

The Importance of Accessible Design Systems

January 8, 2024

Ana Ferreira

"We organize our work around our life, instead of our life around our work."

Ana Ferreira

Designing Distributed: Leading Doist’s Fully Remote Design Team in Six Countries

January 8, 2024

Daniela Magaña Flores

"Doing at least two benchmarking rounds lets you see if optimizations have measurable impacts over time."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022