Rosenverse
Don't botch the bot: Designing interactions for AI

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Don't botch the bot: Designing interactions for AI

Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
Don't botch the bot: Designing interactions for AI
Speakers: Savannah Carlin
Link:

Summary

It seems like every company is adding a conversational AI chatbot to their website lately, but how do you actually go about making these experiences valuable and intuitive? Savannah Carlin will present a case study on a conversational AI chatbot—Marqeta Docs AI—that she designed for a developer documentation site in the fintech industry. She will share her insights, mistakes, and perspectives on how to use AI in a meaningful, seamless way, especially for companies like Marqeta that operate in highly regulated industries with strict compliance standards. The talk will use specific examples and visuals to show what makes conversational AI interactions uniquely challenging and the design patterns that can address those challenges. These include managing user expectations, handling errors or misunderstandings within the conversation, and ensuring that users can quickly judge the quality of a bot’s response. You’ll gain a deeper understanding of the intricacies involved in designing interactions for AI, along with practical advice you can apply in your own design processes. Take-aways What to consider before you add AI to your product to ensure it will be valuable, usable, and safe for its intended workflows The interactions that are unique to conversational AI experiences and the design patterns that work for them Common challenges in designing conversational AI experiences and how to overcome them

Key Insights

  • Defining a clear and specific primary use case is crucial before starting any generative AI chatbot project.

  • High-quality, thoroughly reviewed training data is foundational to delivering accurate and useful AI outputs.

  • Initial state messaging must clearly frame what the chatbot can and cannot help with to reduce irrelevant or off-topic queries.

  • Loading indicators for AI text responses should be subtle, with progress reflected by text appearing rather than distracting animations.

  • Supporting efficient scrolling and prompt review is vital since users frequently check and refine their inputs against often long answers.

  • Error states in AI chatbots shift from traditional fixed errors to helping users write better prompts to get more relevant results.

  • Transparency about accuracy, AI limitations, and source citations builds user trust, especially in regulated domains like FinTech.

  • Providing users with prompt engineering guidance via documentation significantly improves the quality of chatbot interactions.

  • Accessibility considerations, like keyboard navigation and screen reader compatibility, must be integrated from the start, especially given the large text outputs.

  • Chatbots can reduce customer support friction and encourage users to ask questions they might not have otherwise, enhancing user engagement with the product.

Notable Quotes

"If you have any doubts about the quality of the training data, do not proceed."

"You want to assist people in framing the interaction and setting their expectations correctly so they know how to be successful."

"The biggest difference with error states in AI bots is helping people write prompts effectively, not just recovering from simple failures."

"Loading text itself is a loading indicator; the letters appearing show progress better than jumpy animations."

"People often forget what they wrote and then want to check their prompt again before refining it."

"Every output should have at least three source links, almost like citations in a research paper."

"We had to be very careful about accuracy because we're in FinTech and compliance is critical."

"People started asking questions to the bot that they wouldn’t have taken the time to email about."

"It’s really important to be clear and transparent about what the tool is good at and what it’s not good at."

"Accessibility testing included making sure everyone could navigate it using a keyboard alone."

Ask the Rosenbot
Louis Rosenfeld
About the Rosenverse
2026 • Rosenfeld Community
Sam Proulx
Everything You Ever Wanted to Know About Screen Readers
2021 • Design at Scale 2021
Gold
Karen Pascoe
Developing Experience Teams and Talent in the Enterprise
2016 • Enterprise UX 2016
Gold
Phil Hesketh
Designing Accessible Research Workflows
2021 • DesignOps Summit 2021
Gold
Laine Riley Prokay
How DesignOps can Drive Inclusive Career Ladders for All
2021 • DesignOps Summit 2021
Gold
Bria Alexander
Opening Remarks
2021 • DesignOps Summit 2021
Gold
Luke Wroblewski
Designing AI Applications
2026 • Rosenfeld Community
Soma Ghosh
What emerging methods are advancing UX research [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Robert Fabricant
A Genuine Conversation about the Future of UX Research
2024 • Advancing Research Community
Christian Crumlish
Morning Insights Panel
2022 • Design in Product 2022
Gold
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • DesignOps Summit 2022
Gold
Erika Flowers
Introduction to MURAL for UX
2021 • Design at Scale 2021
Gold
Elena Naids
The Power of Difficult Conversations: A Case Study on How We Introduced Design Ops in the Federal Government Space
2023 • DesignOps Summit 2023
Gold
Alison Rand
Scaling Impact with Service Design
2021 • DesignOps Community
Megan Campos
What Did I Miss? The Hidden Costs of Deprioritizing Diversity in User Research
2021 • Advancing Research 2021
Gold
Susan Weinschenk
Evaluating the Maturity of UX in Your Organization
2020 • Enterprise Community

More Videos

Erika Kincaid

"You don’t need to be in the same room to achieve something great."

Erika Kincaid Brenna Heaps Jessica Tsukimura

Connecting the Dots: How to Foster Collaboration and Build a Strong Design Review Culture

June 8, 2022

Catherine Courage

"Saying no with purpose and rationale helped us scale from nine people to over 150 without burning out the team."

Catherine Courage

The Enterprise UX Journey: Lessons From the Voyage & The Opportunity Ahead

May 13, 2015

Shahrzad Samadzadeh

"Designers create value by making artifacts and experiences before fully understanding the space."

Shahrzad Samadzadeh

What Is My Value? Two Takes and Some Mistakes

January 8, 2024

Bria Alexander

"Design ops must learn to advocate for themselves because oftentimes when things go well, no one notices the work behind the scenes."

Bria Alexander Meredith Black Tim Gilligan

State of DesignOps Panel

October 1, 2021

Bud Caddell

"There’s still folks who are entering saying I’m a team of one, no one knows what I do."

Bud Caddell Kristin Skinner Alana Washington

DesignOps Community Sensing Session

May 13, 2021

Ian Johnson

"Multilingual embedding models can cluster similar meanings across languages in a shared latent space."

Ian Johnson

Latent Scope: Finding structure in unstructured data

June 11, 2025

Rob Mitzel

"UX must operate beyond the pixel to have real impact in enterprise transformation."

Rob Mitzel Sébastien Malo

The Tale of Two Companies: Building a Successful UX Practice in a Century-Old Enterprise

January 8, 2024

Gabriela Barneva

"Teams testing components in real-world contexts early can catch issues before products are even built."

Gabriela Barneva

Operationalizing Inclusive Design in Design Ops

September 11, 2025

Vincent Brathwaite

"The future of medicine will be written in algorithms if we let it."

Vincent Brathwaite

Opener: Past, Present, and Future—Closing the Racial Divide in Design Teams

October 22, 2020