Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

Don't botch the bot: Designing interactions for AI

Gold
Tuesday, June 4, 2024 • Designing with AI 2024
Share the love for this talk
Don't botch the bot: Designing interactions for AI
Speakers: Savannah Carlin
Link:

Summary

It seems like every company is adding a conversational AI chatbot to their website lately, but how do you actually go about making these experiences valuable and intuitive? Savannah Carlin will present a case study on a conversational AI chatbot—Marqeta Docs AI—that she designed for a developer documentation site in the fintech industry. She will share her insights, mistakes, and perspectives on how to use AI in a meaningful, seamless way, especially for companies like Marqeta that operate in highly regulated industries with strict compliance standards. The talk will use specific examples and visuals to show what makes conversational AI interactions uniquely challenging and the design patterns that can address those challenges. These include managing user expectations, handling errors or misunderstandings within the conversation, and ensuring that users can quickly judge the quality of a bot’s response. You’ll gain a deeper understanding of the intricacies involved in designing interactions for AI, along with practical advice you can apply in your own design processes. Take-aways What to consider before you add AI to your product to ensure it will be valuable, usable, and safe for its intended workflows The interactions that are unique to conversational AI experiences and the design patterns that work for them Common challenges in designing conversational AI experiences and how to overcome them

Key Insights

  • Defining a clear and specific primary use case is crucial before starting any generative AI chatbot project.

  • High-quality, thoroughly reviewed training data is foundational to delivering accurate and useful AI outputs.

  • Initial state messaging must clearly frame what the chatbot can and cannot help with to reduce irrelevant or off-topic queries.

  • Loading indicators for AI text responses should be subtle, with progress reflected by text appearing rather than distracting animations.

  • Supporting efficient scrolling and prompt review is vital since users frequently check and refine their inputs against often long answers.

  • Error states in AI chatbots shift from traditional fixed errors to helping users write better prompts to get more relevant results.

  • Transparency about accuracy, AI limitations, and source citations builds user trust, especially in regulated domains like FinTech.

  • Providing users with prompt engineering guidance via documentation significantly improves the quality of chatbot interactions.

  • Accessibility considerations, like keyboard navigation and screen reader compatibility, must be integrated from the start, especially given the large text outputs.

  • Chatbots can reduce customer support friction and encourage users to ask questions they might not have otherwise, enhancing user engagement with the product.

Notable Quotes

"If you have any doubts about the quality of the training data, do not proceed."

"You want to assist people in framing the interaction and setting their expectations correctly so they know how to be successful."

"The biggest difference with error states in AI bots is helping people write prompts effectively, not just recovering from simple failures."

"Loading text itself is a loading indicator; the letters appearing show progress better than jumpy animations."

"People often forget what they wrote and then want to check their prompt again before refining it."

"Every output should have at least three source links, almost like citations in a research paper."

"We had to be very careful about accuracy because we're in FinTech and compliance is critical."

"People started asking questions to the bot that they wouldn’t have taken the time to email about."

"It’s really important to be clear and transparent about what the tool is good at and what it’s not good at."

"Accessibility testing included making sure everyone could navigate it using a keyboard alone."

Ask the Rosenbot
Jayne Engle
Civic Design for the Next Seven Generations—A Discussion on Sacred Civics
2022 • Civic Design Community
Alla Weinberg
People Are Sick of Change: Psychological Safety is the Cure
2023 • DesignOps Community
Juhan Sonin
Design Now! The Agenda for Action
2025 • Rosenfeld Community
Melissa Schmidt
How UX Research Hit It Big in Las Vegas
2019 • Enterprise Experience 2019
Gold
Beth Chappell
Leading through ambiguity: Supporting a design team relearning their craft
2026 • Designing with AI 2026
Conference
Louis Rosenfeld
Welcome / Housekeeping
2023 • Enterprise UX 2023
Gold
Ash Brown
Silver Linings: What DesignOps Learned in the Shift to WFH
2020 • DesignOps Summit 2020
Gold
Andrew Custage
The Digital Journey: Research on Consumer Frustration and Loyalty
2023 • Advancing Research 2023
Gold
Vitorio Miliano
Don’t call it AI: Turn words into numbers with quantitative ethnography
2026 • Advancing Research 2026
Gold
Kate Towsey
Participant Recruitment and Management Tools
2026 • Advancing Research 2026
Gold
Christian Crumlish
From tools to staff: What the next generation of agents means for the future of design
2026 • Designing with AI 2026
Conference
Uday Gajendar
Theme One Intro
2023 • Enterprise UX 2023
Gold
Sheri Byrne-Haber
The Importance of Accessible Design Systems
2024 • DesignOps Summit 2020
Gold
Robert Schwartz
We're Here for the Humans
2017 • Enterprise Experience 2017
Gold
Saara Kamppari-Miller
Theme Three Intro
2022 • DesignOps Summit 2022
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold

More Videos

Bria Alexander

"The good vibes you experience are a reflection of the positive collaborative spirit we worked hard to create for you."

Bria Alexander

Theme Two Intro

October 3, 2023

Ariba Jahan

"Check-outs and reflections normalized pausing and being honest about what we don’t know."

Ariba Jahan

Team Resiliency Through a Pandemic

January 8, 2024

Cornelius Rachieru

"If you see a system, you cannot unsee it — systemic awareness changes how you approach design."

Cornelius Rachieru

Handling Complexity: Framing a Scale of Design

June 9, 2021

Megan Clegg

"Automated tests only get you so far; manual QA by people aware of accessibility needs is essential."

Megan Clegg Michael Haggerty-Villa Alexis Morin

Space for Everyone: Reframing Accessibility Through a Wider Lens

June 10, 2021

Saara Kamppari-Miller

"Map tours are my favorite thing—taking a new hire on a guided walk of the entire community."

Saara Kamppari-Miller

Cartography for Design Communities

September 10, 2025

Kara Kane

"Design ideas come back around again and again, waiting for leadership or new contexts to take root."

Kara Kane

Theme One Intro

November 16, 2022

Natalia Radywyl

"Power sharing requires behavior change, reflection, and descending about expertise."

Natalia Radywyl

Co-Designing New Power in Australia's Public Sector

November 16, 2022

Nancy Douyon

"The nobility complex is when we in the West unintentionally create solutions without accounting for our own biases."

Nancy Douyon

We'll Figure That Out in the Next Launch: Enterprise Tech's Nobility Complex

June 15, 2018

Peter Van Dijck

"When GPT came out, my kids immediately adopted it at school and couldn’t pry it from their dead hands."

Peter Van Dijck

Building the Rosenbot

June 4, 2024