Rosenverse

This video is only accessible to Gold members. Log in or register for a free Gold Trial Account to watch.

Log in Register

Most conference talks are accessible to Gold members, while community videos are generally available to all logged-in members.

[Demo] Complexity in disguise: Crafting experiences for generative AI features
Gold
Wednesday, June 5, 2024 • Designing with AI 2024
Share the love for this talk
[Demo] Complexity in disguise: Crafting experiences for generative AI features
Speakers: Trisha Causley
Link:

Summary

AI tools like ChatGPT have exploded in popularity with good reason: they allow users to draft, summarize, and edit content with unprecedented speed. While these generic tools can generate any type of content or perform any type of content task, the user needs to craft an effective prompt to get high-quality output, and often needs to exchange multiple messages with additional guidance and requirements in order to improve results. When you’re building an AI-powered text generation feature, such as a product description or email writer, you typically can’t expect users to craft their own prompts. And unless you’re building a chat interface, you’re unlikely to offer the ability to iteratively improve the output. Instead, your feature needs a robust prompt skeleton that combines with user input to produce high-quality output in a single response. For the designer, this means building an interface that helps users provide the exact information that creates a successful prompt. This process is more complex than simple form design or a mad-lib prompt completion tool. The user input, often including free form text fields, might be required to fill in prompt variables, but it also could change the prompt structure itself, or even override base instructions. The effectiveness of the user input significantly influences the quality of the output, underscoring the need for designers to be deeply familiar with the backend prompt architecture so they can design the frontend. Drawing on recent text generation projects, I'll demonstrate how the interface design can respond to and evolve with the prompt architecture. I’ll talk about how to determine which prompt components to make invisible to the user, which to provide as predefined options, and which should be authored by the user in free-form text fields. Takeaways How prompt structure can impact user interface design and conversely, how design can impact prompt structure Techniques to provide effective user guidance within AI generation contexts to ensure consistently high-quality output Real-world examples and learnings from recent generative AI projects in an e-commerce software product

Key Insights

  • Building specialized AI features requires predefining most of the prompt to guide the LLM effectively rather than having users write their own prompts.

  • Users refine prompts iteratively in chat interfaces, but specialized tools often allow only one shot with limited input variables.

  • Understanding the backend prompt structure is critical for designers to decide which parts are fixed and which are configurable by users.

  • Tone of voice is a complex variable that significantly impacts product description differentiation and brand personality.

  • Free-text inputs for tone proved difficult for users; predefined tone options simplify user choices while maintaining quality.

  • AI models inconsistently detect tone from existing text samples and report high confidence in contradictory results, making this approach unreliable.

  • Successful tone options must be sufficiently distinct to allow users to easily identify and select the best fit for their brand.

  • Embedding detailed tone attribute descriptions (e.g., vocabulary, pronouns, punctuation) in prompts improves AI output quality.

  • The default confident and positive tone of models like ChatGPT may not suit all contexts and needs to be explicitly adjusted in prompts.

  • Providing a minimal, intuitive UI that maps to rich, complex backend instructions balances user simplicity with output quality.

Notable Quotes

"In a chat interface, you’re able to fill in the information that you missed and iterate on the output as it’s being generated."

"When you’re building a specific AI feature, you don’t want to make your user write their own prompt."

"It becomes the designer’s responsibility to figure out what aspects of the instructions to the LLM should be configurable and what should be fixed."

"A prompt is just a set of instructions to an AI in the context of text generation."

"People get stuck during the sessions when asked to describe their brand voice in just one or two words."

"The LLM gave many different answers for the same passage, each time reporting it was one hundred percent confident."

"Tones need to be distinct enough so that merchants can spot the tone that best fits their brand."

"Even though we’ve got a single dropdown field, it maps to a far richer set of instructions in the prompt."

"The confident tone built into these tools may be one of the more problematic features of Gen AI."

"By default, you’re getting I am very confident of my answers."

Ask the Rosenbot
PJ Buddhari
Meet Spectrum, Adobe’s Design System
2021 • Design at Scale 2021
Gold
Sam Proulx
Online Shopping: Designing an Accessible Experience
2023 • Design in Product 2023
Gold
Giff Constable
Financial fluency for product leaders: AMA with Giff Constable
2024 • Rosenfeld Community
Sheryl Cababa
Day 2 Panel
2024 • Designing with AI 2024
Gold
Prayag Narula
HCI 2.0: Humanity Deserves the Attention that UX Research has to Offer
2023 • Advancing Research 2023
Gold
Fredrik Matheson
First-time users, longtime strategies: Why Parkinson’s Law is making you less effective at work – and how to design a fix.
2016 • Enterprise UX 2016
Gold
Erika Flowers
AI-Readiness: Preparing NASA for a Data-Driven, Agile Future
2025 • Designing with AI 2025
Gold
Briana Thomas
When Design Ops Comes in H.O.T. : A Tale of a Transformed Design Org
2021 • DesignOps Summit 2021
Gold
Jackie Ajoux
Leveling-Up: A Single-Player’s Guide to the DesignOps Team-of-One
2024 • DesignOps Summit 2020
Gold
Amahra Spence
Designing for Liberation, Rehearsing Freedom
2022 • Civic Design 2022
Gold
Rusha Sopariwala
Remote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
2022 • Design at Scale 2022
Gold
Lena Shenkarenko
Collaborative Wireframing for Creating Team Alignment and Shipping Better Products
2020 • DesignOps Summit 2020
Gold
Tricia Wang
Spatial Collapse: Designing for Emergent Culture
2024 • Enterprise Experience 2020
Gold
Scott Stephens
The Next Generation in DesignOps Toolsets
2022 • DesignOps Community
Catt Small
Moving from Execution to Strategy as a Designer
2022 • Design in Product 2022
Gold
Jim Kalbach
Peace is waged with sticky notes: Mapping Real-World Experiences
2018 • Enterprise Experience 2018
Gold

More Videos

Nalini Kotamraju

"We made this move in January 2020 — our research and insights team moved out of UX but stayed within product organizations."

Nalini Kotamraju

Research After UX

March 25, 2024

Dean Broadley

"Constraints can channel focus, but being stuck in them limits potential."

Dean Broadley

Not Black Enough to be White

January 8, 2024

Denise Jacobs

"When we start changing our behavior, we start changing the voices and behavior around us."

Denise Jacobs Nancy Douyon Renee Reid Lisa Welchman

Interactive Keynote: Social Change by Design

January 8, 2024

Kim Fellman Cohen

"Failure isn’t discouraging; it’s an opportunity to learn and improve your approach."

Kim Fellman Cohen

Measuring the Designer Experience

October 23, 2019

George Aye

"Why do designers decide how to design this? How do we wean design's addiction from whiteness?"

George Aye

That Quiet Little Voice: When Design and Ethics Collide

November 16, 2022

Nathan Curtis

"You need to be the connector between people trying to drive decisions and others influencing them."

Nathan Curtis

Beyond the Toolkit: Spreading a System Across People & Products

June 9, 2016

Greg Petroff

"Compass is a pretty frank place where people feel safe to speak their minds, even at leadership levels."

Greg Petroff

The Compass Mission

March 10, 2021

Chloe Amos-Edkins

"Diverse research teams blend outsider and insider perspectives for authentic cultural translation."

Chloe Amos-Edkins

A Cultural Approach: Research in the Context of Glocalisation

March 27, 2023

Mackenzie Cockram

"The new site had a 19% dropout rate compared to 32% on the old site, showing clear improvement."

Mackenzie Cockram Sara Branco Cunha Ian Franklin

Integrating Qualitative and Quantitative Research from Discovery to Live

December 16, 2022