Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Enthusiasm for AI tools, especially large language models like ChatGPT, is everywhere, but what does it actually look like to deliver large-scale user-facing experiences using these tools in a production environment? Clearly they're powerful, but what do they need to make them work reliably and at scale? In this session, Sarah provides a perspective on some of the information architecture and user experience infrastructure organizations need to effectively leverage AI. She also shares three AI experiences currently live on Microsoft Learn: An interactive assistant that helps users post high-quality questions to a community forum A tool that dynamically creates learning plans based on goals the user shares A training assistant that clarifies, defines, and guides learners while they study Through lessons learned from shipping these experiences over the last two years, UXers, IAs, and PMs will come away with a better sense of what they might need to make these hyped-up technologies work in real life.
Key Insights
-
•
AI applications must not default to a generic everything chat bot; context and task specificity are crucial.
-
•
Building on a strong infrastructure reduces complexity in AI application development.
-
•
Evaluation should be a major component of AI development efforts; success cannot be gauged on anecdotal evidence.
-
•
Simplicity in tasks leads to higher success rates in AI applications; complex tasks are more challenging to manage.
-
•
User journeys should guide the development of AI features rather than an assumption they can handle everything.
-
•
Understanding task context can lower the ambiguity footprint of AI applications, thus improving usability.
-
•
Improving AI applications often requires organizations to first strengthen their foundational capabilities.
-
•
Visibility of AI interfaces matters; users must understand when and how AI is impacting their experience.
-
•
Dynamic context adds complexity, but can enhance the efficacy of AI tools when managed well.
-
•
A strong foundation in evaluation practices helps teams make informed decisions about AI model effectiveness.
Notable Quotes
"AI isn't the answer to all problems; it's about finding the right context for its use."
"We're not building the perfect model; we're improving the user experience within defined parameters."
"A chatbot is not a silver bullet; understanding user journeys is fundamentally important."
"The ambiguity footprint helps map the complexity of AI applications and the associated risks."
"It's critical to evaluate AI outputs; otherwise, we operate based on gut feelings."
"Ambiguity in AI development is inevitable, but we can manage it to improve our applications."
"When launching AI solutions, build complexity gradually to avoid overwhelming challenges."
"The stakes get higher with sensitive information, making context crucial in development."
"User engagement is more predictable when AI tools complement rather than interfere with tasks."
"Rigorous evaluation processes are as important in AI as they are in traditional product management."
More Videos
"We're humans having human experiences; community building has been happening for millions of years."
Jacqui FreyFlow and Superfluidity for Design Orgs
November 7, 2018
"Each time we communicate, we share in the basic need for human connection."
Laura Gatewood Laine ProkayBeyond Buzzwords: Adding Heart to Effective Slack Communication
September 23, 2024
"You can facilitate space for developing insights through collaborative analysis techniques."
Johanna KollmannInsights-Driven Product Strategy: Get your Research to Count
December 6, 2022
"We will absolutely promise to let you know when these resources are available for you to enjoy."
Bria AlexanderDay 3 Welcome
September 25, 2024
"Don't limit yourself to creating everything yourself—take screenshots, combine them, and really explain what you're going for visually."
Billy CarlsonIdeation tips for Product Managers
December 6, 2022
"When expectations are low for process, it’s the perfect opportunity to rifle through new approaches."
Catherine DubutBridging Physical and Digital Spaces: Approaches to Retail Service Design
March 18, 2021
"The values we uphold guide how we experience and influence team dynamics."
Kim Holt Emma Wylds Pearl Koppenhaver Maisee XiongA Salesforce Panel Discussion on Values-Driven DesignOps
September 8, 2022
"When you experience problems with different disciplines, keep Sturgeon's biases in mind."
Adrian HowardSturgeon’s Biases
September 25, 2024
"Our work can't be evaluated in the same way that engineering work can."
Asia HoePartnering with Product: A Journey from Junior to Senior Design
November 29, 2023
