Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Enthusiasm for AI tools, especially large language models like ChatGPT, is everywhere, but what does it actually look like to deliver large-scale user-facing experiences using these tools in a production environment? Clearly they're powerful, but what do they need to make them work reliably and at scale? In this session, Sarah provides a perspective on some of the information architecture and user experience infrastructure organizations need to effectively leverage AI. She also shares three AI experiences currently live on Microsoft Learn: An interactive assistant that helps users post high-quality questions to a community forum A tool that dynamically creates learning plans based on goals the user shares A training assistant that clarifies, defines, and guides learners while they study Through lessons learned from shipping these experiences over the last two years, UXers, IAs, and PMs will come away with a better sense of what they might need to make these hyped-up technologies work in real life.
Key Insights
-
•
AI applications must not default to a generic everything chat bot; context and task specificity are crucial.
-
•
Building on a strong infrastructure reduces complexity in AI application development.
-
•
Evaluation should be a major component of AI development efforts; success cannot be gauged on anecdotal evidence.
-
•
Simplicity in tasks leads to higher success rates in AI applications; complex tasks are more challenging to manage.
-
•
User journeys should guide the development of AI features rather than an assumption they can handle everything.
-
•
Understanding task context can lower the ambiguity footprint of AI applications, thus improving usability.
-
•
Improving AI applications often requires organizations to first strengthen their foundational capabilities.
-
•
Visibility of AI interfaces matters; users must understand when and how AI is impacting their experience.
-
•
Dynamic context adds complexity, but can enhance the efficacy of AI tools when managed well.
-
•
A strong foundation in evaluation practices helps teams make informed decisions about AI model effectiveness.
Notable Quotes
"AI isn't the answer to all problems; it's about finding the right context for its use."
"We're not building the perfect model; we're improving the user experience within defined parameters."
"A chatbot is not a silver bullet; understanding user journeys is fundamentally important."
"The ambiguity footprint helps map the complexity of AI applications and the associated risks."
"It's critical to evaluate AI outputs; otherwise, we operate based on gut feelings."
"Ambiguity in AI development is inevitable, but we can manage it to improve our applications."
"When launching AI solutions, build complexity gradually to avoid overwhelming challenges."
"The stakes get higher with sensitive information, making context crucial in development."
"User engagement is more predictable when AI tools complement rather than interfere with tasks."
"Rigorous evaluation processes are as important in AI as they are in traditional product management."
More Videos
"Leadership is best when people barely know she exists."
John MaedaMaking Sense of Enterprise UX
June 9, 2016
"We’re talking about different front ends that vary and respect people's different approaches."
Indi YoungThinking styles: Mend hidden cracks in your market
January 8, 2025
"Data scientists are sometimes thought of as the golden unicorns who can solve anything."
Chris ChapoData Science and Design: A Tale of Two Tribes
May 13, 2015
"Effective teams integrate across silos to enhance productivity."
Elizabeth ChurchillExploring Cadence: You, Your Team, and Your Enterprise
June 8, 2017
"We have to make sure that we are being mindful and inclusive."
Rusha SopariwalaRemote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
June 9, 2022
"The design industry is primed for a movement towards ethical practices and accountability."
George AyeThat Quiet Little Voice: When Design and Ethics Collide
November 16, 2022
"I care about my team's resilience, and I'm here to share some strategies and tactics that work well for us."
Ariba JahanTeam Resiliency Through a Pandemic
January 8, 2024
"The best outcomes come from shared goals and alignment between teams."
Julie Gitlin Esther RaiceDesign as an Agent of Digital Transformation at JPMC
June 9, 2021
"We should help our users follow a process and understand what could be useful for us and why."
Matteo GrattonCan Data and Ethics Live Together?
October 1, 2021
