Log in or create a free Rosenverse account to watch this video.
Log in Create free account100s of community videos are available to free members. Conference talks are generally available to Gold members.
Summary
Enthusiasm for AI tools, especially large language models like ChatGPT, is everywhere, but what does it actually look like to deliver large-scale user-facing experiences using these tools in a production environment? Clearly they're powerful, but what do they need to make them work reliably and at scale? In this session, Sarah provides a perspective on some of the information architecture and user experience infrastructure organizations need to effectively leverage AI. She also shares three AI experiences currently live on Microsoft Learn: An interactive assistant that helps users post high-quality questions to a community forum A tool that dynamically creates learning plans based on goals the user shares A training assistant that clarifies, defines, and guides learners while they study Through lessons learned from shipping these experiences over the last two years, UXers, IAs, and PMs will come away with a better sense of what they might need to make these hyped-up technologies work in real life.
Key Insights
-
•
AI applications must not default to a generic everything chat bot; context and task specificity are crucial.
-
•
Building on a strong infrastructure reduces complexity in AI application development.
-
•
Evaluation should be a major component of AI development efforts; success cannot be gauged on anecdotal evidence.
-
•
Simplicity in tasks leads to higher success rates in AI applications; complex tasks are more challenging to manage.
-
•
User journeys should guide the development of AI features rather than an assumption they can handle everything.
-
•
Understanding task context can lower the ambiguity footprint of AI applications, thus improving usability.
-
•
Improving AI applications often requires organizations to first strengthen their foundational capabilities.
-
•
Visibility of AI interfaces matters; users must understand when and how AI is impacting their experience.
-
•
Dynamic context adds complexity, but can enhance the efficacy of AI tools when managed well.
-
•
A strong foundation in evaluation practices helps teams make informed decisions about AI model effectiveness.
Notable Quotes
"AI isn't the answer to all problems; it's about finding the right context for its use."
"We're not building the perfect model; we're improving the user experience within defined parameters."
"A chatbot is not a silver bullet; understanding user journeys is fundamentally important."
"The ambiguity footprint helps map the complexity of AI applications and the associated risks."
"It's critical to evaluate AI outputs; otherwise, we operate based on gut feelings."
"Ambiguity in AI development is inevitable, but we can manage it to improve our applications."
"When launching AI solutions, build complexity gradually to avoid overwhelming challenges."
"The stakes get higher with sensitive information, making context crucial in development."
"User engagement is more predictable when AI tools complement rather than interfere with tasks."
"Rigorous evaluation processes are as important in AI as they are in traditional product management."
More Videos
"Integrated discovery helps with fostering a learning mindset culture."
Mike OrenWhy Pharmaceutical's Research Model Should Replace Design Thinking
March 28, 2023
"It's easier for developers to focus on mobile due to fewer compatibility issues."
Sam ProulxMobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
November 17, 2022
"We will record every talk and have all videos linked from the program page."
Louis Rosenfeld Christian CrumlishOpening Remarks
November 29, 2023
"We need to work on the right problems to maximize our impact within the organization."
Patrick CommarfordDesign Staffing for Impact
January 8, 2024
"If a problem presents itself over and over again, it's not necessarily the concept that fails."
Erin WeigelUX Lessons from running more than 1,200 A/B Tests
July 10, 2024
"Quick wins help establish trust and credibility with stakeholders."
Magdalena ZadaraZero Hour: How to Get Far Quickly When Starting Your Digital Service Unit Late
November 16, 2022
"Continuous slicing of learnings allows for meaningful and viable outcomes."
Sharbani DharBreathing Room for Delight
January 8, 2024
"When designing at scale, AI tools can help create user journeys quickly and effectively."
Kelly DernAI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process
October 3, 2023
"You can measure behavior change in a way that predicts your outputs will change in a measurable way."
Saara Kamppari-Miller Nicole Bergstrom Shashi JainKey Metrics: Comparing Three Letter Acronym Metrics That Include the Word “Key”
November 13, 2024
