This video is only accessible to conference ticket holders.
Log in Create account
For 90 days after a conference, only paid ticket holders can watch conference videos. After that, all Gold members have access.
Contact Support if you are having any issues.
This video is featured in the AI and UX playlist.
Summary
This talk addresses the distinct challenges of designing AI-first products when the foundational technology is advancing at an unprecedented pace. We will discuss the practical implications of this rapid evolution on the design process and share our insights into creating effective user experiences in such a dynamic environment.
Key Insights
-
•
AI models evolve faster than previous tech revolutions, requiring adaptive product development.
-
•
Effective AI products use evals—automated tests on real data—to continuously measure quality.
-
•
Retrieval augmented generation and tool use are practical architectural tricks to reduce AI hallucination.
-
•
AI experiences must account for user trust despite AI’s confident but sometimes inaccurate outputs.
-
•
Human factors like worker anxiety about job security are critical to address in AI adoption.
-
•
Cross-disciplinary teams are essential in AI design due to the space’s novelty and uncertainty.
-
•
Prompt engineering skills grow best through extensive real use rather than frozen techniques.
-
•
Planning AI products with expectations of improved capabilities 6-9 months ahead optimizes outcomes.
-
•
Cultural acceptance of AI use inside organizations must be actively encouraged to avoid user hiding.
-
•
User testing of AI products still requires old-school observation combined with analytics and evals.
Notable Quotes
"You can’t have good AI without good information architecture."
"Everything’s changing really quickly—if you don’t revise your plan in a few months, you’re building for a world that’s already outdated."
"Hallucinations are when AI makes up stuff—it's a fundamental property of these models."
"Retrieval augmented generation is just adding results from a search into the model’s context to make it more grounded."
"Agents are models using tools in the loop, calling normal software functions as part of their process."
"People tend to trust AI a lot because it seems very confident—even when it’s not always accurate."
"Workers are often worried about layoffs and their value when AI speeds up their process, while executives like the efficiency gains."
"You should get good at prompt engineering by using AI a lot, not by memorizing outdated tricks."
"Building evals is like building tests to know if your AI system is good enough, and you can rerun them as models improve."
"User testing for AI still means watching people use it, plus collecting analytics and running evaluations."
Dig deeper—ask the Rosenbot:
More Videos
"As your design practice matures, your design ops team will reach inflection points, and it’s okay to flip the coin and explore the other side."
John Calhoun Rachel PosmanTwo Sides of the DesignOps Coin: Teams Ops and Product Ops
January 8, 2024
"Social pain affects the nervous system in the exact same way as physical pain."
Laura Gatewood Laine ProkayBeyond Buzzwords: Adding Heart to Effective Slack Communication
September 23, 2024
"Picking the wrong size image can impact customer experience as much as what you see on screen."
Erin WeigelUX Lessons from running more than 1,200 A/B Tests
July 10, 2024
"You must remember this. A kiss is just a kiss. A sigh is just a sigh."
Susan Simon-DanielsWar Stories LIVE! Susan Simon-Daniels
March 30, 2020
"Some bias against human-centered UX comes from concern that ordinary folks don't understand technical software requirements."
Noreen Whysel Katie SaindonShort Take #4: UX/Product Lessons from Your Industry Peers
December 6, 2022
"You can be most successful at using new note-taking tools if you bring to the practice a different set of mental models."
Jorge ArangoExploding the Notebook: How to Unlock the Power of Linked Notes (2nd of 3 seminars)
April 19, 2024
"Digital governance is about who’s supposed to make the decision, not what the decision is."
Lisa WelchmanCleaning Up Our Mess: Digital Governance for Designers
June 14, 2018
"Land acknowledgement done as a checkbox is performative; when embedded, it becomes a practice of respect and inclusion."
Zariah CameronStreamlining an Inclusive Design Practice
October 3, 2023
"Building a portfolio of talks, refreshed over time, lets you keep evolving and sharing your ideas in different formats."
Louis Rosenfeld Jemma Ahmed Christian Crumlish Uday Gajendar Chris GeisonCoffee with Lou #3: What Makes for a Successful UX Conference Presentation?
May 2, 2024
