Summary
A 30-minute deep-dive into the building of the Rosenbot. We’ll get both hands-on practical and likely a bit philosophical. What does it take to build a useful AI assistant? What does it mean for a business, strategically? And how do we make sure we are building the future that we want, while doing all this? Take-aways: What does strategy look like in an AI world? What does eval-first mean? What the hell is going on deep inside an LLM? And what does all that mean for the future we want to build?
Key Insights
-
•
Generative AI is a new design material, similar to previous technological advancements.
-
•
The Rose Bot was created to help users navigate Rosenfeld Media's extensive resources.
-
•
Designers need to rethink their processes and techniques to accommodate generative AI systems.
-
•
Evaluation of AI outputs must involve both expert and end-user perspectives to ensure reliability and usefulness.
-
•
Building generative AI systems requires the development of specialized new tools and techniques for prompt engineering and evaluation.
-
•
Conversations with AI are not simple and require structured conversation logic to improve user experience.
-
•
A significant focus on research is necessary to adapt existing methodologies for testing and evaluation in the context of generative AI.
-
•
The role of UX designers is evolving; they must embrace the challenge of co-creating alongside AI technologies.
-
•
The engineering mindset shouldn't dominate; user involvement is crucial in shaping AI technology's practical applications.
-
•
New methods in user research should reflect the distinct characteristics of generative AI interactions.
Notable Quotes
"I was not a fan of crypto and VR, but when I saw my kids using GPT, the use case became immediately clear."
"This is a new design material; we need to reinvent how we design."
"The job market is crazy; junior UX people are questioning if they joined the right discipline."
"If we build a chatbot, it needs to read everything and analyze it for semantic meaning."
"If you don’t do eval, you’re just building a demo — demos are easy, good quality production is hard."
"Spending 30 minutes on one prompt in a usability test is super fascinating."
"The future is co-created by users and technology; it's not solely driven by tech."
"We need to research this stuff, but we must make up how we research it."
"Let's invent tools that are good for people; it's equally hard to build bad products as it is to build good ones."
"Research right now is super exciting; how do we know if generative AI outputs are good?"
More Videos
"Job security in design is high because the work is complex."
John MaedaMaking Sense of Enterprise UX
June 9, 2016
"There are a lot of variability in how people approach their intent or purpose."
Indi YoungThinking styles: Mend hidden cracks in your market
January 8, 2025
"Measuring the wrong success indicators can stifle progress and growth."
Chris ChapoData Science and Design: A Tale of Two Tribes
May 13, 2015
"Time management in collaboration is more than just meeting deadlines."
Elizabeth ChurchillExploring Cadence: You, Your Team, and Your Enterprise
June 8, 2017
"Our work here is not done. This is a journey."
Rusha SopariwalaRemote, Together: Craft and Collaboration Across Disciplines, Borders, Time Zones, and a Design Org of 170+
June 9, 2022
"The most common way people give up their power is by thinking they don't have any."
George AyeThat Quiet Little Voice: When Design and Ethics Collide
November 16, 2022
"I care about my team's resilience, and I'm here to share some strategies and tactics that work well for us."
Ariba JahanTeam Resiliency Through a Pandemic
January 8, 2024
"Design thinking is a set of behaviors, not just a toolbox of methods."
Julie Gitlin Esther RaiceDesign as an Agent of Digital Transformation at JPMC
June 9, 2021
"We are living in a world made of data and everything that is digital is generating it."
Matteo GrattonCan Data and Ethics Live Together?
October 1, 2021
