Summary
A 30-minute deep-dive into the building of the Rosenbot. We’ll get both hands-on practical and likely a bit philosophical. What does it take to build a useful AI assistant? What does it mean for a business, strategically? And how do we make sure we are building the future that we want, while doing all this? Take-aways: What does strategy look like in an AI world? What does eval-first mean? What the hell is going on deep inside an LLM? And what does all that mean for the future we want to build?
Key Insights
-
•
Generative AI is a new design material, similar to previous technological advancements.
-
•
The Rose Bot was created to help users navigate Rosenfeld Media's extensive resources.
-
•
Designers need to rethink their processes and techniques to accommodate generative AI systems.
-
•
Evaluation of AI outputs must involve both expert and end-user perspectives to ensure reliability and usefulness.
-
•
Building generative AI systems requires the development of specialized new tools and techniques for prompt engineering and evaluation.
-
•
Conversations with AI are not simple and require structured conversation logic to improve user experience.
-
•
A significant focus on research is necessary to adapt existing methodologies for testing and evaluation in the context of generative AI.
-
•
The role of UX designers is evolving; they must embrace the challenge of co-creating alongside AI technologies.
-
•
The engineering mindset shouldn't dominate; user involvement is crucial in shaping AI technology's practical applications.
-
•
New methods in user research should reflect the distinct characteristics of generative AI interactions.
Notable Quotes
"I was not a fan of crypto and VR, but when I saw my kids using GPT, the use case became immediately clear."
"This is a new design material; we need to reinvent how we design."
"The job market is crazy; junior UX people are questioning if they joined the right discipline."
"If we build a chatbot, it needs to read everything and analyze it for semantic meaning."
"If you don’t do eval, you’re just building a demo — demos are easy, good quality production is hard."
"Spending 30 minutes on one prompt in a usability test is super fascinating."
"The future is co-created by users and technology; it's not solely driven by tech."
"We need to research this stuff, but we must make up how we research it."
"Let's invent tools that are good for people; it's equally hard to build bad products as it is to build good ones."
"Research right now is super exciting; how do we know if generative AI outputs are good?"
More Videos
"The very innovation you may need might already be written."
Jacqui FreyFlow and Superfluidity for Design Orgs
November 7, 2018
"Every time we communicate, we are indeed sharing a basic need for human connection."
Laura Gatewood Laine ProkayBeyond Buzzwords: Adding Heart to Effective Slack Communication
September 23, 2024
"You can facilitate space for developing insights through collaborative analysis techniques."
Johanna KollmannInsights-Driven Product Strategy: Get your Research to Count
December 6, 2022
"Take advantage of our sponsor sessions because they are really high quality and free."
Bria AlexanderDay 3 Welcome
September 25, 2024
"If you have really good ideation techniques built into your design process, you can save a lot of time later on and a lot of headaches."
Billy CarlsonIdeation tips for Product Managers
December 6, 2022
"It's easy to stay immersed in digital, but physical contexts matter immensely for designing multi-channel experiences."
Catherine DubutBridging Physical and Digital Spaces: Approaches to Retail Service Design
March 18, 2021
"Peer recognition programs like shout-out Thursdays are low-effort yet high-impact."
Kim Holt Emma Wylds Pearl Koppenhaver Maisee XiongA Salesforce Panel Discussion on Values-Driven DesignOps
September 8, 2022
"90% of everything is crud, including the people we work alongside."
Adrian HowardSturgeon’s Biases
September 25, 2024
"Good time zone everyone."
Asia HoePartnering with Product: A Journey from Junior to Senior Design
November 29, 2023
