Summary
A 30-minute deep-dive into the building of the Rosenbot. We’ll get both hands-on practical and likely a bit philosophical. What does it take to build a useful AI assistant? What does it mean for a business, strategically? And how do we make sure we are building the future that we want, while doing all this? Take-aways: What does strategy look like in an AI world? What does eval-first mean? What the hell is going on deep inside an LLM? And what does all that mean for the future we want to build?
Key Insights
-
•
Generative AI is a new design material, similar to previous technological advancements.
-
•
The Rose Bot was created to help users navigate Rosenfeld Media's extensive resources.
-
•
Designers need to rethink their processes and techniques to accommodate generative AI systems.
-
•
Evaluation of AI outputs must involve both expert and end-user perspectives to ensure reliability and usefulness.
-
•
Building generative AI systems requires the development of specialized new tools and techniques for prompt engineering and evaluation.
-
•
Conversations with AI are not simple and require structured conversation logic to improve user experience.
-
•
A significant focus on research is necessary to adapt existing methodologies for testing and evaluation in the context of generative AI.
-
•
The role of UX designers is evolving; they must embrace the challenge of co-creating alongside AI technologies.
-
•
The engineering mindset shouldn't dominate; user involvement is crucial in shaping AI technology's practical applications.
-
•
New methods in user research should reflect the distinct characteristics of generative AI interactions.
Notable Quotes
"I was not a fan of crypto and VR, but when I saw my kids using GPT, the use case became immediately clear."
"This is a new design material; we need to reinvent how we design."
"The job market is crazy; junior UX people are questioning if they joined the right discipline."
"If we build a chatbot, it needs to read everything and analyze it for semantic meaning."
"If you don’t do eval, you’re just building a demo — demos are easy, good quality production is hard."
"Spending 30 minutes on one prompt in a usability test is super fascinating."
"The future is co-created by users and technology; it's not solely driven by tech."
"We need to research this stuff, but we must make up how we research it."
"Let's invent tools that are good for people; it's equally hard to build bad products as it is to build good ones."
"Research right now is super exciting; how do we know if generative AI outputs are good?"
















More Videos

"Conversations around climate shouldn't be abstract but must include practical discussions."
Sheryl Cababa Alexis OhThinking in systems to address climate with Sheryl Cababa
June 12, 2024

"We can use AI to generate ways to visualize and iterate designs, but human input remains essential."
Louis Rosenfeld Billy Carlson Jon Fukuda Maria TaylorHow AI will Change DesignOps Tooling
October 3, 2023

"Make sure your prompts have constraints; otherwise, you will get unrefined ideas."
Kelly DernAI as a Design Partner: How to Get the Most Out of AI Tools to Scale Your Process
October 3, 2023

"Our work must reflect equity and care in how we operate."
Ariel KennanTheme Two Intro
November 17, 2022

"We need to coalesce different experiences to create a practice around what Design Ops really is."
Dave Malouf Meredith Black Farid SabitovThe Past, Present, and Future of DesignOps: a 2-part DesignOps Community Call (Part 1)
February 17, 2022

"Understanding context requires being physically present; remote can miss crucial nuances."
Sarah RinkRemote User Research: Dos and Don'ts from the Virtual Field
June 11, 2020

"You can try accessibility tools on mobile without needing corporate approval, making testing much easier."
Sam ProulxMobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
September 8, 2022

"Our panels have allowed researchers to engage with non-English speakers quickly and effectively."
Wyatt HaymanGlobal Research Panels
August 8, 2020

"Public displays foster engagement and trust among teams."
Elizabeth ChurchillExploring Cadence: You, Your Team, and Your Enterprise
June 8, 2017