Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey
Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • AI product design is best understood through a model, context, and experience framework.

  • Models are stateless and require full context to be sent with each interaction.

  • Prototyping to explore model capabilities is critical before building full products.

  • Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Joerg Beringer
Scaling User Research with AI: Continuous Discovery of User Needs in Minutes
2025 • DesignOps Summit 2025
Gold
Mike Davidson
Fireside Chat
2022 • Advancing Research 2022
Gold
Jill Fruchter
Inconvenient Insights: The Researcher's Role is to Stay Curious
2023 • Advancing Research 2023
Gold
Jamie Beck Alexander
How can you find your role in climate?
2024 • Climate UX Interest Group
Harry Brignull
Beyond Clicks and Tricks: Why deceptive design has grown into a regulatory faultline
2026 • Rosenfeld Community
Peter Van Dijck
Building impactful AI products for design and product leaders, Part 3: Understand AI architectures: RAG, Agents, Oh My!
2025 • Rosenfeld Community
Megan Blocker
Positioning insight: Structuring teams, roles and careers for a changing research landscape
2025 • Advancing Research 2025
Gold
William Newton
How to Lead With Data, and Without Data
2023 • Enterprise UX 2023
Gold
Dr. Jamika D. Burge
How UX researchers can partner with (and not be replaced by) AI [Advancing Research Community Workshop Series]
2023 • Advancing Research Community
Prerna Makanawala
Achieving Balanced Design Consistency
2021 • Design at Scale 2021
Gold
Taylor Jennings
Research Debate Club
2026 • Advancing Research 2026
Conference
Savina Hawkins
Harnessing AI in UXR: Practical Strategies for Positive Impact
2024 • Advancing Research 2024
Gold
Lisa Welchman
Cleaning Up Our Mess: Digital Governance for Designers
2018 • Enterprise Experience 2018
Gold
Alberto Ferreira
Making it Count: Developing a custom digital metric framework that works
2021 • QuantQual Interest Group
John Mortimer
Panel Discussion
2024 • Advancing Service Design 2024
Gold
Megan Blocker
A Selectively Scrappy Approach to ResearchOps
2018 • DesignOps Summit 2018
Gold

More Videos

Jon Fukuda

"Who needs a designer when we have generative AI?"

Jon Fukuda

Theme One Intro

October 2, 2023

Kim Holt

"Trust takes time, hard work, and consistency to build, especially in times of change when time is a luxury we don't have."

Kim Holt Emma Wylds Pearl Koppenhaver Maisee Xiong

A Salesforce Panel Discussion on Values-Driven DesignOps

September 8, 2022

Ariel Kennan

"Embedding inclusivity from the start through equitable research practices is essential."

Ariel Kennan

Theme 2 Intro

December 9, 2021

Brendan Jarvis

"Are the questions that you are asking and the stories that you are telling yourself serving you?"

Brendan Jarvis

Framing Tomorrow by Questioning Today

June 8, 2022

Deanna Smith

"More frequent change isn't always better; good communication and team pulse are key."

Deanna Smith

Leading Change with Confidence: Strategies for Optimizing Your Process

September 23, 2024

Steve Baty

"If you’ve got homogenous people in the room, they’re all blinkered in the same way, and you end up with the same idea."

Steve Baty

Breaking Out of Ruts: Tips for Overcoming the Fear of Change

June 9, 2016

Anat Fintzi

"Asking questions made our direct communication partners think I didn’t know what I was talking about; telling statements worked better."

Anat Fintzi Rachel Minnicks

Delivering at Scale: Making Traction with Resistant Partners

June 9, 2022

George Abraham

"Once you use Indigo.Design in Sketch or XD, you’re just a few clicks away from having a running Angular app based on our developer toolkit."

George Abraham Stefan Ivanov

Design Systems To-Go: Introducing a Starter Design System, and Indigo.Design Overview (Part 1)

September 30, 2021

Sara Conklin

"Every job is a climate job. Wherever we are, the climate needs us to address it in our role."

Sara Conklin

A UXer’s 12-Month Journey from Climate Concern to Climate Credibility

June 26, 2025