Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey
Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • AI product design is best understood through a model, context, and experience framework.

  • Models are stateless and require full context to be sent with each interaction.

  • Prototyping to explore model capabilities is critical before building full products.

  • Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Kurdin Bazaz
Culture, DIBS & Recruiting
2021 • Design at Scale 2021
Gold
Ignacio Martinez
Fair and Effective Designer Evaluation
2024 • DesignOps Summit 2024
Gold
Kristin Skinner
Five Years of DesignOps
2021 • DesignOps Summit 2021
Gold
Jennifer Kong
Journeying toward AI-assisted documentation in healthcare
2024 • Designing with AI 2024
Gold
Davis Neable
How to Drive a Design Project When you Don’t Have a Design Team
2021 • Design at Scale 2021
Gold
Russ Unger
Getting Out from Under Everyone: How to Escape the Paralysis of Getting Started
2016 • Enterprise UX 2016
Gold
Lija Hogan
Doing more with more: Lessons from the Front Lines of Democratization
2022 • Advancing Research 2022
Gold
Michelle Bejian Lotia
Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
2023 • Advancing Research 2023
Gold
Brianna Sylver
Lead With Purpose
2020 • Advancing Research 2020
Gold
Sam Ladner
How Research Can Drive Strategic Foresight
2022 • Advancing Research 2022
Gold
Marina Martin
Lives on the Line: The Stakes of UX at the Scale of Government
2018 • Enterprise Experience 2018
Gold
Maria Skaaden
Panel Discussion: Methodologies and Work Environments
2018 • DesignOps Summit 2018
Gold
Bria Alexander
Theme Two Intro
2022 • DesignOps Summit 2022
Gold
Christian Madsbjerg
Influencing Strategy
2020 • Advancing Research 2020
Gold
Dave Malouf
The Future of DesignOps
2024 • DesignOps Summit 2020
Gold
Scott Jensen
Short Take #2: UX/Product Lessons from Your Industry Peers
2022 • Design in Product 2022
Gold

More Videos

Frances Yllana

"Power is liquid — you might have it in one space and not in another."

Frances Yllana Ann Buechner Jess Jones Betsy Ramaccia

D.E.A.R.R. Diaries (Discipline, Experience, Architecture, Reflection + Revolution)

November 16, 2022

Katie Hansen

"When we hear experiment, most think AB testing, but experimental research goes beyond to uncover the why behind outcomes."

Katie Hansen

Experimental research: techniques for deep, psychology-driven insights

March 12, 2025

Dave Hoffer

"Networking isn’t just asking for favors — it’s about building relationships and doing something for others first."

Dave Hoffer Joanne Weaver

UX Job Search AMA #2 with Joanne Weaver and Dave Hoffer

May 21, 2025

Lisa Gironda

"I wasn’t planning to learn SQL scripts but I did. And it was fun. It was a new language."

Lisa Gironda

Opener: Chief of Staff–An unexpected journey

January 8, 2024

Ted Booth

"Building a distribution list of user-centered design success stories created a groundswell with 1,200 subscribers hungry for more."

Ted Booth Sam Ladner Fredrik Matheson Russ Unger

Discussion

June 8, 2016

Dan Willis

"Once people have seen the value of involving users and design, they can’t unsee it—no matter the churn or setbacks."

Dan Willis

Filling the Void

November 7, 2018

Maria Giudice

"We have to go from shipping a minimal viable product to creating a minimal lovable experience."

Maria Giudice

Remaking the Making Company: Moving from Product to Experience

June 9, 2016

Victor Udoewa

"Community members giving their time sacrifice something far more valuable than paid professionals do."

Victor Udoewa

Radical Participatory Design: Decolonizing Participatory Design Processes

December 10, 2021

Renee Reid

"Let me just take you along this journey, where it began."

Renee Reid

Becoming a ResearchH.E.R (Highly Enterprise Ready)

June 3, 2019