Rosenverse
Building impactful AI products for design and product leaders, Part 1: The new product journey

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey

Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • AI product design is best understood through a model, context, and experience framework.

  • Models are stateless and require full context to be sent with each interaction.

  • Prototyping to explore model capabilities is critical before building full products.

  • Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Heidi Trost
When AI Becomes the User’s Point Person—and Point of Failure
2025 • Rosenfeld Community
Chris Geison
Theme 1 Intro
2022 • Advancing Research 2022
Gold
Sarah Flamion
Complex Problem? Add Clarity by Combining Research and Systems Thinking
2020 • Advancing Research 2020
Gold
Silke Bochat
5 Antifragile Strategies for a DesignOps 2.0
2024 • DesignOps Summit 2024
Gold
Kit Unger
Theme 2: Introduction
2021 • Design at Scale 2021
Gold
Amy Parness
Scaling Sustainability: Complementary strategies that drive long-term success
2025 • Climate UX Interest Group
Courtney Maya George
Scale Your Organization and Grow Your Designers
2022 • DesignOps Summit 2022
Gold
John Calhoun
Have we Reached Our Peak? Spotting the Next Mountain For DesignOps to Climb
2021 • DesignOps Summit 2021
Gold
Bruce Gillespie
Learning from journalism: Balancing impactful communication with compassionate storytelling
2025 • Advancing Research 2025
Gold
Kurt McCulloch
Faster alone, further together: Rebuilding collaboration in the age of AI research
2026 • Advancing Research 2026
Gold
Chris Hodowanec
Agile + User Experience: How to navigate the Agile landscape as an UX Practitioner
2022 • Civic Design 2022
Gold
Daniel Gloyd
Warming the User Experience: Lessons from America's first and most radical human-centered designers
2024 • Rosenfeld Community
Sam Proulx
Mobile Accessibility: Why Moving Accessibility Beyond the Desktop is Critical in a Mobile-first World
2022 • DesignOps Summit 2022
Gold
Boon Yew Chew
Making Sense of Systems—and Using Systems to Make Sense of the Enterprise
2023 • Enterprise UX 2023
Gold
Jemma Ahmed
Convergent Research Techniques in Customer Journey Mapping
2020 • Advancing Research 2020
Gold
Michelle Bejian Lotia
Rolling Out a Repository: How Zapier Centralizes Insights from Across their Organization
2023 • Advancing Research 2023
Gold

More Videos

Jeff Sussna

"The very thing that got us into trouble—the complexity of small pieces—is exactly our way out of it."

Jeff Sussna

What DesignOps Can Learn From DevOps

November 6, 2017

Bill Scott

"You have to design for throw away ability because the majority of experience gets thrown away."

Bill Scott

Lean Engineering: Engineering for Learning and Experimentation in the Enterprise

May 14, 2015

Boon Yew Chew

"Good services are verbs, bad services are nouns – using verbs helps users know where they are in the workflow."

Boon Yew Chew

Making Sense of Systems—and Using Systems to Make Sense of the Enterprise

June 6, 2023

Bob Baxley

"Design always means change; if it amplifies what’s already working, that itself is change."

Bob Baxley Sara Asche Anderson Sharon Bautista Frank Duran Jamie Kaspszak Abbey Smalley Sylas Souza

Theme 4: Discussion

January 8, 2024

Sarah Brooks

"The policy was about walking distance as the crow flies, but that wasn’t walkable—so the policy needed to change."

Sarah Brooks Jennifer Pahlka

Fireside chat with Sarah Brooks and Jen Pahlka

October 21, 2021

Andrew Webster

"Leading indicators include behavioral impacts like asking user-centered questions and gathering user feedback early and often."

Andrew Webster

Scaling Design Capability: How Involved Should You Be?

September 30, 2021

Mansi Gupta

"Safety is a fundamental human need which is often taken for granted and maybe that’s why it’s forgotten about."

Mansi Gupta

Drawing from Feminist Practice to Make Inclusive Design Operational

September 9, 2022

Hugh Dubberly

"You don’t think of raising a child or teaching a student as solving a problem, it’s an ongoing process—and so is design today."

Hugh Dubberly

Problems with Problems: Reconsidering the Frame of Designing as Problem-Solving

June 19, 2019

Lin Nie

"Science doesn’t just check truth; it has to win hearts and minds."

Lin Nie

When Thought-worlds Collide: Collaborating Between Research and Practice

March 10, 2021