Rosenverse

Log in or create a free Rosenverse account to watch this video.

Log in Create free account

100s of community videos are available to free members. Conference talks are generally available to Gold members.

Building impactful AI products for design and product leaders, Part 1: The new product journey
Wednesday, July 16, 2025 • Rosenfeld Community

This video is featured in the AI and UX playlist.

Share the love for this talk
Building impactful AI products for design and product leaders, Part 1: The new product journey
Speakers: Peter Van Dijck
Link:

Summary

This talk is for product people, UX designers, researchers, and leadership preparing to work on AI products. How do we plan and build helpful and impactful AI products? What is different in this new world? Products that use AI as a core part of the experience are often very easy to prototype, but really hard to build and make actually useful and impactful. A lot of the knowledge on how to build these is locked up in foundation model companies and engineering teams in startups. Peter van Dijck, expert in both AI and IA, will provide an easy to understand framework for understanding how to think about this new world we all live in now, how to plan for effective, helpful and impactful AI products, how to think about model capabilities, AI architectures and evaluations. You will learn vocabulary and frameworks to understand what all the acronyms mean, and how to think about them.

Key Insights

  • AI product design is best understood through a model, context, and experience framework.

  • Models are stateless and require full context to be sent with each interaction.

  • Prototyping to explore model capabilities is critical before building full products.

  • Evals, automated and manual, are essential to measure model output quality and decide if it’s good enough.

  • Models gain enhanced abilities through post-training on specific capabilities like reasoning and conversational skills.

  • Synthetic data generation enables scalable training and evaluation by producing many good and bad examples.

  • Tool use and function calling empower AI agents to autonomously perform multi-step tasks.

  • Agents like Cloud Code gain product-market fit by making model reasoning and actions visible to build user trust.

  • UX principles remain vital in AI product development, especially when designing evals and assessing experience.

  • In high-risk domains, explicit risk identification and eval development are necessary, alongside clear ownership of go/no-go decisions.

Notable Quotes

"Today’s best AI products are designed by engineering first, but UX and product people have to join the party."

"Models are stateless; they take input and give output and forget everything immediately after."

"You want to trust the model, which means not overly instructing it but giving it the right context."

"Evals are the key to knowing when an AI’s output is good enough."

"Model companies post-train models on capabilities by showing them millions of good and bad examples."

"Tool use and function calling are what make agents possible, letting models call tools and reason through tasks."

"Synthetic data is exceedingly useful because you can generate it at scale for training and evals."

"The reason Cloud Code has such product market fit is because it shows what it’s doing, building trust."

"UX skills especially shine in defining realistic, user-centered evals and collaborating with experts."

"If a capability exists and is useful, it’s very likely the model will improve significantly on it over time."

Ask the Rosenbot
Greg Petroff
Software as Material—A Redux
2023 • Enterprise UX 2023
Gold
Catherine Dubut
Bridging Physical and Digital Spaces: Approaches to Retail Service Design
2021 • Enterprise Community
Tamara Hale
War Stories LIVE! Tamara Hale
2020 • Advancing Research 2020
Gold
Sam Proulx
Designing For Screen Readers: Understanding the Mental Models and Techniques of Real Users
2021 • Civic Design 2021
Gold
Ben Davies
Expert Panel: The Principles of Research Repository Design
2022 • Advancing Research 2022
Gold
Ned Dwyer
The Future of DesignOps is Tool Consolidation
2024 • DesignOps Summit 2024
Gold
Erin Hoffman-John
This Game is Never Done: Design Leadership Techniques from the Video Game World
2017 • DesignOps Summit 2017
Gold
Kayla Farrell
What It's Like To Be a User Researcher at Compass
2021 • Advancing Research 2021
Gold
Erin Weigel
Real-world lessons to improve your conversion rates
2024 • Rosenfeld Community
Jamie Beck Alexander
How can you find your role in climate?
2024 • Climate UX Interest Group
Karen McGrane
AI for Information Architects: Are the robots coming for our jobs?
2024 • Rosenfeld Community
Katy Mogal
But Do Your Insights Scale?
2021 • Advancing Research 2021
Gold
Mitchell Bernstein
Organizing Chaos: How IBM is Defining Design Systems with Sketch for an Ever-Changing AI Landscape
2021 • DesignOps Summit 2021
Gold
Kaitlin Tasker
Fast and Fearless Inclusive Research
2023 • Advancing Research 2023
Gold
Dr. Karl Jeffries
The Science of Creativity for DesignOps
2024 • DesignOps Summit 2020
Gold
Bria Alexander
Opening Remarks
2022 • DesignOps Summit 2022
Gold

More Videos

Laine Riley Prokay

"Gender-coded words influence someone’s decision to apply and feel they belong even within internal career ladders."

Laine Riley Prokay

How DesignOps can Drive Inclusive Career Ladders for All

September 30, 2021

Kate Stern

"My design Ops superpower is the ability to simplify incredibly complex processes, problems, and ideas."

Kate Stern

Scaling Learning for the Future

September 9, 2022

Liam Thurston

"We ran ambient workshops so folks could contribute on their own time and remove the friction of synchronous meetings."

Liam Thurston

Why Your Design Team Is Quitting, And How To Fix It

June 10, 2022

Pippa Lomas

"The anxiety and emotional dysregulation create a perfect storm that makes change feel overwhelming for neurodiverse people."

Pippa Lomas

Paving the Path for Neurodiversity in Design

October 4, 2023

Jessamyn Edwards

"If someone asks you for a cup of coffee, here’s the cup of coffee. You did it. But always have proposals on the side for more."

Jessamyn Edwards

Surviving Your UX Career in Enterprise Design

December 2, 2021

Josh Clark

"In-painting allows precise, targeted edits that don’t disturb the rest of the content."

Josh Clark Veronika Kindred

Sentient Design: New Postures for AI-Mediated Experiences (2nd of 3 seminars)

January 29, 2025

Tatyana Mamut

"Pipetting at a biology bench gives empathy no survey or photo can provide."

Tatyana Mamut

Opening Keynote: Breaking Conway's Law--or How to Work Differently and Not Ship Your Org Chart

June 3, 2019

Daniela Magaña Flores

"We use a holistic methodological approach combining explicit and implicit quantitative methods to fully get the user's perspective."

Daniela Magaña Flores Ariane Rahn Jeff Ephraim Bander

Ahead of Competition: Learn What UX Benchmarking Can Do for Your Business Today

March 10, 2022

Theresa Neil

"We were doing all the things until we got the advice to specialize; that moment changed everything."

Theresa Neil

Designing for Wellness: Specializing in Healthcare

May 22, 2024