Rosenverse
Upcoming Event
Building impactful AI products for design and product leaders, Part 2: Evals are your moat
Wednesday, July 23, 2025 • Rosenfeld Community
Share the love for this talk
Building impactful AI products for design and product leaders, Part 2: Evals are your moat
Speakers: Peter Van Dijck
Link:

Summary

The secret ingredient for impactful AI products is “evals”—an architecture for ongoing evaluation of quality. Without evals, you don’t know if your output is good. You don’t know when you’re done. Because outputs are non-deterministic, it’s very hard to figure out if you are creating real value for your users, and when something goes wrong, it’s really tricky to figure out why. Simply Put’s Peter van Dijck will demystify evals, and share a simple framework for planning for and building useful evals, from qualitative user research to automated evals using LLMs as a judge.

Upcoming events

More Videos

Ed Mullen

"Designers play a key role in shaping the developer experience to enhance productivity in government agencies."

Ed Mullen

Designing the Unseen: Enabling Institutions to Build Public Trust

November 16, 2022

Peter Levin

"Organizations want to solve big existential, durable questions."

Peter Levin

Solve a Problem Here, Transform a Strategy There: Research as an Occasion for Expanding Organizational Possibility

March 25, 2024

Bria Alexander

"We need to see OKRs as a journey, not just a concrete goal."

Bria Alexander Benson Low Natalya Pemberton Stephanie Goldthorpe

OKRs—Helpful or Harmful?

January 20, 2022

Kate Koch

"One of the coolest things an operations team can have is building the tools that we need to enable growth at scale"

Kate Koch Prateek Kalli

Flex Your Super Powers: When a Design Ops Team Scales to Power CX

September 30, 2021

Amy Marquez

"Our lenders want the best experience for our borrowers."

Amy Marquez

INVEST: Discussion

June 15, 2018

Brenna Fallon

"What you build matters, but what matters more is if you learn."

Brenna Fallon

Learning Over Outcomes

October 24, 2019

Brian Moss

"Resilience should be supported by organizational structures, not just individuals."

Brian Moss

What Does it Mean to be a Resilient Research Team?

March 9, 2022

Adel Du Toit

"We hold the mirror up to our product teams and also to every single one of our 25,000 employees."

Adel Du Toit

Get Your CFO To Say: 'Our Strategic Goal is User Obsession'

June 10, 2022

Kate Koch

"We love to flex our superpowers, but we’re trying to use them for good and not evil."

Kate Koch Jean-Claire Fitschen

Flex Your Super Powers: When a Design Ops Team Scales to Power CX

September 29, 2021