Rosenverse
Upcoming Event
Garbage in, garbage out? Measuring error rates to get ready for AI
Thursday, January 8, 2026 • Rosenfeld Community
Share the love for this talk
Garbage in, garbage out? Measuring error rates to get ready for AI
Speakers: Caroline Jarrett
Link:

Summary

We’re all aware of a big push to implement AI everywhere, including in the services that many of us are working on. It seems only fair to try to give the AI some good quality input in the hope of getting decent output from it. Or, being more pessimistic: we probably expect to get some level of errors from the AI, but what do we know about the error rates in what we’re putting into the AI? In this session, we will compare our ideas on identifying errors and measuring error rates, including thinking about errors in six ways: 1) Problems along the way 2) Wrong result 3) Unnecessary action 4) Delayed-impact problem 5) Non-uptake or over-uptake 6) Technology problem We’ll wrap up with “tips and next steps”: an opportunity to consider what we now need to find out or do differently.

Upcoming events

More Videos

Susan Simon-Daniels

"Never assume and always ask why."

Susan Simon-Daniels

War Stories LIVE! Susan Simon-Daniels

March 30, 2020

Carla Casariego

"A tea party of one can really be very lonely, so we decided to seek out some friends to make our tea party more fun."

Carla Casariego Sarah Spencer

DesignOps in Wonderland

October 24, 2019

Alnie Figueroa

"Being a DOT connector means you can see silos, whether people are just not talking to each other, and you can bridge people, process, and tools."

Alnie Figueroa

Teamwork: Strategies for Effective Collaboration with Other Program Management Teams

September 8, 2022

Theresa Marwah

"Legal focuses on risk mitigation; research focuses on ethical best practices."

Theresa Marwah

How Atlassian is Operationalizing Respect in Research

February 27, 2020

Deanna Washington

"Make a business case for yourself by bragging about your capabilities and making them applicable."

Deanna Washington Bria Alexander Jon Fukuda Saara Kamppari-Miller Farid Sabitov

Connecting the Ops: Plenary Panel and Closing Circle

September 9, 2022

Bria Alexander

"The theme is clarity. But really I’m gonna say the theme is you — the 613 people from all over the planet."

Bria Alexander Louis Rosenfeld

Welcome

September 8, 2022

Kim Holt

"Always remember the power of your values in times of doubt and change; lean into what your team really stands for."

Kim Holt Emma Wylds Pearl Koppenhaver Maisee Xiong

A Salesforce Panel Discussion on Values-Driven DesignOps

September 8, 2022

Daniel J. Rosenberg

"About 200,000 digital health products exist, but only 1% of 1% have FDA clearance for accuracy."

Daniel J. Rosenberg

Digital Medicine Design

September 26, 2019

Bria Alexander

"The official conference time zone is Pacific Daylight Time, but you can adjust it based on where you are."

Bria Alexander

Opening Remarks

March 28, 2023