Peter Van Dijck

" Avoid generic built-in evals, because then you're not defining quality for your specific situation. "

" Your short term memory is the context window; too big, and the model has trouble accessing everything. "

" You want to ask an LLM to evaluate the fuzzy stuff because there’s no black and white output. "

" Evals are like a way to define what good looks like. "

" We want you to take all AI-generated info with a grain of salt but it gets you part of the way. "

" If you understand how engineers think about these models, you won’t be scared of concepts like synthetic data or tool use. "

" Evals are really your intellectual property—they define what good looks like in your domain. "

" Model companies post-train models on capabilities by showing them millions of good and bad examples. "

" Hallucinations are when AI makes up stuff—it's a fundamental property of these models. "

" If you don’t jump in and learn, you’ll spend your time reading stuff on it and not really understand the tech. "

" The Rose Bot has read everything—every piece of Rosenfeld’s knowledge—to help users access it. "