How Evals and Prompts Shape Agent Behavior — Preetika Bhateja & Daniel Bump, YouTube Ads

Hi everyone. Uh, sounds like everybody came back from lunch, so I hope everybody is recharged and not sleepy at all. It's always interesting to do a talkright after lunch because you never know, it's a mixed crowd. Uh, but we're very happy to be here, happy to see you all.

Our talk is going to be about evals, of course. We're in the evals track. We're going to talk you through what are some things that worked for us while we were building evals, uh, especially for YouTube ads. We work on the YouTube ads team as part of the we do image and video models for YouTube ads.

So, building an agent is hard. I think anybody who's here in the audience probably has built an agent as a side project or as part of production systems. It's a very hard thing to do. It's laborious. It takes a lot of time.

Uh, making it reliable is harder. So having it do things that you actually want it to do in production, understanding the different kind of things that it can play with, how it's going to react when you launch it to your end users, that's always a very hard thing to do, which is why evals are a pretty handy way to manage that.

Uh, yeah. And then, um, basically, uh, the first step when you're doing this is, of course, you need to have your agent foundation. So, um, when you're building, uh, your agent, you know, you will want to have a focused and strong set of LLM-friendly tools to give your agent a very good foundation.

Um, so yeah, I would say it's important to first optimize these tools and make sure they're the best they can be before just jumping onto, um, larger agent evals. Um, so once your tools are optimized, uh, you can also take some other steps like making an independent critique agent,right, with a remediation loop.

And this can fill more gaps as far as, um, having a self-correction mechanism and filling those gaps where, uh, maybe your base tool set, uh, has limitations. Uh, and then once your base structure is defined, uh, you can have, uh, you can then go to, um, having an eval.

And having a strong eval is very important as this gives you, um, like a way of proving the value of changes you make, as well as, uh, running ablation experiments on any changes you make. So I would say this is a very essential tool for climbing the quality ladder.

Uh, but again, it's very important to have that, uh, good foundation to begin with.

Um, so yeah, the, the reliability of your, uh, agent is basically a function of the capabilities of the agent, uh, the guardrails, and the evals. Um, so understanding, uh, what your agents do in the real world, uh, basically, uh, generative AI outputs, as I'm sure you're all familiar, are not exactly deterministic,right?

So it can, uh, often fail in certain areas or, uh, one time it can succeed, one time it can fail. So we can't really guarantee how it will behave in the wild. And for some use cases, this is extremely important,right?

We and we need a way to, uh, measure at scale and make sure that it is, uh, getting the output we want, uh, despite the nondeterminism of these models. So we need to define, um, what's good here. And evals allow us to basically understand and improve, um, how the models behave in the real world by defining what good looks like.

Uh, it's basically just setting this is, uh, you know, our target output. Uh, so to build evals that actually scale, they really need to be strict and measurable.

And, uh, so an interesting, uh, thing here that I think might be somewhat counterintuitive is that early on, vibing can actually be kind of good for you. Uh, and what I mean here by vibing is basically, um, doing things that are not exactly scalable to begin with.

Um, so when you're first, uh, starting out, it may be that, uh, you, you know, you could, uh, take a track of basically just going ahead and making the super comprehensive eval,right? Um, but we found it actually works better to first do intuition-based approach where you kind of, um, first see the capabilities and look at the outputs.

And at this stage, it's pretty easy to tell what the issues actually are,right? Uh, so even though this is non-scalable, it will still give you like a very good idea of when you change this, what happens. Um, and like, uh, it allows you to more quickly iterate as well.

So at this stage, prompt tweaks can also have like large performance gains. You can make a radical change to the architecture, um, and your eval is not kind of like hindering you in this way. So it's like a very good way kind of, you know, like an early stage company of just like first, you know, doing something, making more radical changes quickly.

Um, so yeah, this way, I think you can also get very familiar with what you're building, what the failure patterns are, and, uh, it gives you more of a, a sense of depth in understanding it, uh, which allows you to help climb in a targeted way.

And these will basically be very useful learnings when you're actually like building the more comprehensive eval. Uh, yeah, and then there's a chart here showing kind of, uh, you know, if, if you, uh, jump to scale to, um, these scaled raters like too early, uh, it can cause you to kind of have like very big ups and downs as you might be iterating and calibrating the eval as you are struggling with, uh, changing the model radically.

Um, so another thing here is, um, you can you should start early and start small. So, um, you don't as I said before, you don't need to have like a massive golden set on day one. Um, you can just kind of start with a few core tasks.

So you can look through your agent and define what are the primary things that you want to target,right? Um, and just, uh, basically start with those like high-level things and can slowly come to a more detailed representation as you move on.

Um, and so here it's important to also test the negatives. Checking if the model like didn't do something as bad, uh, something bad is just as critical as checking if it did the task.

Um, so yeah. And, uh, yeah, there's a funny visual here about, uh, writing the evals can be a very small point. And humans arguing over what the rubric should be is, uh, is kind of like a very large task here.

Uh, cool. So we've established that we want to start small and we want to wipe code at the start. Not wipe code, but like wipe eval at the start and try to get a sense of like what our model's doing good, where the agents are falling, find out all the patterns.

Eventually, you will get at a stage where you will try to involve more team. So if it's just a core team of like PM, NG, UX working at the start, then you will bring in more team. You'll have a bigger golden set, a bigger data set that you want to test out.

So you will think about scaled raters, LLM raters, all of that. So we'll get a little bit more into what that looks like. So just a couple of things on like working with scaled raters and things that worked for us.

Uh, one was that providing them with a clear rubric of what they were actually rating with very clear examples. So we had a lot of situations, especially early on when you're building things. Of course, like there's so many edge cases and difficult cases that we've not tested out that a rater might encounter.

So they're coming back to you saying, "Oh, what, what, what should I do in this case?" And then sometimes we as a team are like disagreeing on like, "Should this be a pass? Should this be a fail?" Things like that.

So I think that's very important to do early on. As much as clarity and examples you can give the raters, that would be super helpful. So yeah, to that point, like human-human agreement should be strong within your team of what you consider a good use case and a good pass case for an eval.

Uh, the second things that we noticed that helped us a lot was getting explanations from raters. So when you do have your teams or other scaled members rate evals, it's good to not just anchor on if it's a pass or a fail.

That doesn't really tell you much about where should the agent improve, what was the thinking that went behind coming to that conclusion. So it's helpful to get explanations of why they're rating something a certain way. And this is true for like if you do single-side evals or side-by-side evals, like when you're testing two models at the same time, having explanations of why one thing failed or one thing worked can be super helpful.

Uh, other things to keep in mind is that you could also do like in our case, it was multi-output. So we were asking scaled raters, um, when we were building ads, like, are these ads accurate? Like, did we do theright things for it?

Is it brand safe? Is it like something that we expected it to be? Things of that nature. So we had like almost like a multi-turn eval system. If you're building those kind of cases, it can get a little tricky because it's not exactly a pass/fail.

Your, your raters could be like, "Oh, well, it does very well in well in brand safety, but it does not do really good in like accuracy or things of that nature." So explanations really help you like get to the bottom of like where is it that the agent's actually like missing things.

And then you can also use that input to train your agent better. Okay. So now we talked a little bit more about involving cross-functional teams and human agents, but what about if you're using LLM raters, auto raters, LLM-as-judges?

So we ended up going down that path also. Few things that we tried to do to kind of set this set a more, uh, comprehensive structure for ourselves is one, we, we monitored disagreements or in some cases teams monitor agreements.

So basically, if you can have a sample pipeline of sorts that is monitoring how a human rater or some expert would rate an eval versus how an LLM would rate it, you can get a sense of like how it's trending and if the agreement rates are in the ballpark that you would expect it to be.

Uh, the second thing was we went a little bit beyond pass and fail. So we also looked at agent traces, which we'll get to later. But when we were doing pass/fail evals and trying to understand how things were getting rated, we did a couple of spot checks to understand the reasoning behind those logic.

So we could really see what was going on, how did it come to the conclusion that something was a final pass or no. Uh, and again, like just reinforcing the high-quality ground truth point that Daniel made earlier. So we want to give a golden set that's like super expansive.

It covers a broad range of use cases, and it also has very high human-human agreement within your team. Okay. So this is a quick example of what we had seen in the agent. I'll walk you folks through it.

So, uh, it basically says if you if you want to know what it's doing, look at it at its thinking. Uh, we had given the agent one of the prompts, and it was that for legal reasons, disclaimers can never be removed.

And we had mentioned that to the agent a couple of times in the prompt. We had like trained it on that, and it was all going fine. But then we started seeing that there were edge cases in which the agent was seeing the prompt, and it was seeing that there's a disclaimer present in the ad and then still removing it.

And we could not find that if we were just doing a categorical like the this X percent pass rate or not. So we really had to look at the traces to see what was going on. And in this example, you can see in the initial trace, it actually detects that there is a disclaimer in what it's searching for.

And it says, "Okay, I found a disclaimer, and now I'm going to go ahead and remove it," which was not what we asked it to do. Uh, this is a sample image that I created that I ran through the agent.

It says, "America, we can do better. It's a public parks ad." And if you'll see at the bottomright, it says, "Paid by the community of parks of Key Parks Clean." And we sent it to the agent, and it just removed it when we told it explicitly not to.

So those kind of things will happen. So it's really important to like check the reasoning and how it's like getting to the things that you care about. Uh, cool. One other thing. So like all ML systems, some things still are applicable, which is my favorite part.

So agents will not generalize very well depending on the kind of like data sets that you've trained it on. It's usually a good idea to have some sort of data set to test for like edge cases, like broader capabilities.

Uh, also very good practice to have a test set of sorts. If some of you have worked in like traditional ML systems, it was always good to have test validation sets of that sort. Same concept applies here. Uh, if you have a test set, use it sparingly and also refresh it with prod data.

Daniel.

Yes. Um, so then, um, yeah, this slide is basically, uh, talking about hill climbing and how it can be quite rewarding. So, um, yeah, you might find that, uh, after human eval,right, uh, you can, uh, this, this, this diagram basically shows like the, uh, kind of stack of what you can do.

So after, um, human eval, if it meets the bar, obviously you can finish there, but, um, it probably won't. And then, uh, when it doesn't meet the bar, uh, you can first review your eval set, uh, and find numbers like precision and recall.

Um, and then you can, um, iterate and basically, uh, you know, make changes to the eval or adjust your, uh, rating guide, things like this, uh, or just adjusting the model itself or the agent, uh, adjusting its cooling.

Um, and then through all of these things, you can both, uh, iterate on your eval as along with, um, iterating on the agent. And this loop, uh, once you have a, a very good eval defined, it works quite well for iterating on your agent, um, and doing a quality hill climbing, as I mentioned before, doing ablations and whatnot.

And then, um, so as far as, uh, launch readiness, you, uh, basically need to understand, um, regressions. So identify where and why the model performance is, uh, degrading so that you can distinguish between acceptable trade-offs and critical failures.

Um, so yeah, this is basically just saying like it's very important to understand from these evals,right, what is the exact issue that you're having and figure out, um, the trade-offs here. And then, um, you should also this is a very important point.

So you should focus on patterns rather than isolated runs. So a tempting thing is to hyper-fixate on very small examples from the model,right? So you might have, uh, one run that you do with your agent, and then you find, uh, it fails on this case, and you might think to yourself, "Okay, well, I should update the prompt based on my eval and, uh, the, the trace of the agent,"right?

But, um, if you do this, this is kind of a trap because, as I said before, these are non-deterministic systems. So what's more important is that you actually rely on patterns. So in your golden set, it's important to have multiple examples that can cover these kind of, uh, patterns that you might want to see.

And you basically want to look at the entire picture of how often is it failing on that pattern, not that specific individual example. Um, and yeah, it's important here also to invest in online evals, uh, and, and making sure that your data is matching the real-world representation.

Okay. Awesome. So we talked a bunch about like what worked for us while we were building evals. Of course, your mileage may vary depending on your application. Things can, uh, things can differ. Uh, some of the things that we wanted to recap here was, uh, it's like what, what we think makes a good eval system generally is that it should be representative of what you want your product to be great at.

And that will differ depending on the state at which your agent is. When you're building MVP cases, it would look differently versus when you're doing production rollout. So that would differ. Uh, but it still needs to be very much, uh, centered around what do you want your product to be good at and optimizing for that.

Uh, important to, of course, keep it evolving. That's why we talked about having your online evals, having test sets that are refreshed with production data, having sampling pipelines, all sorts of things. Uh, highly curated golden sets, which will also evolve as your use cases evolve.

So training teams, whether it's scale raters or your cross-functional teams on how to rate things, how what are you expecting out of them, that's also very important. That's, uh, I think now it's getting more mainstream, so hopefully it's less, less controversial.

But like six months ago, our teams were like still figuring out, "Okay, how do we like do this? What's expected out of it?" So I think investing in those trainings can be helpful. Uh, and then rater templates and rubrics with clear set of examples.

So you don't have scale raters coming back to you saying, "I'm not sure how to rate this." Um, lots of like things getting marked as like unknowns or I don't know, things like that. Also choosing theright launch metrics.

So some of the launch metrics, launch slides that we showed you, this is a very high level of generally how you would do a launch readiness. You would like check it, you'll do a bunch of iterations on the model, you'll do an A/B diff or ablation, and you'll try to see, "Okay, where is the regression happening?

What's an acceptable regression versus not?" Things like that. As you're doing these systems, it's important to like, uh, get some clarity early on on what is your gatekeeping rule. Like what's your launch criteria? Is there a certain precision recall number that you're looking at?

Is there some other metric that you look at? If you're doing a model eval, then probably that metric looks different than just the usual precision recall. So those things can also be important to keep in mind. Uh, yeah, those are all the tips that we have to build production-grade evals.

Thank you.

We're at time.

Awesome. Thank you very much. Uh, do we have time for questions, staff? One? Do we have time for questions? Just one. Allright. You went up first, sir. Go ahead.

Are, uh, all your eval judgments being performed by humans, or are you also using LLM as a judge? Um, and if so, what's your calibration process look like for calibrating that judge to provide good evaluations?

Yeah, I think that's a good question. I think I wouldn't say all. I think it depends on various use cases, like depending on like what kind of systems you're trying to build. We have, of course, like a plethora of use cases.

So I won't say all. I kind of go into details about what the benchmarking and all of that system looks like, but some of the things that we talked about in terms of disagreement rates and monitoring like sampling pipelines, those things hold true generally.

Yeah. Happy to chat more offline. Yeah.

Awesome. Great. Thank you very much. Uh, Daniel and Preetika, please give them a round of applause. You can always ask them questions in the back. Thank you.