fighting slop with slop — Vaibhav Gupta, Boundary

Fighting slop with slop. My name is Vaibhav, and I'm going to talk about something that is a little, I would say, maybe a little silly at first. I'm going to show you our team's engineering practices really quickly. We do no code reviews, we require every engineer to work on things in parallel, and we have no standardization on how people do AI.

And I know immediately what almost all of you are thinking: we're probably a Zoomer YC startup, and I can guarantee you I'm clearly a millennial. So what do we actually do at our company without code reviews? Well, we about 3 years ago, we decided to build a programming language.

That's something that has absolutely no room for slop. It's something that has to work every single time exactly the same way. It's something that you can't just change a year later or a month later because you made a bad design decision.

You have to be correct. And for the last 3 years, we've been in an onslaught of war against slot. And when I first met this enemy, I went to my great mentor, Slopsu, and he taught me something. To defeat the slop, we must become the slop.

So we began, and we prepared, and then we started winning. So when we think about it, what is slop? Slop is just any code you don't read. And whether any of you admit it or not, this is the least amount of slop that your code base will ever have.

Cherish it. So we started fighting back against the slop, and we started fighting back with slop. So how do we go ship a stable programming language with these engineering practices? Well, the first skirmish we ever had was the skirmish of standards.

The hard part about hiring great engineers is you sadly can't tell them what to do. Some of them want to use Claude, some of them want to use Codex, some of them want to use the latest thing that they just found on Hacker News.

So instead of trying to hold standards in our code base, we did something that is an invariant. We built an architecture.md file. Instead of using Claude MD, just pick something that every model can just understand. This file has to be incredibly small, and it can only have things that will not change for months or for years.

In our case, it's the layers of the compiler. You go deeper into the compiler, tell the agent to just talk to at least one other person. That slows it down a little bit. So now we have standards so anyone can use whatever they want.

But the real foe we faced was actually the battle of design. Everyone here knows that you have to write perfect design docs. And we have a very simple rule in our team: code can be slop, writing cannot. And of course, if I tell every engineer this, they write beautiful writing, and they handwrite everything they don't use AI.

Well, sadly not. So we built a design tool, Design Doc Tool. What this Design Doc Tool does, it's a replacement for both Notion and GitHub, effectively for design docs. It allows versioning, commenting, all the stuff you want. And obviously, we do this, people use this.

Well, sadly not. We built another tool on top of that. And this tool was a Slack integration for that tool. Every time a design doc got updated, this channel got notifications. And what ended up happening is this channel became the most popular channel in our company really fast.

At 2:00 a.m., someone shipped a new design doc. Three people started reading itright away because it's just interesting. The most interesting stuff is design docs that are not going to change. But this wasn't enough. All of this is actually backed by Markdown files and simple CLI scripts that make it treat like GitHub without being GitHub itself.

So now agents can go do this. But the real problem with all this is I built this, and then I hit a little bit of AI psychosis, and I started shipping 10 design docs a day. And soon, the team was fighting my slop.

So we had to go and add the last rule. This last rule was, if you're going to ship a design doc, you require people to actually go read it. And with this last standard, we suddenly had design docs that were incredibly high quality.

But what about the battle of architecture? How do you have your code base converge? We built another tool. This tool basically visualizes our dependency graph internally with some external dependencies as well. It allows us to watch the code base change.

It has semantic boundaries, individual packages. But what's more interesting is we can go build CLI tools that guarantee that certain invariants can't be broken. And what this does is when Claude builds a new package or adds a dependency that's leaky, we now have CI/CD changing or a simple Git commit history that tells us exactly where things break.

And by this, we're actually able to make our architecture change. We haven't changed our architecture in the last 3 or 4 months.

But as much as we might do design docs and as much as we might have stable code, would you genuinely ship code without reading it? Would you trust your team to go do that? And think about a programming language.

A programming language has so many invariants. You have generics, you have closures, you have memory allocation, you have FFI boundaries. Could you trust that system? Python has bugs 25 years later. Well, here's where we did something slightly different.

What we did was we built a system that actually has agents constantly running and creating BAML programs. We take these BAML programs one second, and we have agents try and spin something up from scratch. We then look at the entire Claude transcript, see what tools it used, see what happened.

And obviously, we as humans can inspect them. But more importantly, we can have agents go inspect them. And agents find what was good, what was bad. And not just what was bad in terms of what was incorrect in the language, but what took three tool calls when it should have only taken one.

And then we can go ahead and find issues. And we can have humans collaborate with these issues to figure out which ones are real, which ones are hallucinations, which ones don't have taste, as much as I hate to use that word.

And then we can have agents go ahead and create fixes to these problems and go address them. And most importantly, instead of trying to just detect these issues, we can go one step further. What if you could find language features?

And instead of guessing what was good, guessing what skill was good, you could go and A/B test it. You could figure out which ones took less tool calls, which one made less errors, which one produced the correct outcome, and deterministically know what's going on.

The point is, you can start building data-driven systems without ever writing a single line of code. And the thing that really I care about the most over here is not that any one of these tools is specifically what you should go build.

But the fact of the matter is, in order to build a programming language, it wouldn't have taken eight people. It wouldn't have taken less than two years. It would have taken hundreds and thousands and tens of thousands of man-hours, and then you would still have a broken system.

And today, we can just spend billions of tokens and make it work, and we can make it stable. And you too can go home and build these internal tools and these sloppy tools to make sure that your code bases can ship without really having to read necessarily every single line of code.

Because your engineers aren't going to. And I think we can start winning this battle against slop. And as we win this battle, slop can be defeated. But sadly, I have a sad thing to say. I think we're still going to lose the war.

And I think the reason that we're going to lose this war is because some of the foundational stuff that we try and go use itself is broken. How many of you have used TypeScript? Probably most of you, hopefully, at this point, or at least your agents have, so something around there.

Did you know that TypeScript's main design goal is to strike a balance between correctness and productivity? And there's an asterisk here because what they really mean is human productivity. And if you think about it, there are things you would never do in a programming language at the very core layer if you were designing in a world where humans never wrote a single line of code.

Let me show you what that really means.

I'm going to write something and try and guess what this code does.

Pretty safe.

What about this one?

Or even more so, this one. Why do we turn things to strings when we sort them? This is just slop baked into the language, whether you like it or not. What about this? I love this part of TypeScript.

And you know what my agent loves? This part of TypeScript.

This is slop baked into the language. And whether you like it or not, the systems will have slop if you build using these tools. Oh, I'm sorry. Wrong talk. But if you think about what JavaScript does, JavaScript exists.

And then after JavaScript existed, we started building systems to layer it on. We built CoffeeScript, then TypeScript, and now we're trying to build Effect. But the thing is, the thing underneath is already broken. And more so, the way we write code is also different now.

So why are we trying to patch something like this? Why don't we just try and do something a little different? And I think what we might need if we try and go do that is basically going to be a made-up language.

So let me show you what BAML really can do. And when you start thinking from first principles, how you can try and combat slop from a very foundational layer itself. I keep talking about not reading code. Does it even matter?

Well, let me show you a new way to think about code. And this isn't to say we all have to go do thisright away. But what if every single time I looked at code whoops.

What if every single time I looked at code, what I really saw was not the code itself, but a quick little thing that could actually visualize all the code for me? As I clicked around, it took me to exact lines of code that I was linked to.

If I wanted to have a slightly broader view, I could zoom in and click around and have it expand. And I could navigate my code bases more interestingly. I'm going to let this run really quickly. But while it runs, I'll show you a different pipeline.

Without any of you ever reading the code, you know I'm setting up stuff, and I have an agent loop because the semantic boundary is in there. I can expand this. I can keep expanding this. And I can say, nope, that's too much slop.

Let's let that be slop and walk away. So instead of having to understand all the code, I can opt into what parts of the code I want to read and understand and go to the exact lines when I really care about them.

But if we go back to the previous pipeline that was running, what if while it's running, I can actually get a full execution trace? In a world where we don't read all the code, the only way to understand the code is actually by the execution trace and actually by seeing exactly how much time was spent on what parts of my program at any given time.

If you want to go and actually track your program through, think about how slow your program would be if you had to go trace everything in Python or TypeScript. It's untenable. And the best part here is, if you start from first principles, you can make this effectively zero performance cost.

Not only can we make it great for humans, but because it's all built for agents anyway, you can go ahead and make it so that every single file has a tracing system that Claude can navigate through. So Claude can find what were bugs, what were errors, and what were inefficiencies and start optimizing your code without you having to do it yourself.

And I think if we go start thinking about it from this way, it's not so much about reading all the code, but it's more so about, as a human, understanding the system that you're working with. And the tools that you can build can give you information about the system that you're working with.

But I think there's another layer to it. We've spent decades building IDE tooling. And think about how long it took before someone like me, who does not know how to escape Vim to this date, can finally start using VS Code.

It was a beautiful day when that happened. I became a real programmer. Well, according to some people. I'm still not because I can't write Vim code. But what does agent-first tooling look like?

I think we're all familiar with Grep, so I'm not going to go and talk about it. But I will talk about RIP Grep because Grep should not be used anywhere. If I wanted to Grep through my code base and understand what it was, I would RIP Grep say something like calculate, and it would give me a bunch of code where everything was being used.

And maybe it would be somewhat useful. But what if you could instead just start describing code and say, can you describe calculate for me? What if it came with all the doc strings? What if it came with the actual source code?

And what if it also told you everywhere it was actually used under the hood? We can make something that used to be multiple tool calls a single tool call all of a sudden. What if the way you wanted to learn about libraries that you were using, instead of having to do a web search,

you could just ask for any external library as well, and it would just give it to you? Because when I first started learning how to code, one valuable lesson I had was the code is always a source of truth.

Don't read anything but the code itself. The docs may lie. The actual description or architecture file or README file will definitely lie. But the code cannot lie, except if you're working on some weird architectures. And then when you go down this road, you go from not reading the code to understand the architecture.

You go from not searching the code to understanding exactly what you're getting in every one tool call. But what's the next thing you do? Well, the last thing I do to truly understand code is I run the code.

So what if every single thing you ran, every single function you ever wrote, was immediately available? And I'll pop this code over here

was immediately available as a simple CLI command. So if I run add, add becomes a CLI command that has A and B parameters attached to it. And I can just run it really quickly and see what happens. What if every single CLI tool I had could be packed

into a CLI that's completely standalone? Multiply that I can just run without ever having to actually execute any of the code.

And it's now a total CLI binary that has a function just bundled in. Suddenly, we can build really quick tooling where agents don't have to go Grep through what's happening. Everything is type safe. Everything is deterministic. And everything is actually guessable.

And the best part is, imagine you could build on any system, and your agents don't have to worry about deployments across Windows, Mac, and Linux. And you can just target any layer you want, and it builds for any architecture, including Wasm systems.

All of a sudden, as an engineer, you're supercharged. You're no longer bottlenecked by what you can do in the systems underneath you are preventing. You can just move very fast. You can move at agent speed. But a lot of the stuff that I've been talking about to this day has been about tooling.

What if we tried to fix some of the real sins of JavaScript? Some of the stuff that is deep in the language, not the sort stuff, but I mean more important stuff like errors. Have you seen error handling be beautiful ever, other than Rust?

No.

What I see agents do over here is you do try-catch. And then they keep nesting try-catch after try-catch after try-catch. And eventually, they give up and say, console.log, some error happened, and deal with it. What if we could do error handling from very first principles?

What happens in that world?

Well, I showed you add, multiply, subtract. I didn't show you divide. Divide is dangerous. It's spooky. So let's go look at divide. You can see over here, divide throws a division by zero error. But what else happens? The function actually knows that it throws division by zero error without you having to write any code.

If I go up to the calculate function, which at some point calls divide, this function also knows it throws division by zero error. So error types now get inferred without you ever having to do any guesswork. That means if you catch or handle errors, we can do exhaustive guarantees.

And the compiler can prove that you have handled the error or not handled the error. It's no more guessing. There's no unknowns. It's guaranteed to be proven. So if you wanted to ship an API

where it guarantees that it never throws, well, this system is broken because it doesn't meet the constraints. It has two errors that you're not throwing. If you wanted to go catch that, well, I can write the code for that in a second.

But you can start catching certain errors.

I'm just going to return a sentinel value for now. And now this parse thing, which previously threw division by zero error, is now guaranteed to no longer throw the division by zero error because if I catch any exceptions in here, I return a zero value every single time.

The compiler and the tooling can do a lot of work for us. And we're already used to this in our code bases. Many of us probably don't know how compilers work under the hood, and we trust them. Code is a matter of trust.

The reason that we don't use LLM code blindly is because we don't trust it yet because the systems underneath them don't have enough rigidity.

One more thing. But before I tell you all to go write a bunch of BAML code because I've been there, and I can tell you what someone would tell me if I said, hey, use this new programming language.

It's going to solve all your problems. It's just going to come with a whole slew of new problems. So we said, I think we'll lose the war on slop if we try to ask everyone to rewrite all their code in the world into this new system.

So what does a solution like that look like where you don't have to rewrite all your code? Well, what we started to do was we started to think about that about two years ago. And we said, what if you could use BAML not just standalone like I showed today, but from within any existing language of your choice, from Python to TypeScript to Rust to Go to Ruby to Java to anything new that comes up even after it?

What if every function in BAML is immediately accessible in the language of your choice? So in this case, I'm calling the BAML calculate function directly from Python, and it's completely type safe. Not only do we get calculate, we get calculate async in case some of us want to write async code.

So BAML, while it has no function coloring, it does give you the benefit of having to do whatever you want across your code. But what if you went a little bit sillier? What if you started passing around lambdas across language boundaries?

I have a function here called width timeout. This function times out after a certain number of milliseconds if this work doesn't complete. And it's guaranteed to no matter how long it takes. Well, in that world, you can even pass Python lambdas across the bridge.

You can pass generics across the bridge. You can pass closures. It should just work so engineers don't have to go fuss with it. And more importantly, so when the agent does something, the type system never lies. The type system becomes the absolute center of truth that prevents invariants from entering your code base.

And what I really wanted to talk about today was not any one specific thing, but it's this general concept. You can build incredibly complex systems without traditional systems like code reviews. You can work on things in parallel, and you can use AI however you want without requiring any sort of standardization.

But the most important part is you have to be incredibly thoughtful about how your engineering team actually uses the systems under the hood. When we started building BAML, I didn't think it would be possible to build some of the software we did.

And just yesterday, one of our engineers built a partial C compiler purely in BAML. So when I start pushing the boundaries of these systems and you stop reading the code, in some ways, in my mind, it releases a floodgate for your engineering team to actually cover the gaps of what existed in your old process.

Have you ever worked at a company that had no CI/CD? They said adding CI/CD would slow us down. They do slow down for three months while they add it. But after that, they move a lot faster. Our processes have to evolve if we're going to ship at agent speed.

And remember, this is the least amount of slop your code base will ever have to this day. So just embrace it and start fighting it back. I fell in love with software about 15 years ago, and it was the first thing that truly changed the way I perceive the world.

And I really, genuinely don't want slop to win. And I think we can all build a world of beautiful software. And I think what it takes is I want each of you to go home today and build these sloppy tools, make your systems more robust, make your processes more robust.

And then for the bravest of you, I want you to go back and think about these core foundation layer systems. Think about how they're broken and see if you can imagine a way to fix them. I think we do need a new Git.

I think we do need a new database. And yes, I think we need a new programming language. I'm Vaibhav, and I work on BAML. Thank you.