# Prototyping as Leadership: How a CTO Ships with AI Agents — Hursh Agrawal, The Browser Company AI Engineer · 2026-08-20 Hursh Agrawal, CTO and co-founder of The Browser Company, argues AI coding agents let leaders keep building: despite 15+ meetings, 7 direct reports, and a toddler, he ships 2–10 PRs/week. He says frontier models turn over every three months, so hands-on building is the only way to judge them and show engineers working prototypes. His method is an overnight loop: a coworker agent gathers Slack/Jira/Notion context into a prompt at 5pm, coding agent runs for hours, and a morning hour reviews tests, CI, and AI code review. He details three overnight uses: building features, hill-climbing evals from feedback JSONs, and training custom models like a PII classifier on AWS. He warns leaders to avoid critical path work and to rely on trustworthy CI, feature flags, a prototype branch, and readable PRs. ## Questions this episode answers ### How does a CTO with a full manager schedule still ship 2 to 10 PRs a week? Hursh Agrawal says AI agents turned the manager schedule into building time. He has 15 plus recurring meetings a week and 7 direct reports, plus a toddler at home, yet consistently ships 2 to 10 PRs a week. He fits in a morning coding block, steering blocks, and a 5pm block for overnight runs. [0:48](https://aiengineer.podhood.com/fa037856-6b96-438e-976b-2ef064007404?t=48000) ### What is Hursh Agrawal's overnight loop for using coding agents? At 5pm he gathers context, often using a cowork agent linked to Slack and Jira, and writes a detailed prompt for Claude Code or another coding agent. The agent works four, six, eight hours overnight, writing tests first, verifying with computer use, managing CI, and leaving a morning report. He reviews the results in a morning coding block. [6:28](https://aiengineer.podhood.com/fa037856-6b96-438e-976b-2ef064007404?t=388000) ### What four types of work should technical leaders build with AI, according to Julie Zhou? Hursh Agrawal cites Julie Zhou's poll of Bay Area technical leaders: leaders can build internal tools, artifacts to celebrate people on the team, and vision prototypes that push product boundaries. He agrees leaders should not take critical path work, because meetings and fires make depending on them risky. Building these demonstrates what new model families make possible. [4:52](https://aiengineer.podhood.com/fa037856-6b96-438e-976b-2ef064007404?t=292000) ### Can AI coding agents train other machine learning models overnight? Yes, Hursh Agrawal describes training a PII classifier overnight. Opus and Haiku were expensive and had poor precision-recall, so he collected training data and had Claude clean it and generate synthetic data. He gave the agent OpenAI and Anthropic keys, AWS access in a sandbox, and instructed it to train two models, test them, deprovision resources, and report back in the morning. [13:23](https://aiengineer.podhood.com/fa037856-6b96-438e-976b-2ef064007404?t=803000) ## Key moments - **[0:00] Intro** - [0:48] Hursh Agrawal ships 2-10 PRs a week with 15 recurring meetings, 7 direct reports, and a toddler at home - **[1:49] Judging models** - [2:30] Hursh Agrawal: frontier model capabilities change every three months, so leaders must use them hands-on to cut through Twitter noise - **[3:03] Show don't tell** - [3:41] Hursh Agrawal: leaders' agent prompts are more impactful per token than ICs' because they hold the most business context - **[4:52] What to build** - [4:52] Q: What should technical leaders build? Hursh Agrawal cites Julie Zhou: internal tools, celebration artifacts, vision prototypes, not critical-path work - **[5:50] Daily loop** - [6:23] Hursh Agrawal's overnight loop: a 5pm steering block sets up agents to work 4-8 hours, then he reviews the package each morning - **[6:50] Feature overnight** - [8:44] Hursh Agrawal prompts overnight agents to write tests first, verify with computer use, keep PRs reviewer-friendly and CI green - **[8:55] Verification** - [10:44] Hursh Agrawal: new models like Opus 4.8 or GPT can handle in one overnight run what used to be weeks of work - **[11:02] Hill climbing** - [12:54] Hursh Agrawal turns feedback-button JSON dumps into eval sets and has agents hill-climb prompts overnight to optimize AI features - **[13:23] Training models** - **[14:54] Task scope** - [15:10] Hursh Agrawal quoting Simon Last: 'These modern models are just capable of so much more than we think.' - **[15:35] Caveats** - [16:18] Hursh Agrawal admits his agent-generated prototype code has caused sevs and annoyed engineers, so trustworthy CI and hygiene are prerequisites - **[16:35] Hygiene tips** - [17:23] Hursh Agrawal says don't add reviewers to code you haven't read: a senior engineer once caught his obviously bad unread code ## Speakers - **Hursh Agrawal** (guest) ## Topics Coding Agents, Developer Productivity, AI Strategy ## Mentioned AWS (company), Anthropic (company), OpenAI (company), The Browser Company (company), ARC (product), Claude Code (product), Codex (product), Confluence (product), Cursor (product), Dia (product), Haiku (product), Jira (product), Notion (product), Opus (product), Slack (product) ## Transcript ### Intro **Hursh Agrawal** [0:14] Hi everyone, thanks for coming. I'm Hursh Agrawal, I'm the CTO and co-founder of The Browser Company, makers of the ARC and Dia browsers, and I'm going to talk about prototyping as leadership as you get to a manager schedule, how you can keep building. So, this is my actual calendar from last week. I imagine this is familiar to some of you in leadership. Um, that was my actual week. I have a whole org that reports up into me, so I have 15 plus recurring meetings a week: standups, reviews, recruiting meetings, etc., and 7 direct reports. And I've noticed something over the last several months, which is I've started to consistently ship, you know, 2 to 10 PRs a week. And this is new. This was not possible several months ago. It's really because of these new AI agents. And notably, I also have a toddler at home, so I, like, cannot work 9-9-6. You know, I'm working 40, 50 hours a week, so I really have to fit all this in into a regular week. And pre-AI agents, as you grew as a leader, you had more of the org reporting to you. You would sort of, the way you would influence the org was through communicating to people. So you would write roadmaps, docs, meetings, and you'd sort of incept your context and what you wanted to build with your employees and your engineers. And now, interestingly, in the last few months, as coding agents have become more autonomous and able to handle longer tasks, the manager schedule, as Paul Graham put it, is suddenly usable as building time. You can actually ship stuff. And so I think building is part of the job now. You can do it, first of all, as a leader, but it's, I think it's also becoming necessary as part of the job for two reasons. First, the technology world is changing where suddenly the core technology that's part of our products are these frontier models that change every three months, which is a new dynamic that's come out. ### Judging models **Hursh Agrawal** [2:12] And as each new frontier model comes out, its capabilities change, the contours of what it's useful for change, it's, you know, how it actually reacts to prompting changes. And there's so much noise on Twitter, even internally for us, and so many opinions with each new model release on what's good and what's bad. And I found it is impossible to tell what a new model is good for unless you have your hands in it and you're using it all day long. And so having an ability to slot in building time into your schedule means you can figure out the contours of what is this new model family capable of, and both how do I direct my engineers in terms of setting expectations on how they should be building, but also getting a sense of how does this fit into our product, how does this fit into our business, you know, how is this, how is our strategy going to change because of this, what's going to happen in three to six months when the open source models catch up. All of that intuition comes from actually using the models and building. I've also found, even if you've built the intuition, it's actually tough to communicate that to other people who haven't played with the models. And so you're like, "Oh, this new thing is possible. ### Show don't tell **Hursh Agrawal** [3:17] It's going to be amazing." And your engineer's like, "Okay, yeah, sure." So it's really helpful to have some time to actually build stuff so you can show them. You know, you can be like, "Hey, I built this prototype with this new model family. It works in the product. Here's an actual prototype you can play with." And this is just so much faster and more efficient than trying to convince people every three months when a new model family comes out. I also think leaders are really well suited for it. You know, you all have context, more context than anybody else in the organization about the business, the strategy, the trade-offs, the decisions to be made, what needs to be done, what's, what's coming up, what are imperatives. And so the steering of a leader as you are prompting these models is per token more impactful than an ICs. And so you can really fit this into a little bit of time. The delegation skill also transfers. Like, you delegate to people. That's sort of one of the core skills of being a leader, and that transfers to agents pretty well. You know, it's setting goals, giving context, checking in, even coaching the agent, you know, "What if you tried this?" And then I've also found, I think this is maybe specific to now, maybe this will change in several months as these models get better, but the models are really good at execution, but still not unbelievable at judgment. You know, often the model will come back and be like, "Hey, that algorithm idea is not possible," or like, "I can't do this." And so you have to, as a leader, have to be like, "Oh, have you tried this thing?" And the model's like, "Oh, okay, cool, cool, cool. I'll try that." And so it's a really nice dynamic you can have with these models if you have time slotted out for building that really leverages all the context you have. ### What to build **Hursh Agrawal** [4:52] So the great Julie Zhou had a great tweet about this a month ago. She polled some Bay Area technical leaders on what can you actually build, what should you be building in this new world. And four categories emerge. You could build internal tools, so internal efficiency, quality of life improvements, just like gardening around the code base and the product. This is really helpful. I really like this. I learned from this, the celebration story. You can build artifacts to celebrate people on your team. And then arguably, I think the most important is the vision piece. It's really playing with the new model families, understanding what's possible viscerally yourself, and then trying to fit that into the business and building products that can show that off to your engineers so you can really push the boundaries of what's possible with the product and business. She'sright. I would not take any critical path work. The last thing you want to do is to have that dependent on you, especially you're going to be dragged into fires and recruiting calls and meetings, etc. So really trying to do one of these four categories is optimal. ### Daily loop **Hursh Agrawal** [5:50] So this is, this is my, how I work and what two to three hours a day of coding can actually look like. So I usually have a morning coding block about an hour, and this is when I look over last night's code, what the agent did last night, review it. I'll talk a little bit more about this. And then a few maybe steering blocks throughout the day interspersed between one-on-ones and reviews and meetings and everything else you need to do. And then the most important block is that 5pm block at the end of the day where you really set up whatever overnight run it is you want to run. And I'll talk about that, whether that's coding or training models or whatever you want the agent to do. I'll go over sort of three examples of tasks you can do, but really what I found worked for me is this one overnight loop. You first, in that 5pm block, gather context. I'll talk about that. You set up the run, you know, you answer any clarifying questions, and then Claude Code or whatever your coding agent is overnight does the thing, you know, does the work for four, six, eight hours. And in the morning, you get this beautiful report and you figure out what to do with it, and then you ship whatever it is. So I'll talk about building features, which is sort of the most obvious one. I found this worked really well with evals and hill climbing, so optimizing AI features. ### Feature overnight **Hursh Agrawal** [7:03] And then a new thing, actually, in the maybe more recent generations of models is you can have these models train other ML models overnight too, and this works really, really well. So building features. The big mindset shift for me that really helped was starting to think about building features not in terms of how do I build this feature and break it up and then give it prompting on how to do the individual tasks, but rather what is all the context this frontier model needs to be able to make decisions like I would make. So trying to give it as much context as possible, because if it's working for six hours, eight hours overnight, you're not going to be there to steer it. And so you want it to have as much context as you do about the business goals of whatever you're trying to get it to do. So a little tip I found that has worked well for me, if you have a cowork agent, whatever your Cloud Cowork or Codex or I recommend Dia, I feel like that's the best one, not biased at all. Whatever it is that's connected to your Slack and your Jira, Confluence, Notion, the repo, etc., just I, before a meeting or something, I'll be like, at like 3, 4pm, I'll be like, "Hey, I really want to build this. Go and do like 20 minutes of research and go dig through all of Slack and Notion and everything else and come up with a Claude Code prompt that I can post and paste into Claude Code. And just give me as much context as possible with the trade-offs, what we tried before, what works, what doesn't, why we're doing it, what is like the business context of this feature I want to build or this model I want to train." And that takes like 30 seconds to whisper flow into Dia or whatever your cowork agent is. And the nice thing is this comes back with a giant prompt that then you can paste into Claude Code or Cursor or Codex or whatever your coding agent is. So this is an example of a 5pm prompt I'll have that I give it before I go to bed. I'll say, "Implement this whole feature," and then just paste all of that context I got from my cowork agent. ### Verification **Hursh Agrawal** [8:55] And then it's important to think about verification. So as this agent is running overnight, you want it to verify how it's doing as it's building and testing the feature. So first I'll be like, "Hey, write the tests first so they capture what you do." This is really helpful because I've found with the coding agents, if you, if they write tests afterwards, they're a little sloppish. And then I'll tell it to test the end-to-end flow with computer use. You know, go around, click around, just make sure the flow works and matches the business context I gave you. And so the modern models are really good at reasoning about, "Okay, does this flow actually solve the problem I was trying to solve?" And then I usually prompt it to do as much work as possible so that when I wake up in the morning, the stack is ready. So split this into reviewer-friendly PRs with clear descriptions, manage CI and get CI green, and just make sure you're monitoring just to make sure the whole stack is passable. AI code review is really important. We have a bunch of internal AI code reviewers. You know, the previous talk was talking about Kodo, another AI code reviewer. If you don't have any, I would look on Twitter for a bunch of AI code review skills and then just prompt it, "Hey, once you're done, run this AI code review skill in a clean sub-agent and then fix those things." And then watch the PRs. Fix every bot comment, every, you know, CI check that shows up, any sort of, anything you need to do, resolve the threads, run this autonomously, don't ask me questions. And then I'll, like, throw in a little encouraging something, you know, be like, "You'll do great. I believe in you. It's going to be great." I don't know if that helps or not, but it's, it's, I tend to do that. And then I tell it, "I'm going to bed. Just leave me a report in the morning on what you do. I need the full stack ready and a report on what trade-offs you made, how you did." And it does great. It actually, modern models, new Opus 4.8 or the new GPT, they can handle what used to be, you know, weeks of work in one overnight run, and you come back in the morning with this beautiful package ready for you. It actually makes the mornings kind of nice because you have this little present ready for you. Another example is optimizing AI features. So first, you're at the AI engineering conference. I assume you all are familiar with this. When you first build an LLM call or agent into a feature, it's not optimized. You know, you need to optimize it hill climb on some evals to get the quality, the latency, the cost to where you want it to be. This is also all doable overnight, especially with the modern coding agents. So usually if I'm prototyping a feature, I will put a little feedback button on it. ### Hill climbing **Hursh Agrawal** [11:18] So after the LLM call or the agent runs, I'll be like, "Hey, just add a feedback button and a little text box." And then in one of my one-hour slots or 30-minute slots throughout the day, I'll just take the time to collect a bunch of feedback. So I'll, you know, run it a few times, collect a bit of feedback, be like, "Hey, this was bad or this was good or this could have been better," and then just have it save JSON dumps on my downloads folder. So it'll save a dump of each of the runs with the system prompt and the inputs and my feedback. And you just collect a handful, you know, like even four, five, ten are fine. If you can get some other coworkers to run it, that would be great. Collect like 20 or 30. And then similarly, at 5pm when you're setting up an overnight run, just say, "Hey, like, here are, you know, 10, 20, 30 feedback JSONs. This is all the information they have in them. Turn this into an eval set. Like, just do it locally with SQLite or Markdown or whatever you want. And then over the next few minutes, talk me through how would you design evals or scoring functions to optimize for this. And let's do it interactively." And then I want you to, A, build a harness that runs this call against the evals and hill climbs until the score goes up. Like, just do whatever you need to do to get these scores to go up. And obviously with this kind of thing, if you have such little data, overfitting is a problem. But even that, I found these modern models, you can just tell it to not overfit and it actually does a pretty good job. And so just say, "Don't overfit. Just keep it general. Run overnight autonomously until we align on a scoring rubric and then give me a full report in the morning." And then a little tip, it's also nice to just say, "Hey, save this flow as a generic skill so I can reuse it." And that allows you to improve the flow over time and improve that skill. And this works. Actually, in the morning you wake up to ideally something hill climbed. And actually when using that feature, when we shipped these features to employees to further dogfooding to production, it actually does improve the quality. You know, these are not overfit. And we use this flow quite a bit internally. We have an internal code reviewer similar to some of the external ones. And same thing there. We had that hill climb to pretty good quality just with these overnight runs. ### Training models **Hursh Agrawal** [13:23] And then the last example, and this is again pretty new and really interesting. You can train custom models with this mechanism overnight, you know, in one night. So this is an example of a modern-birth PII classifier we trained. We were trying Opus and Haiku for it, and it was expensive and latency was not great, and we just couldn't get the precision recall to be amazing. And so we collected a bunch of training data. And then overnight, one night, I was like, "Here is a bunch of business context and a bunch of training data we collected. A, clean up the training data, Claude, just bolster it with synthetic data, whatever you need to. Here's a bunch of OpenAI and Anthropic keys. Use an ensemble of frontier models to push the quality." I don't know anything about what ML model will be good here. So like, you decide, you know, just give me the best model class, give me a few options. In fact, train two. Like, don't even train one, train two separate ones. And then I gave it AWS access. I was like, "Give me a provision sandbox. Do not give it prod. That's not a good idea. That's how you take prod down. Pick theright GPU and EC2 cluster. Like, train it, test against eval examples, deprovision it, whatever you need to do. Just give me back the trained models and then give me a report in the morning." And in fact, even put in how I would host this with my codebase on inference once this model's ready. And again, ask me any clarifying questions. You can do it. I believe in you. You're going to do great. And in the morning, just have this ready. And of course, it works beautifully. In the morning, you have two ML models trained, a full report, great results. And we've pushed a bunch of these to production. So those are three examples. ### Task scope **Hursh Agrawal** [14:54] I would try, try your own. But I think the main takeaway for me in doing these is really to push on task scope. So this was a tweet by Simon Last, one of the Notion founders. And he, he called out, "These modern models are just capable of so much more than we think." And I think it's our jobs as leaders to really understand the contours of how much we can push these models because it affects both how we lead and how we influence our teams, but also how they fit into our products and what these new products are capable of in terms of utility. So I'd really push you overnight. Try to think of how do I give it more and more scope so that we can do, you know, weeks of work, months of work overnight and see what these models are really capable of. I will say a caveat. A lot of this works because of existing organizational scaffolding. So you'll have to build this up in your organization. You know, AI code reviewers, this really helps. We have an internal one we trained, but also just external ones, whatever you need. ### Caveats **Hursh Agrawal** [15:49] Agents.md hygiene, you know, Claude.md, agents.md, proper CI that you can trust. And then we have a bunch of other things like really sophisticated feature flags. We have a prototype branch that you can push to that goes to employees, but it doesn't go to production. Just layers like that. So you're not taking prod down basically as you're prototyping and showing these things off. Not amazing for the CTO to take prod down. I will say again, code hygiene really matters. It really matters. I've been humbled a lot. You know, my code has annoyed engineers. It's caused sevs. Coding agents are not perfect yet. So just your mileage may vary. You will get humbled, but I still think it's worth it because you learn so much and you can model what taking on more and more scope, even with a busy calendar, looks like. Some tactical tips on hygiene. I would test everything before the PR goes up. That morning slot, that's a lot of what I'm doing is just testing what the overnight run did. Even though there's computer use, etc., it's really important you test it yourself. ### Hygiene tips **Hursh Agrawal** [16:48] Small readable PRs are really helpful. I think that you're modeling to the rest of the team what good looks like. So if you're putting up three, 5,000 line PRs, other engineers on the team are going to start doing that and that's not amazing. So it's really important. Your hygiene is really good because it's modeling for the rest of the team what good looks like. And then, ooh, this is so tempting. It's so tempting to put other reviewers on code you haven't read yet. Don't do it. It's mostly because you're going to look like an ass because there's going to be something in that code that is going to be so obviously bad. And then some senior engineer is going to call you and be like, "Yo, why didn't you read this?" So I speak from experience. Read the code, review it properly before you add anybody else to it. So hopefully this convinces you. Building is part of the job now. You can do it even with just one or two hours a day and you should. You'll start to feel the models every three months as the new model families come out and you'll learn the skill on how to push scope as you are instructing them. And you'll be able to show the team and the org what's possible as well. You know, just because you, as you have this capability, you automatically think of ideas on what to slot in for these overnight runs. And then you can model for the team, "Hey, this is what the future looks like." Thank you. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.