# Multiplayer agentic engineering — Arjun Singh, Superconductor AI Engineer · 2026-08-09 Arjun Singh explains how Superconductor enables multiplayer agentic engineering by making agents model-agnostic, cloud-isolated, and reachable from Slack, desktop, and GitHub as one shared session. He argues agents should run in a configurable network sandbox for least privilege, letting non-technical staff trigger real work without dev setups; a meeting bot left in a Google Meet at their expo booth picked up a passerby's idea, opened a ticket, and added acceptance-criteria fields. He advises benchmarking agents on your own codebase because SWE-bench is Python while they are Ruby on Rails, citing one month: 10.5 billion tokens, 3,300 Claude Code runs worth about $10,000, and Codex running four times as many sessions for less money. Takeaways: sandbox your code, integrate agents into human interfaces, stay model-agnostic. ## Questions this episode answers ### How did Superconductor's meeting bot automatically turn a conversation at their expo booth into a working feature? Arjun Singh described a bot left in a four-hour Google Meet at their expo booth. It heard someone suggest that coding agents need clear acceptance criteria, and without any manual prompting, it created a ticket, started working, and added two acceptance criteria fields to their product's ticket form, then produced a screenshot. Singh noted he wouldn't ship that exact change, but it provided a concrete, immediately testable prototype. [6:46](https://aiengineer.podhood.com/762a9865-1cd2-4170-b117-0b654cff43cf?t=406000) ### Why does Arjun Singh argue that running coding agents in an isolated cloud environment is crucial, beyond just avoiding laptop dependency? He stresses least privilege: agents on a developer's machine could find a token that accidentally points to production and cause damage, like wiping databases. An isolated environment with a configurable network sandbox prevents exfiltration and requires approval for new external access. This also lets non-technical team members trigger real work without setting up dev environments, as everything runs in a secure, shareable sandbox. [9:05](https://aiengineer.podhood.com/762a9865-1cd2-4170-b117-0b654cff43cf?t=545000) ### How does Superconductor benchmark coding agents on their own Ruby on Rails codebase, and what results did they see from Claude and Codex? They replay pull requests representing good engineering work and plot quality against cost and time. For their codebase, Anthropic agents improved but stayed expensive and not faster; Codex and Cursor were fast and quite good. In one month, they had 3,300 Claude Code runs costing $10,000 in token value, while Codex had four times as many sessions for less, so they made Codex their default. [13:25](https://aiengineer.podhood.com/762a9865-1cd2-4170-b117-0b654cff43cf?t=805000) ## Key moments - **[0:00] Intro** - [0:42] Arjun Singh co-founded Gradescope, used by millions of students worldwide, before creating Superconductor with his team. - **[1:50] Be Agnostic** - [1:50] Arjun Singh recommends being model and harness agnostic because the best options change weekly and token sellers' incentives don't align with yours. - **[2:58] Every Interface** - [3:37] Superconductor aims to have the same agent session reachable from Slack, desktop app, and GitHub, making work visible and collaborative across the team. - **[4:11] Visible Work** - **[5:35] Signal to Code** - [5:35] Arjun Singh's third lesson is to automatically turn external signals like Slack chats, meetings, and bug reports into code the team can quickly evaluate. - [7:44] Superconductor's meeting bot listened to a four-hour Google Meet and turned a passing comment about acceptance criteria into a concrete prototype with new ticket fields. - **[8:50] Cloud Isolation** - [10:24] Arjun Singh argues that running agents in isolated cloud environments prevents lit anxiety and stops resourceful agents from finding and misusing production tokens. - [12:14] Non-technical team members at Superconductor trigger real code changes directly from Slack because cloud sandboxes eliminate the need for local development environments. - **[13:25] Benchmarking** - [13:56] Arjun Singh warns that public Python benchmarks don't apply to Ruby on Rails, so Superconductor benchmarks agents on its own codebase for quality/cost/time. - [15:07] Superconductor made Codex its default agent after benchmarking showed it cheaper and faster on Ruby, then switched to Fable temporarily when it outperformed. - [16:26] In one month, Superconductor's small team used 10.5B tokens, 3,300 Claude Code runs worth $10k token value, while Codex ran 4x as many sessions for less cost. - **[17:00] Recommendations** - **[18:17] Giveaway** ## Speakers - **Arjun** (guest) ## Topics Agent Engineering, Coding Agents ## Mentioned Anthropic (company), Claude (product), Claude Code (product), Codex (product), Cursor (product), Fable (product), GLM (product), GitHub (product), Google Meet (product), Gradescope (product), Kimi K2 (product), Linear (product), Minimax (product), Notion (product), Sentry (product), Slack (product), Superconductor (product), Teams (product) ## Transcript ### Intro **Arjun** [0:13] Alright, hey everyone. I'm Arjun Singh. Today I'm going to talk to you about multiplayer agentic engineering, or how to enable your whole team and your best agents to work together. If you go to the talks or go around the expo, you're going to see that a lot of people are just talking about putting the agents at the center of everything. It makes sense. They're really powerful. They're really cool. But you don't see a lot of people talking about the people. Like, this is all for us, to make us our lives better or more productive or whatever. And so we're going to really focus on how the people fit into these agentic workflows. So just a little bit about us first. So our team has worked together building software for over a decade. My co-founder, Sergey, and I, we met in the PhD program at Berkeley. I worked on robotics. We worked on computer vision. And during that, we co-founded a company called Gradescope. Some of you may have used it. It's used by millions of students worldwide at thousands of universities. It helps instructors grade their students' work. And pretty much the entire team working on Superconductor used to work together on Gradescope. And so we've had a team that's worked together productively from first user to acquisition, working on something new together again. And I think it's kind of an interesting experiment because, you know, over the past year, we've very aggressively integrated agents into our workflows. And we've kind of surfaced all the different bottlenecks and friction points that come up, and how to do that productively and keep collaborating the way we used to, but with the new power of agents. So today I'm going to talk to you about how, you know, we the lessons we learned from kind of solving those friction points and solving those bottlenecks. And in the talk description, I mentioned five lessons. And I'm going to be an engineer and start from zero and add a sixth one in there. The first one I'm going to start with is just to be model and harness agnostic. So there's a few reasons for that. The best model and harness can change weekly. It could change because a new one comes out. It could change because the best one got taken away. Things happen. And you don't want that to disrupt your entire team's flow. ### Be Agnostic **Arjun** [2:04] The other thing is that open weight models are actually pretty good now. We've been really happy with GLM 5.2. They're much cheaper. And you want to be able to kind of explore with them and integrate them without, again, having to change your entire workflow. And the last thing I'll mention on this is that the incentives of the people selling you tokens aren't really aligned with yours. You're here for a reason. So you're working on things for a reason. You're trying to make your customers' lives better, make your product better, delight your customers. And they want to sell you more tokens. And you might be happy to pay for as many tokens as it takes, but you don't want to pay for more than that. And so, again, kind of being able to switch between things lets you stay in control of all of that. And so, you know, as I go through the talk, I'll mention a couple of places where our product makes it easy for us. But whether you use us or not, I'm just going to leave things with you that I think are really important for you to be able to work collaboratively effectively. So the next one is to turn every human interface into an agent and human interface. So, you know, typically when people are working with coding agents, they're on their laptop. They're kind of stuck on that laptop. Nobody else can talk to that agent. ### Every Interface **Arjun** [3:13] So the first place people go to kind of expose more interfaces for them is Slack. Claude has a Slack bot. Codex has a Slack bot. We have a Slack bot. It's really cool. You can say, add Superconductor, do X, Y, Z. It does it. Somebody else can talk to it. But it's not enough. Because now we've taken it from trapped on somebody's laptop to kind of trapped in Slack. And a lot of work happened in Slack, so that's better than nothing, but certainly not all work happens in Slack. So what we really wanted was to be able to work with the same session from every relevant interface. It could be Slack. It could be our app. It could be GitHub. It could be elsewhere. And so one possible flow is you start and collaborate on a session in Slack. And then maybe you continue in a kind of more engineer-focused environment in the desktop app or the mobile app. And then you can finish it up in GitHub. And the important thing here is it's the exact same agent session. So it's like the agent didn't forget what you did in one place in Slack when you go and talk to it from GitHub. It's the same session. It's got the same context. And the second lesson builds on top of that, which is to make the agent work visible and collaborative across the team. And so obviously, Slack makes it more collaborative. But here, we've got that kind of app view. And, you know, Sergey made this ticket. ### Visible Work **Arjun** [4:25] I've been talking to the same ticket. Our growth person hopped in as well. And so you can kind of see at the top here all the different people that interacted with this. So I can see who's going to be notified about this session, who's seen it. That's especially important when you have work triggered by non-technical people,right? So it's like, you know, our customer support person created a ticket. It's working really well. I want to understand, like, has this been vetted by an engineer or not? I can kind of see who's involved really easily. And then if I'm reviewing something, I can just pop in and say, like, hey, why did you do it this way? And again, because it's the same agent session, I don't need to wait for Sergey to kind of get my notification on GitHub and respond to me. The answer to the question is almost certainly in this thread. I also don't want to read the entire thread. So I can just ask the agent. Or how we most often kind of make the work visible is with artifacts. So it doesn't matter where the work started or where it's finishing. The agent can show you the work it's doing as a screenshot or video or other. And you can see it from everywhere. So again, you don't have to worry about, like, where is that thing? I got to go to GitHub to see the image. Or I got to go to Slack to see the image. It's just everywhere. The work is visible everywhere. You can collaborate from anywhere. ### Signal to Code **Arjun** [5:35] So the third lesson I'm going to talk about here is to turn every external signal into code that your team can quickly evaluate. And I was hoping to show this live, but the Wi-Fi is not quite there. So I'm going to show you something from yesterday. But what do I mean by external signal? So it could be a Slack conversation. It could be a meeting you have with a customer, an onboarding call, a sales call. It could be an internal team meeting. It could be something from Sentry or a bug tracker, a bug report from a customer, an email, feature request. Andright now what's happening is, like, all that stuff already exists. It's in all those different systems. People hook them together with MCPs. So now your coding agent can check the email or check Notion or whatever it might be. But, like, how does it know what to work on,right? It's like, it's still kind of stuck everywhere. And so, like, some humans are involved in, like, kind of taking stuff from one place and telling it, solve email number 48 or ticket number 6000. But that's still a lot of coordination. And so what we do is we have several different ways to automatically ingest these signals, prioritize what to do with it, and kind of act on them. And my favorite one, the most fun one, is what we call our meeting bot. And so I'm going to switch over to my browser here for a second. And, okay. So we've got a booth at the expo. And we had the meeting bot running all day yesterday. So this is a four-hour meeting of a Google Meet. You just kind of invite the bot to meet, resume, or Teams, or whatever it might be. And it listened all day. And it created all sorts of stuff as it was listening. If it finds existing work, it'll link to it,right? So it's not going to just, like, create new work if it's something you're already working on. Some of this is, you know, people testing the meeting bot out and telling it to do some weird things or interesting things or just creative ideas. But a lot of it's actually just, like, really good ideas that come out of people looking at what we're doing, asking questions, having new ideas on what to do with it. And so it's kind of nice because the last idea that was here was someone was saying, hey, like, when I work with coding agents, I want to make sure that the agent has clear criteria to evaluate whether it did a good job on the work before it tells me that it's done. And so they had that idea. The bot just picked up on it. None of us did anything manually. It created this ticket and started working on it. And then I was able to just say, hey, take a screenshot of what you did. And here's that screenshot. And it kind of modified our ticket form to add these two new fields of acceptance criteria. Now, am I going to ship this one exactly how it is? Like, no, probably not. But it's a new idea. It's concrete. I can play with it. I can go and actually, like, use the live preview and, like, see if this improves performance. And so it takes, like, this, you know, your hundreds or thousands of ideas that are everywhere. And it helps you kind of move with the speed of what your customers are asking you for and what they're thinking. And it's really fun because every time we have an onboarding or a customer call or team meeting, we almost always have dozens of new ideas that are prototyped. But more importantly, at least a few shippable PRs with very minimal intervention. So we talk. Stuff comes out. We look at it. We ship it. It's so much fun. Put this back. So the next thing I'm going to mention is that, you know, these three things that I've talked to you about really rely on having your workflow, your code base, your project set up to work in an isolated cloud environment. ### Cloud Isolation **Arjun** [9:05] So that way the agents aren't trapped on an individual's machine. So there's several reasons why this is important. So the first one is to eliminate what some people are calling lit anxiety. You want to be able to close your laptop. You've probably seen people running around the conference with their laptops open while stuff is working or at the airport. Or, you know, there's some posts on Twitter or whatever about, you know, like, people having their laptop tethered to their phone in their car as they're driving home. This was actually probably the impetus for me and for a few people on our team to even start working on this. You know, last year I started working with Claude Code a lot. I had, I think at the time, like, six months old. I, like, didn't want to, like, be tied to my laptop or have that stress. It's like, I don't ever want to think about whether I can, like, step away from a laptop or not. And so we moved everything to the cloud. Things are working always. It eliminated that problem for us that people have been talking about for the past year. It's really helpful. It's important to me. But I don't think that's the most important reason to do this. I think the most important reason to do this actually was touched on in the previous talk if you were here for it. I think you should only give your agents access to only what they need,right? So if you think about what's happening, you have a bunch of developers with these agents running their laptop. Their laptops, unless you have, like, impeccable hygiene, probably have a bunch of stuff on it that you don't want the LLMs or agents to have access to. And yeah, like, everybody's working on these sandboxes and approval flows. And so you, but really, you're in one of two camps. You're either approving a bunch of stuff, or you're hoping that your auto-approval flow or your YOLO mode or whatever is configured properly and your sandbox configured properly and doesn't read a bunch of stuff on your laptop that it should have. And, you know, as the previous talk mentioned, like, these agents are getting more autonomous. They're getting really resourceful. They're trying to please you and do what you said. And so when you say, hey, you know, wipe the staging database and it finds a token on your laptop that it can use and it thinks it's working with staging, but actually it's production and now it just deleted everything. I'm not trying to say this is happening constantly, but it still happens. And for us, the peace of mind of just, like, letting anybody run with these experiments and ideas and prototypes and real code without having to worry about this is really worthwhile. To go one step further on that, it's not just, hey, make sure they don't have the credentials that they shouldn't have. It's also make sure they can't exfiltrate your code or your projects or your secrets or your content to somewhere they shouldn't be able to. And so you have a configurable network sandbox and you say, look, these are the places you're allowed to access. These are the ones you can't access. And anytime it tries to access something that it shouldn't, it just pops up and says, hey, tried to access something. Do you want to give it access? Maybe you're trying to integrate a new vendor and need documentation. And you can do it on a per-ticket basis or for the whole project. And so again, that peace of mind of, like, people can do things. If they need new access, it's easy to grant it. And we're not going to leak a bunch of important data by running agents in YOLO mode. And the last thing I'll mention about that is that this is the key for allowing your non-technical team members to trigger real work,right? Your non-technical people don't have development environments set up on their computers. But we've gotten our support people, our growth people to actually be meaningfully impacting the product by just talking to users, seeing bugs, experiencing it themselves, and just go to Slack or the app itself and say, hey, fix this. They fix it. Screenshots are shown. Engineer gets it. It gets merged. Without that, they'd have to put it in linear and engineer wouldn't eventually pick it up and a PM would triage it or whatever. None of that here. You just ask for it and it's done. Now, the reason people didn't do this up until somewhat recently, like, this was really painful. Getting your full thing set up in this, like, kind of sandbox environment used to be really, really painful. But the agents have gotten better. We have our own environment setup assistant that kind of takes your project and gets it to work in one of these sandboxes. But honestly, whether you use us or not, I highly recommend you get your project working this way. And you can just get Claude Code or Codex to do this for you. You don't have to use us. But, you know, we think it's the best way. ### Benchmarking **Arjun** [13:25] And the last lesson is to benchmark agents on your code base. So the way we do this is we select pull requests that represent great engineering work. It could be agent-created, it could be human-created, it could be a hybrid. It doesn't matter. You pick the agents you want to use and benchmark. And then you get a quality versus cost and time breakdown in your code base. Now, why do you want to do this? There's many reasons. But one is that if you're kind of going off the public benchmarks, SWE-bench or Terminal Bench or other stuff, like, those tasks may have absolutely nothing to do with your task. Like, SWE-bench is all in Python. We're Ruby on Rails. It is not the case that the benchmarks are identical for them. There's trends that do compare. But the results can be very, very different. And I'm going to swap over to my browser one more time here. So these are results on our code base of all these different harnesses. This one here is quality versus cost. This one here is quality versus time. I'll start with this one. You can see some trends here,right? You can see that the Anthropic agents have just been consistently getting better, but not really any faster. The Codex agents and Cursor are actually pretty fast and quite good. The open stuff has been getting better and better over time, but they're kind of slow. This is for our code base again. I'm not trying to make any general claims here. By cost, the Anthropic stuff is clearly just so much more expensive for us. And the Codex stuff has been cheaper for us. And so this causes us to change our behavior. We still use the different models. There's different use cases for them. We like the variety. We still use all these things. But when we saw these results, they kind of matched our vibe check. We wanted to kind of, like, have hard data too. We switched our default to Codex at that time. Then Fable came out. And it was great. Kind of switched our default to that for, like, the few days we had it. And then it went away and switched back to Codex. But the most important thing is, like, because we're agnostic, like, none of that had any meaningful disruption on our work. Like, we're able to just kind of switch back and forth really easily. So next day something new comes out. See if it's good. And go. And the last thing I want to mention around that is, like, I don't know if this resonates with you all, but I've got a lot of friends that, like, okay, I heard Minimax is good. I heard, you know, GLM is good and Kimi K2 is good. But, like, I haven't had the time to try it out. And we keep telling them they need to because it's so much better and faster and cheaper. And you kind of have that anxiety for a little while. And then, like, finally you take the two hours to try it. And it's like, oh, actually, like, didn't really work for us. So, like, what's, you know, I just wasted those two hours. This kind of eliminates that. And it helps you kind of stay on the cutting edge really, like, seamlessly. Let me go back. So what that kind of turned into for us is, you know, essentially 100%, like, 99.9% of our pull requests are, like, heavily agent-generated. We know that quality and reliability and security are really important. So we still have humans look at everything. We have agents help with it all, but everything's human-reviewed. You know, for our relatively small team, we had 10.5 billion tokens over the past month. And you can kind of see what we were saying about Claude here. It's a little small, so I apologize. But we had 3,300 Claude Code runs that cost $10,000 in token value. We have planes, so we didn't spend $10,000 on it. And Codex had four times as many sessions. And it was cheaper overall. And so again, the vast majority of our work currently is merged through Codex. We still use the other models. More and more is happening through GLM 5.2. You can invest in that. And one thing that we're really excited to do going forward with this benchmarking is automatically, like, you've probably heard about people, you know, routing tasks to theright models and all that. ### Recommendations **Arjun** [17:00] But how does, like, some third party know what to route for your code base? Like, this is a way that you know what's going to work best for which task for your project. And we're going to kind of automatically route it now for you. So I'm going to leave you with a few recommendations. So first, get your code base and agents working in a sandbox. It unlocks a lot of different things, a lot of different workflows, everything I've talked about and more. Second, integrate agents into the relevant human interfaces so your team and your agents can work together and don't have to, like, context switch and copy context back and forth. Obviously, we think Superconductor is the best way to do it, but plenty of people are home rolling things, packing things together. Figure out how to make this happen because if not, the friction is just really high. And lastly, find a way to benchmark and become model agnostic. So you're not tied to anybody and you can just constantly stay at thatright part on the frontier of cost, speed, quality. So thank you so much. We've got a booth in the expo. Please feel free to come by. You can sign up at superconductor.com or you can email me with any questions at arjun@superconductor.com. I will be out in the back as well for any questions. Thanks so much. ### Giveaway **Arjun** [18:17] One last thing. If, you know, at the booth we're mentioning, we're giving away a MacBook Neo. If you are here because you signed up for that, just meet us outside and we will announce the winner. Thank you. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.