# AI Agents Are Just Distributed Systems Now — Salman Munaf, TikTok AI Engineer · 2026-08-29 Salman Munaf, TikTok site reliability engineer, argues AI agents are distributed systems once they call external services; deterministic controls must surround the probabilistic coordinator. A refund timeout illustrates it: timeout means unknown, not failure, and retrying can double-refund without request IDs, idempotency keys, and status lookups. He urges persisting every step, defining compensating actions, treating action-influencing context as cacheable state with invalidation and provenance, and adding guardrails: circuit breakers, budgets, exponential backoff, scoped read/write credentials, and approvals bound to action, timestamp, actor, expiration. Observability must trace prompts, calls, writes; tool contracts embed idempotency; ask what system lets the agent do when wrong. ## Questions this episode answers ### Why are AI agents considered distributed systems? Salman Munaf explains that AI agents are distributed systems because, unlike chatbots, they run an agent loop that calls external services and changes state. He calls the model a 'probabilistic coordinator' that replaces deterministic decision trees, so the surrounding systems need deterministic controls to bound and recover from its actions. [3:33](https://aiengineer.podhood.com/8551ed85-d23f-431d-af19-d4748c740b85?t=213000) ### What does a timeout mean when an AI agent calls a refund tool? A timeout means unknown, not failure, says Salman Munaf. When a refund tool call times out, the agent cannot know if the refund happened, so retrying without safeguards can double-refund. He recommends request IDs, idempotency keys, and status lookups so duplicate requests do not create duplicate side effects. [8:15](https://aiengineer.podhood.com/8551ed85-d23f-431d-af19-d4748c740b85?t=495000) ### How should AI agent context and memory be handled? Salman Munaf says that when context can influence an action, it is state, not just context. It can go stale or conflict with authoritative data, so teams should treat short-term and long-term memory like a cache, with invalidation and provenance, and decide which source is the source of truth. [10:26](https://aiengineer.podhood.com/8551ed85-d23f-431d-af19-d4748c740b85?t=626000) ### How should human approvals be scoped for AI agents? Salman Munaf says a human approval shouldn't be a blanket approval; it should be tied to action, timestamp, actor, and expiration. For example, an approval for a $30 refund should not turn into approval for a $300 refund. This prevents a small consent from being reused for a larger transaction. [15:52](https://aiengineer.podhood.com/8551ed85-d23f-431d-af19-d4748c740b85?t=952000) ## Key moments - **[0:00] Intro** - **[0:33] Distributed shift** - [0:33] AI agents became distributed systems once they could call external services and cause real-world side effects, says Salman Munaf. - [1:03] Salman Munaf cites Replit AI deleting a production database and Air Canada's chatbot refund error as AI incidents preventable by systems thinking. - **[1:29] Incidents** - **[3:33] Probabilistic coordinator** - [3:33] Salman Munaf frames AI agents as probabilistic coordinators, unlike the deterministic coordinators of traditional multi-step workflows. - [4:43] Each step of an AI agent loop — planning, action, observation, persistence — crosses a system boundary and can fail or cause side effects. - **[4:57] Agent loop** - **[6:54] Tool failures** - [7:58] Q: If an AI agent's refund call times out, did the refund happen? A: Timeout means unknown, so use request IDs, idempotency keys, and status lookups. - **[8:32] Idempotency** - [10:26] AI agent context that can influence an action is state, not just context — it goes stale and must be invalidated like a cache, says Salman Munaf. - **[10:57] State and memory** - **[12:08] Compensating actions** - [12:08] When an AI agent succeeds early then fails, the whole multi-system transaction needs compensating actions — e.g., an apology email after a wrong email. - **[13:19] Guardrails** - [13:19] AI agents need circuit breakers, rate limits, and budgets to prevent retry storms and cascading failures in downstream systems. - [15:33] "A harmless model can become dangerous when it can perform unsafe operations" — Salman Munaf on scoped credentials and tool allowlists. - [15:52] An AI agent's human approval must be tied to action, actor, timestamp, and expiration — approving a $30 refund should not approve a $300 one. - **[16:32] Observability** - [16:32] Logs aren't enough to debug AI agents; teams need traces of model calls, prompts, tool calls, retrieved context, writes, and approvals. - **[17:31] Recovery** - [17:31] Smarter models reduce mistakes but cannot eliminate network failures, stale data, or adversarial input; systems must bound, observe, and recover, says Salman Munaf. - [19:23] "When building AI agents, we should also ask what the system lets it do when it is wrong" — Salman Munaf's closing test. ## Speakers - **Salman Munaf** (guest) ## Topics Guardrails, Observability, Agent Engineering ## Mentioned Air Canada (company), Replit (company) ## Transcript ### Intro **Salman Munaf** [0:13] Hello everyone, good— good afternoon. Uh, today I will be talking about AI Agents are also distributed systems. So, as the models have started to become more complex, initially the LLM models were just text in, text out, without performing any actions. And the effect that they can produce was just a wrong model output. However, with now the capability of agent— the agentic— the rise in agentic capabilities, where the systems can now talk to external systems, it has turned into a distributed systems. ### Distributed shift **Salman Munaf** [0:55] And it is important to incorporate distributed systems thinking and concepts when building AI Agents. So I will be going over that in this talk. So you guys might have heard about incidents being caused by AI Agents. For instance, the Replit AI Agent deleting a production incident production database, or Air Canada chatbot basically making an, uh, an incorrect refund. And both of these incidents, or a lot of these incidents, could have been prevented by good systems thinking when building these AI Agents. So for instance, for the Replit ### Incidents **Salman Munaf** [1:46] incident, we could have good, uh, uh, we could have robust backups, we could have scoped authority. We should— we shouldn't, ideally, have allow AI Agents to delete production databases. Moreover, for Air Canada chatbot, it would have been a good idea to have an authoritative source of truth retrieval so that it's not making decisions based on stale or incorrect policies. So let's go over the transition from chatbot to production system. So initially, when we were in the, uh, in the age where LLMs were just chatbots, we had prompt in and we were outputting text. There were no side effects. The agent was not interacting with any other system. However, due to agentic— in the agentic era, in the agentic revolution, now those agents, by ingesting prompt, can run an agent loop, call external services, call tools, and also perform state changes. The architectural boundary now has moved way beyond an LLM model. And the difference is that it can now cause side effects in the outside world. So when basic— when building AI Agents, it is important to recognize the external systems that it is talking to, the states that it is interacting with, and what credentials does it have, and the actions that it can perform. ### Probabilistic coordinator **Salman Munaf** [3:33] I ideally like to think about it as, uh, AI Agents as basically having a probabilistic coordinator. In distributed systems as well, we used to have services which were coordinating multi-step workflows. However, they were deterministic in nature. But in the case of AI Agent, the AI acts as a probabilistic coordinator. The amount of action, the kind of actions that it can take, can vary quite a lot. It is not just a decision tree that we typically in traditional systems would have mapped out. And those actions can have severe consequences if they are not confined by our determinist— by, uh, by having deterministic controls in place. So it is important to ensure that we have deterministic controls in place to ensure that agent, or the AI Agent, is not performing any, uh, any actions that might be problematic. So, uh, let's discuss how a typical agent loop might look like. So at first, it might do some planning. Then, based on that plan, it will perform an action. And it will then observe the results of those actions. ### Agent loop **Salman Munaf** [5:04] And it might persist that into some data, some data store, and then decide what to do next. Each step in this loop is basically crossing a boundary. During planning, it can interact with data sources to retrieve some data. During action, it can call external APIs, tools, databases, and perform any actions. During observation phase, it can perform, it can get partial results, and basically plan or make subsequent actions based on those partial results. It can persist incorrect data, or, uh, and, uh, when deciding, it might also decide to perform an incorrect action. Or worse, it can also do a retry storm. So it is very important when building an agent loop to persist every step of the process. Whatever actions the agent is doing, whatever context it is retrieving, it is important to persist that so that if anything fails, the agent is able to recognize where it failed and it can perform a reversible action. It can perform undo operations. Similarly, there should be explicit transactions identified for each step. So for instance, if an agent is making a call, if it fails, what it should do? What should be the transaction to compensate for a, uh, for an irreversible or unsafe operation? For instance, if an agent makes sends an email to a, a wrong email to a customer, what should it do to compensate for that? So, uh, tool calls are just wrappers around, uh, external, external APIs, databases, queues, and so on. And, uh, with, uh, when calling, the when making these remote calls, there are some failures that you incorporate, such as network delays, timeouts, you can make duplicate requests, or worse, the server-side request might succeed. ### Tool failures **Salman Munaf** [7:26] However, the client, however, the client might be reported an error. We have seen, uh, instances where a database might have written the data. However, due to some other errors, the server might have reported to us the error. And, uh, with humans in the loop, we can make a, we can basically perform corrective actions based on, uh, by seeing the database and actual source of truth. But in agents' case, we need to ensure that we have, uh, we have proper guardrails in place. So for instance, an agent calls refund customer tool call, which basically performs a refund to the customer. The request times out. Uh, did the refund happen or not? What will the agent infer from that? Would it retry refunding to the customer? Basically, the timeout does not actually mean that a failure had occurred. It means unknown. And it is important to have, uh, when designing these tools, it is important to have request IDs, idempotency keys, so that when making duplicate requests, they are not causing duplicate side effects. ### Idempotency **Salman Munaf** [8:48] And the system can always do a status lookup, like what the previous request was and what was the status of that, so that it is not making side effect, uh, it is not making side effects with duplicate actions, with duplicate requests. So AI Agents, when they, whenever they, uh, uh, whenever they, uh, they face failures, they retry. Their first action is to perform retries. So it is really important to have idempotency baked in. It, if a same request is coming in to an external API or the tool, it should recognize that this is a duplicate request and ensure that no side effects are being taking, are taking place. Moreover, we should also prevent AI Agents to perform retry storms to external APIs because this can cause cascading failures. Uh, we should have max turns, budget spend, and max parallel calls to prevent, uh, to, uh, to ensure that the fan out is not that large. Moreover, we should have exponential backoff in place to ensure that the downstream dependencies are not being, uh, burdened. And we should also have compensation, uh, operations in place for, uh, operations that, uh, that, that can have side effects. Uh, so, uh, a lot of, uh, teams when building AI Agents think of AI Agent context as just AI context, the context that the AI Agent has as, uh, as just a context. However, when that context can influence an action, it's a state. And that state can become stale, that can conflict with the authoritative data, or corrupt future actions that the agent might perform. I like to classify it into two different types of, uh, memory that the agent has. First is the short-term memory, which is the chat thread, uh, that the agent has, uh, the, which is tied to a single execution thread. ### State and memory **Salman Munaf** [11:14] And the second is the long-term memory. It can be project files, uh, system prompts, uh, databases that it interacts with, the cache layer, and so on. It is important to, uh, to, to decide what will be the source of truth when these, uh, different data sources have conflicting information. And we should ideally treat memory as a cache, which, uh, can be invalidated, which can have provenance attached to it. So for instance, whenever a data store or a database is updated, or the source of truth is updated, we invalidate the context or the memory that the agent has to ensure that it is not making actions based on the, uh, incorrect or stale data. ### Compensating actions **Salman Munaf** [12:08] So usually these agents perform multi-step actions. And the agent can succeed on, uh, on, uh, on the first couple of steps, and then it fail. Uh, it is important to reverse the entire transaction that was performed. And these can, uh, then can cross system boundaries. So for instance, an agent can update an internal ticket, uh, send an email to a customer, and fail to update the CRM. We need to figure out what is the, uh, correct compensation operation when it, when it hits that failure. So for instance, uh, as I mentioned earlier, that, uh, it improperly, uh, it improperly sends an incorrect email to the customer. It is important that the compensation operation is defined for the AI Agent to ensure that it is sending an, uh, uh, an apology email to the customer or any, or an email, or an email that is correcting that mistake. ### Guardrails **Salman Munaf** [13:19] So, uh, the AI Agent basically runs in a loop. And whenever, uh, like, it can, it can do multiple calls. It can, it can have a retry, uh, retry loop that it can run based, uh, whenever it fails. So it is important to have circuit breakers whenever it is making external calls, uh, to ensure that the, uh, that the, that it is not, uh, burdening the downstream system. Uh, for instance, if a downstream is unhealthy, there should be system breakers, circuit breakers in place that prevents AI Agents to call, call that dependency. Moreover, it also prevents cascading failures when, for instance, the downstream dependency is, uh, unhealthy or, uh, is saturated. It is also important to assign rate limits and budgets. Uh, an agent can, uh, go over, uh, can run your cost, uh, if it's not assigned proper budgets and rate limits. It will, uh, keep retrying and try, try to, uh, try to solve the problem that if it's, if it's facing. So it is important that it is, uh, that we have, uh, set up max turns, max parallelism, max spend, uh, to ensure that the model is not, uh, uh, not, uh, crossing the, uh, the budget boundary that we have set. Moreover, uh, ideally, uh, usually whenever we are building AI Agents, uh, we usually try to give all the permissions that it can have to ensure that it has all, that it can perform, perform the task that we have. That's the, that's the, uh, first, uh, uh, thing that we have. That's the first step that we take usually that to give the AI Agents all the, uh, creden, all the, uh, privileges to perform any actions. Like for instance, if it's interacting with a database, we just give it all, uh, the rewrite access to the entire table. However, uh, it is important to give scoped credentials to it. There should be separate read and write permissions. And there should be allowlist for the tools that it can call. A harmless model can become dangerous when it can perform unsafe operations. Moreover, uh, a human approval shouldn't be tied, uh, to a blanket approval. It should be tied to, uh, action, timestamp, actor, and expiration. So for instance, if a user has given, uh, an approval to approve a $30 refund, it shouldn't turn into a subsequent approval for a $300 refund. It is important that whenever an approval is given, it should be tied to the, uh, particular parameters that it was, uh, asked for. So, uh, observability is an important requirement when building AI Agents because, uh, and logs are not enough. Teams need to reconstruct when an agent failed, what happened, what information was it reacting to, and why it failed. And logs alone are not enough to, uh, for an agent to, uh, for teams to determine that. ### Observability **Salman Munaf** [17:00] It is important to trace the model that was called, the prompt that was, uh, that was, uh, given to it, and, uh, also the tool calls that were made, uh, the request, uh, that was made, the response from the tool, the errors that it got, the retrieved context, what the agent was, uh, was the, the retrieved information that the agent was reacting to, the writes that it made, and the approvals that it got, and so on. ### Recovery **Salman Munaf** [17:31] So, uh, I would like to, uh, end with, uh, the idea that, yes, model capability matters. Having good models, uh, improves the, uh, likelihood of it making, uh, correct operations. Smarter models reduce mistakes. It, uh, it, uh, improves the capability that the model has. However, it cannot eliminate network failures, stale data, or adversarial input. It is important when building this architecture, we also reason about can we bound, observe, and recover from actions performed by the AI Agent. It is important to have tool contracts in place to ensure that, uh, it is only allowed to make, uh, operations that it is, uh, given, that it is provide the contract. And the contracts are clearly establishing the request and response, uh, response, uh, response types, uh, the schema, and all these tools have idempotency baked into it so that, uh, when repeated requests are sent in, uh, it is not causing unsafe operations to be retried. Moreover, there should be source of truth decisions made, uh, when there are conflict, conflicting, uh, memory states. It is important, uh, for the agent to realize this is the source of truth data that it should rely on. And we should have re retry policies, uh, like rate limits set in to ensure that the agent is not, uh, retrying, uh, exte uh, aggressively. Moreover, uh, permissions should be set up. There should be traces and recovery paths. So when building AI Agents, we should also ask what the system lets it do when it is wrong. Thank you. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.