# Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates AI Engineer · 2026-07-23 Subbiah Sethuraman and Abhilash Asokan of ZS Associates explain why they killed their multi-agent pipeline for pharma commercial analytics: the system produced incoherent output because no single agent owned end-to-end reasoning, domain knowledge was missing, and LLMs were used for deterministic signal detection. Observing Claude Code in an empty directory, they rebuilt with a deterministic pipeline that detects signals before the agent wakes up, consolidated to a single agent that owns reasoning and spawns sub-agents only for focused lookups, and added a knowledge graph as a control plane where every edge is a hypothesis the agent tests against data. The new system does in 20 minutes what an analyst did in a month, achieving bounded search and coherent output. ## Questions this episode answers ### Why did ZS Associates kill their multi-agent pipeline for pharma analytics? Subbiah Sethuraman explains the multi-agent system produced incoherent answers because no single agent owned the full picture. The root cause was correctly identified, but the action was misaligned. Context was lost in handoffs between agents, and the system lacked shared business domain knowledge. An LLM shouldn't determine signals, and distributed reasoning led to faulty output. [4:32](https://aiengineer.podhood.com/400fd909-803d-42e0-9e3c-4e5e0b74c88c?t=272000) ### After killing the multi-agent pipeline, how did ZS redesign their agentic system? Abhilash Asokan describes observing Claude Code, then rebuilding. They separated signal detection into a deterministic pipeline with statistical methods and thresholds, so the agent only investigates pre-identified signals. They consolidated reasoning into a single agent that dynamically spawns sub-agents only for focused investigative tasks, not for judgment, resulting in a lighter architecture. [6:01](https://aiengineer.podhood.com/400fd909-803d-42e0-9e3c-4e5e0b74c88c?t=361000) ### How does the knowledge graph act as a control plane in ZS's new agent system? Subbiah and Abhilash designed the pharma knowledge graph not as a lookup, but as a control plane that dictates the agent's investigation path. Every edge represents a hypothesis the agent tests against real data. The agent starts at an entity, traverses neighborhood edges, evaluates each hypothesis, and only moves if evidence supports it, creating a bounded, disciplined search that avoids wild inferences. [11:05](https://aiengineer.podhood.com/400fd909-803d-42e0-9e3c-4e5e0b74c88c?t=665000) ### How much faster is ZS's rebuilt system compared to a human analyst? Abhilash Asokan states that after the rebuild, the agent system can produce in 20 to 30 minutes what a human analyst would take three to four weeks to complete. This 50-plus turn reasoning process, consuming many tokens, demonstrates the efficiency gain from consolidating reasoning, deterministic signal detection, and the knowledge graph control plane. [13:29](https://aiengineer.podhood.com/400fd909-803d-42e0-9e3c-4e5e0b74c88c?t=809000) ## Key moments - **[0:00] Pharma Analytics** - **[1:32] Analyst Steps** - **[2:30] Multi-Agent V1** - [2:30] ZS mimicked an analyst's workflow with separate agents for signal detection, source localization, driver attribution, and synthesis, connected by an orchestrator. - [3:19] Multi-agent pipeline correctly blamed an 18% Rx drop on a payer tier change but wrongly pushed more sales reps, showing no agent owned the full picture. - **[4:05] Incoherent Output** - **[4:40] Root Causes** - [4:40] Pipeline failed because LLMs handled deterministic signal detection, context got lost in handoffs, and agents lacked shared pharma business knowledge. - **[6:01] Claude Code Rebuild** - [6:01] Instead of redesigning topology, ZS gave Claude Code an empty directory with bash and a database, observed its actions, and derived a lighter architecture. - **[7:05] Deterministic Pipeline** - [7:05] ZS built a deterministic pipeline with statistical methods to detect signals before the agent wakes, so the agent only investigates. - **[8:00] Single Agent** - [9:22] ZS built a pharma knowledge graph as a control plane: every edge is a hypothesis the agent tests against data, bounding investigation rather than a lookup. - **[9:25] Knowledge Graph** - **[11:05] Control Plane** - [13:29] Single-agent system with knowledge graph produces in 20 minutes what an analyst took 3-4 weeks, after 50+ hypothesis-testing turns. - **[13:45] Key Takeaways** - [13:45] Key takeaway: Don't impose human workflow on agent architecture; let it emerge from observing LLM problem-solving. ## Speakers - **Abhilash Asokan** (guest) - **Subbiah Sethuraman** (guest) ## Topics Multi-Agent Orchestration, Agent Memory ## Mentioned ZS (company), Claude Code (product) ## Transcript ### Pharma Analytics **Subbiah Sethuraman** [0:13] Uh, okay, so I'm Subbiah. I head AI engineering at ZS. **Abhilash Asokan** [0:17] Uh, I'm Abhilash. I'm the director of AI engineering at ZS. **Subbiah Sethuraman** [0:20] So, ZS, we are a tech firm. We work with many of the top companies in the world, including a lot of the top pharmas, actually. Um, so today's talk, I think we wanted to, as I think already introduced,right, we wanted to talk from our experience,right, building multi-agent pipelines. Uh, what are the mistakes we did,right, and what did we learn and how did we fix them. So, um, so I'm going to orient it more on pharma, commercial domain, and for people in the room probably who are not aware, quickly, uh, pharma has two main functions. One is R&D,right, the drug discovery,right, uh, and the clinical trials part, and then there's the commercial, how do you take a drug to a patient, basically. And, and within commercial, there are different functions, like once you create a drug,right, um, what is the performance of a brand, how is the drug performing in different markets,right. Uh, then there are things around your field forces, your reps, how effectively are they engaging, uh,right, with everyone. There are things around patient journey, how patients are adopting a drug,right, I think if there is any therapy switch which is happening. So as you can think about, there is a lot of analytics which really happens in a commercial domain. And how do typically analysts work? So there are four steps,right, what analysts do,right. So first, there is always something called a signal detection,right. So signal can be something like, okay, the prescriptions, what a doctor is writing, maybe is there, is there a drop in the prescription, so that's a signal. So once you got a signal, the second thing what an analyst does is why is this signal really failing, what is the reason for it. ### Analyst Steps **Subbiah Sethuraman** [1:46] So is it like there is a competitor drug which has come in, because of that, is it, is it reducing, is it, is it because maybe a payer coverage for the drug has reduced, or maybe the, the reps on the ground, there's no proper, uh, they're actually not taking the benefits to the doctors. And once you arrive at the reason, the next step becomes, okay, what is the action do you take. So if reps, suppose if reps the coverage is not good in a particular region, should do we have to increase that. And once you do that, what is the, what is the outlook,right, is my brand, is my sales performance, is, is it going to improve,right. So these are the four things which happens. Now, for some of the top pharmas, what we have done is how do we in an agentic way,right, I think how do we actually mimic this, this analytics work. So what we did, we built agents for every step,right, signal detection. We said, okay, we'll have an agent for signal detection. It'll identify, identify the signals for me. Second, what are the root cause,right, for the signals,right. So in this case, we have two agents. ### Multi-Agent V1 **Subbiah Sethuraman** [2:44] One, we call it a source localization. So for example, if my sales is dropping at a national level, is it because it's dropping at, say, a particular region, or is it dropping for a payer. So we need to understand that. We, we are trying to identify the source of it. And then once we understand what is the real reason,right, I think the, the sales performance has gone down, that's another agent, the driver attribution agent. And then the last step is your synthesis,right. So depending on the cause, now what is the action you have to take and what is the outlook, like typically how an analyst used to do. And all of this, we used to have an orchestrator agent which connects all these agents together. So now what happened, once we had this system, what did it generate,right. It generates something, an information packet like this,right. It clearly tells you the signal,right. So first it says, maybe my brand's prescriptions have dropped 18% in some territory,right, in some time frame, maybe four weeks. It then tells you the reason. Why did it, why did it drop,right. The reason it says because a payer actually,right, the coverage for this drug has actually, they moved it to a lower tier. So for patients, it's expensive actually to, to buy this drug. The action, it says, okay, because doctors are writing less prescriptions, maybe send more sales reps to talk to doctors,right, and increase the number of, uh, prescriptions which you're writing. And then if you take this action, maybe your outlook, your sales performance is going to increase. ### Incoherent Output **Subbiah Sethuraman** [4:05] So all of this looks good high level, but then if you, if you look at it closely, it's not very coherent,right. The cause isright. It identified theright cause,right, because patients can't afford the drug,right. But the action it said, it didn't really focus on the payer part, the insurance part of it,right. It just said reps,right, uh, focus more reps actually,right. And then the outlook, because the action is wrong, the outlook is not going to match. So at each level, if you see, it has actually derived theright fact, but then there is no single agent which is owning, which understands the end-to-end picture basically. ### Root Causes **Subbiah Sethuraman** [4:40] So why did this happen, like why did it fail,right. So obviously it's not the LLM which failed,right. It's the way how we split the work,right, because we tried mimicking the analyst behavior and we did it. The first, first key issue is like a language model is actually det-determining your signals. So signals, like things like your sales drop, is a simple information which you can use statistical methods to actually go and fetch this information. You don't need a language model actually,right, to fetch this information. Second is as you have multi-agents, there is a lot of context handoff which is happening and context is actually getting lost at each of these handoffs. So for example, the driver attribution agent is actually determining theright cause, but then the next agent, the synthesis agent, is actually, is not able to understand why is that,right. The, the, the payers are finding the drug to be expensive. It's not understanding the weightage of an insurance coverage going, going down. So that is a key information,right, which is getting lost. And the last big piece is there is no shared understanding of the business domain knowledge for all these agents. All these agents don't understand metrics,right. So things like TRX, the number of transactions, the number of prescriptions which a doctor writes,right. What is the relationship between them? Why does it go up or down. So those are the, the reasons. So then what did we do, uh, so I called Abhilash to come and solve for this. ### Claude Code Rebuild **Abhilash Asokan** [6:01] Okay. Uh, thank you, Subbiah. So we had like three problems. Uh, so we were thinking how are we going to solve this. Uh, our first instinct was we'll go back to the drawing board. Uh, we'll start deciding it again. So maybe the topology was wrong, the skills were wrong, the tools were wrong, or maybe the, maybe we have to de-de-define a better handoff, design a better handoff or a better schema between agents. But we took a step back. We didn't do any of that. We, like all of us, we went back to Claude Code. So we opened a very pla-plain empty directory. I ran Claude Code, then give it just bash and the database. Then give it an actual signal which we identified, then started observing what it is doing. So while we look at what Claude does, we are able to figure out fix for all the three issues we discussed. The first, the first part. So what was happening was the agent was looking at data and deciding on a signal. ### Deterministic Pipeline **Abhilash Asokan** [7:05] Sometimes it applied some statistical methods, sometimes it barely looked at the data and said this is a signal. Sometimes it's actually a signal, sometimes it's a noise. This is something we don't want an agent to do. This is a completely deterministic workflow. So we separated it out from the agentic system. So we built a pure deterministic workflow with different statistical methods. We put guardrails, we put thresholds, we put prioritization. Everything happened before the agent even kickstarts. So we ran an automated pipeline which scans through the data, identifies signals for each of the KPI. Is anything happening with that? Any anomalies which is happening? Any trend which is very, uh, any trend which is breaking. Based on that, we identified a signal, we put it on a queue. The moment a signal comes to the queue, the agent wakes up. So the agent's job is to investigate, not to identify. The second part. So mainly the issue which we previously what Subbiah was mentioning, there is no coherence in the output which the agent is produced. So we started consolidating. We look at how Claude Code operates. It's able to do a lot of operations, so we started designing around that. ### Single Agent **Abhilash Asokan** [8:18] So it's repeatedly writing a function and querying database. So we give it a tool for that. So this consolidated the entire process into a single agent. That doesn't mean that we didn't do parallelism. We still do parallelism. What we removed is do we need distributed reasoning. We didn't want the judgment to be distributed between agents. That we wanted to consolidate to a single agent. So that was exactly what we did. Then occasionally we observed Claude Code, it's launching sub-agents dynamically for a very particular focused task. So we did the same thing. Because if suppose you want to understand if rep activity in a particular region, that's an investigation which you need to run. That you can still delegate to a sub-agent. You can get back the, uh, results back. Not the reasoning or the judgment. That is still controlled by the, by the main agent, but the investigation part of it, we did, we're delegating it to a sub-agent. So these are some of the things we kept based on observing what Claude was doing, Claude Code was doing. So that gave us a more lighter architecture than what was initially there. But still it doesn't solve the problem. ### Knowledge Graph **Abhilash Asokan** [9:25] That's why you see a knowledge graph in the diagram. So it still doesn't have the business context. It still doesn't understand all the entities, the domain, the KPIs, how do they relate to each other. So that's something we wanted to solve for. Because the agent was looking at data, looking at tables, then trying to infer the relationship that which was not scalable. And it often produced relationships which, which is not actually exist in the data. So what we did. So we've been in this, working in this field for a lot, lot, whole, whole lot of years. So we've been doing this for our clients. So we had a lot of domain experts who understand the pharma domain very well. We're doing commercial analytics for the clients. So we start, start with them, start building a knowledge graph. We start, start trying to map out the domain. So we are able to build a knowledge graph. We are able to identify different entities, their relationship between that. So if you see, there are like geographic entities, there are payers. How, how geography entity is connected to a payer or an account. How does this connect to a brand. Then from ba-brand or payer, how does it go to a KPI. How does a KPI relate to like a, uh, secondary KPI, tertiary KPI. How does one KPI drive another KPI. So we started mapping out all of this information and then creating our, our knowledge graph. Once we have the knowledge graph, then we wanted to let the agent to navigate the knowledge graph. So the knowledge graph is not just something the agent looks up for data. It is a control plane for the agent. So what do you mean by that. So the knowledge graph dictates what the agent can look into, what path it can take, what investigation hypothesis, uh, it can evaluate. So for example, if you when you do this analysis, the root of the problem, say something like your TRX is declining at a national level. ### Control Plane **Abhilash Asokan** [11:25] This could be when you do source localization, so that's our, uh, contact for how to find the where. It might happen within a region. It could be concentrated in a territory. It would be concentrated in a combination of a territory or a payer or an account. So there are a whole lot of dimensions which the agent needs to evaluate. Now, it's a lot of permission, permutation combination. So the knowledge graphs guides the agent on how to find where this is concentrated. **Guest** [11:50] Question again. What's the. **Abhilash Asokan** [11:51] Then the why, why part of it. **Guest** [11:53] Yeah. **Abhilash Asokan** [11:53] Why it is happening. When you figured out that, okay, something is concentrated at, say, the decline is concentrated at a particular region, now you need to figure out the why. This is where, where the, uh, KPIs and the relations com-comes in. Once the agent is able to narrow down the where, then it's able to go and figure out the, uh, figure out the why part of it, which KPIs is driving what. Graph access the control surface. So agent, every edge is a hypothesis. So the agent can go and evaluate that hypothes-hypothesis. Uh, it doesn't go outside of this. So that gives a more bounded surface for the agent to investigate. So like everyone was mentioning about loop, so we also build a loop. So what the agent does. So the agent first start with an entity. It goes to the graph. It looks at the neighborhood of the graph. Then it, it figures out the edges. So it's got some hypothesis. It'll go, it'll go back to the original data, evaluate that hypothesis, look at the actual numbers, then reason over it. Then it'll, it'll either find it, find it contradicting or it fi-finding supporting the, the evidence supporting the hypothesis. If it's supporting it, it started traversing through the graph. So this repeats this until it ran out of all the hypothesis or it's able to figure out the root cause. So this, this concludes this run. This run's and it'll be able to pro figure out the, uh, root cause of the problem. So once we build this, so maybe like after like 50 plus turns, a whole lot of tokens, it's able to produce something an analyst, uh, was able to produce maybe in three or four weeks in like maybe 20, 30 minutes. Thank you. **Guest 2** [13:45] So just to wrap it up, um, key takeaways. First thing is I think we should not be introducing human constraints or design constraints into architecture. I think let the architecture be derived actually. Number one. Second, I think any complex workflows will have deterministic parts and agentic parts. ### Key Takeaways **Guest 2** [14:02] Don't let agents actually run the deterministic part,right. So I think we need to break, break that off. The third, you need to have one agent,right, which owns the reasoning end-to-end,right. This agent can actually take the call to use sub-agents, tools, skills to actually spawn off other, other tasks basically,right. But you need one agent to own the, the reasoning. And the last, probably the most important, I think graph cannot be treated just as a lookup layer. I think graph has to be treated as a control plane which the agent uses to navigate and takes the next decisions basically. Okay. With that, I think thank you. Thanks. Thanks for attending this. --- This library is powered by PodHood (https://podhood.com), the podcast website platform.