Susheem Koul

FinOps for AI Agents: Who Spent All the Tokens? — Tisha Chawla & Susheem Koul, Microsoft

Aug 22, 2026 · 21:24

Microsoft engineers Tisha Chawla and Susheem Koul present TokenOps, a runaway token governance control plane for AI agents, arguing the agentic era's missing control surface sits between code and model calls, not at request-level gateways. The design has an out-of-band SDK with boundary annotations, a ledger, and a governor, plus a control plane that groups runs into segments, sets budgets, and applies policies. Enforcement comes in two flavors: halt is a circuit breaker, while steer uses a cost guard that watches budget consumed and token velocity and, predicting an overrun, injects an instruction to keep outputs succinct instead of killing the run. They show preview mode with enforcement off, then enforced halt and steering. Benchmarked on Browser Use and MetaGPT, the full policy suite cut average agent spend by about 78% and lifted run completion from 67% to roughly 96% versus throttling.

Your Agent Failed in Prod. Good Luck Reproducing It. - Tisha Chawla & Susheem Koul, Microsoft

Jun 29, 2026 · 14:10

Tisha Chawla and Susheem Koul of Microsoft argue that debugging production agent failures requires replayability, not bitwise determinism. They illustrate with a stock-selling agent that sold 1,000 shares instead of $1,000 worth due to a reasoning error, a problem temperature-zero decoding cannot fix because GPU non-determinism and batch invariance cause varied outputs. Instead of freezing model output, they propose recording every state transition at method boundaries using their open-source tool Chronicle. Chronicle annotates nodes to capture inputs, outputs, and metadata like model version, then enables deterministic replay by stubbing LLM calls during testing, letting engineers fix guardrails and verify with the same trace. They distinguish deterministic testing (for tools and guardrails) from behavioral testing (e.g., LLM-as-a-judge), advocating for replays over chasing API determinism to make on-call cycles bearable.