Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain
Aug 12, 2026 · 20:02
Vivek Trivedy, lead of applied research at LangChain, argues that improving agents is fundamentally a data mining problem: ship agents, collect traces, then mine them to drive continual learning. He claims observability and continual learning are the same problem because agents operating in environments produce trace data, which is the substrate for all improvement. Trivedy details how LangChain sends agents to read other agents' traces to find good/bad interactions, detect degradation after compactions, and test counterfactuals like swapping GPT-5.5 for GLM 5.2. He shares that with Harvey on a legal benchmark, an open model matched Opus's trace judging at one to two orders of magnitude lower cost, achieved through harness engineering informed by traces. His rule for when to stop prompt tuning and start fine-tuning is feedback speed: harness engineering answers in about two minutes, so exhaust that ceiling first, then fine-tune to break through, then return to harness engineering. He…
