Harvey

Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain

Aug 12, 2026 · 20:02

Vivek Trivedy, lead of applied research at LangChain, argues that improving agents is fundamentally a data mining problem: ship agents, collect traces, then mine them to drive continual learning. He claims observability and continual learning are the same problem because agents operating in environments produce trace data, which is the substrate for all improvement. Trivedy details how LangChain sends agents to read other agents' traces to find good/bad interactions, detect degradation after compactions, and test counterfactuals like swapping GPT-5.5 for GLM 5.2. He shares that with Harvey on a legal benchmark, an open model matched Opus's trace judging at one to two orders of magnitude lower cost, achieved through harness engineering informed by traces. His rule for when to stop prompt tuning and start fine-tuning is feedback speed: harness engineering answers in about two minutes, so exhaust that ceiling first, then fine-tune to break through, then return to harness engineering. He…

Scaling up Continual Learning — Ronak Malde, Trajectory

Aug 12, 2026 · 23:03

Ronak Malde, founder of Trajectory and former Windsurf research lead, explains how on-policy self-distillation (OPSD) scales continual learning beyond GRPO's limits. He argues GRPO requires parallel rollouts and collapses feedback into one sequence-level score, like being handed 87 out of 100 on an essay. OPSD instead matches per-token log probs between a student and a teacher given privileged hints, optimizing the entire vocabulary and reducing tokens to solve tasks. Scaling to 120B models with 100+ tool calls reveals the 'but wait' problem—models hedge into 'maybe'—solved via step-level KL weighting, and hint leakage, countered with residual guidance. Trajectory applies this to production agent traces for Harvey, Decagon, and Rogo.

State of Startups and AI 2025 - Sarah Guo, Conviction

Aug 2, 2025 · 23:52

Sarah Guo of Conviction argues that AI's value creation is massive and early, with companies like Cursor reaching $100M ARR in 12 months and Harvey exceeding $70M ARR. She predicts that by end of 2026, AI agents will ship code directly to production, voice AI will replace text for most business communication, and inference costs will drop below a cent per million tokens. Reasoning is a new scaling vector unlocking higher-stakes use cases, and agent startups have increased 50% in the last year. Multimodal models from HeyGen and Eleven are already rocketing past $50M ARR. The model market is more competitive than ever, with GPT-4 costs falling from $30 to $2 per million tokens in 18 months and open-source like DeepSeek competing. Guo advises builders to focus on thick wrappers around LLMs, leveraging domain and workflow knowledge, and warns against generic text boxes: 'The prompt is a bug, not a feature.' Execution, not first-mover advantage, is the moat.

Scaling Enterprise-Grade RAG: Lessons from Legal Frontier - Calvin Qi (Harvey), Chang She (Lance)

Jul 29, 2025 · 16:40

Calvin Qi of Harvey and Chang She of LanceDB detail how Harvey builds enterprise-grade RAG for legal use cases, tackling massive scale (tens of millions of documents per corpus), spikey workloads, and non-negotiable privacy. They explain why domain-specific reranking is essential for ambiguous, expert queries like 'What is the applicable regime to covered bonds issued before 9 July 2022 under Directive EU 2019/2062?' and stress that evaluation-driven development—from expert reviews to automated metrics—is key to accuracy. Chang shows how LanceDB's AI-native multimodal lakehouse, built on the open-source Lance format, enables low-latency, high-throughput retrieval across billions of vectors (indexing 3–4 billion in under 3 hours) while supporting offline analytics, GPU indexing, and compute-storage separation. The talk emphasizes that creative data understanding, iteration speed via good evals, and flexible infrastructure are critical for scaling RAG in domain-specific, privacy-sensitive environments.

Layering every technique in RAG, one query at a time - David Karam, Pi Labs (fmr. Google Search)

Jul 29, 2025 · 20:22

David Karam of Pi Labs (formerly Google Search) walks through layering every technique in RAG, from in-memory retrieval to planet-scale search with 70+ corpus mix of token, embeddings, and knowledge graphs, jointly retrieved and re-ranked at 160,000 queries per second in under 200msec. He advocates a Quality Engineering Loop: baseline with simplest methods, analyze losses, then apply incremental techniques based on complexity-adjusted impact. Karam explains why queries like "falafel" are notoriously hard due to ambiguous intent, highlights failures of chunking documents, and shows when BM25 suffices, when relevance embeddings are needed, and when custom embeddings or domain-specific signals (price, popularity, user clicks) become essential. He covers query orchestration via fan-out, supplementary retrieval across back ends, and distillation for cost optimization, concluding that at sufficient complexity, problems must be punted to LLM or UX for graceful degradation.

From Text to Vision to Voice Exploring Multimodality with Open AI: Romain Huet

Jul 10, 2024 · 23:39

OpenAI's Romain Huet demonstrates GPT-4o's real-time audio, vision, and screen-sharing capabilities at the AI Engineer World's Fair, showing how the omni-model achieves human-like conversational latency, understands emotion, and can be interrupted naturally. He walks through live demos of ChatGPT Desktop recognizing drawings, reading book pages, and pair-coding a responsive travel app using Tailwind CSS. Huet also previews Sora's text-to-video generation and OpenAI's unreleased Voice Engine, which can clone a voice from seconds of audio and translate it into multiple languages while preserving the original speaker's tone. He outlines OpenAI's four focus areas: advancing textual intelligence, faster/cheaper models (GPT-4o is twice as fast as GPT-4 Turbo at half the price), model customization via fine-tuning, and enabling multimodal agents. The talk emphasizes that today's models are the dumbest they'll ever be and urges developers to build for a future where AI works across text, voice, and video.