Claude Code

Building ambitious software — Jonathan Kelley, Dioxus Labs & Cognition

Sep 11, 2026 · 19:14

Jonathan Kelley, founder of Dioxus Labs, whose cross-platform Rust app framework now has nearly 37,000 GitHub stars and an estimated 200 million end users and was acquired by Cognition, argues that AI coding agents have made code cheap but quality remains the scarce resource, so architecture now takes most engineering time. He traces Dioxus's five-year effort building everything from scratch, including the Blitz rendering engine with a browser-grade CSS engine lifted from Firefox and the Subsecond hot reload engine that patches running native code in about 100 milliseconds. When agents got good at Rust, his team maxed out their Claude Code subscriptions and produced tens of thousands of lines that mostly never cleared the merge bar, a failure mode he calls becoming a slop cannon. He credits agents for deeply integrated Kotlin and Swift build plugins shipped in two to three weeks, release checklists, backports, and documentation accuracy, while noting they write tests for any API but rarely the right ones, though they excel at fuzzing harnesses. Rust's learning curve, once fought, is now a feature because agents absorb the borrow checker and edge cases.

The Spatial Harness: Bringing Agents to the Canvas — Max Drake, tldraw

Sep 10, 2026 · 18:52

Max Drake, product engineer at London's tldraw, argues coding agents excel because code is text in and text out, while canvas agents need engineering to understand and act in 2D space. tldraw, the whiteboard app and SDK behind canvases including Replit's, ships an MIT-licensed agent starter kit that teaches models to read a canvas from screenshot plus JSON and set their own todos, moving their viewport to explore. Fairies renders agents as figures you can throw and recolor; selecting several opens a group chat with an orchestrator assigning and reviewing work — you read ten agents' state at a glance, not a chat log. Last, a dependency graph of coding agents and a desktop build that lets agents script the editor — a colleague made it a window manager playing Pong with real windows.

500 Skills, Zero Fine-Tuning: LinkedIn's Playbook for AI Agents — Ajay Prakash, LinkedIn

Sep 9, 2026 · 20:25

Ajay Prakash, senior staff software engineer at LinkedIn, explains how Contextual Agent Playbooks make coding agents reliable at enterprise scale, turning an on-call alert into a mitigated incident in minutes. Early vibe coding failed: agents trained on open source hallucinated across LinkedIn's 1,000 internal repos. An internal MCP server with code search, docs, and Jira fell short: tribal knowledge sat in stale wikis, context overloaded, and sessions started from scratch. Playbooks, instructions published as tools, work when self-contained and split into small referenced pieces, and agents open PRs to refresh stale ones. Since MCP degrades past 30 or 40 tools, everything surfaces behind three meta tools: search, get schema, execute, now 1,300 tools, 600 playbooks, 8,000 daily users.

Build-Time vs. Run-Time: Why Dev Tools Fail in Production — Averi Kitsch & Prerna Kakkar, Google

Sep 9, 2026 · 20:26

Averi Kitsch, technical lead for MCP Toolbox on Google Cloud databases, and Prerna Kakkar, tech lead for Eval Bench at Google, argue build-time tools like natural language to SQL don't belong in production; run-time tools need structured SQL with preconfigured parameters to block injection and hallucination. They show a demo where an agent hit an error and deleted the table. Security hinges on the confused deputy attack and Simon Willison's lethal trifecta, shown via a triage agent that queried a salary table and posted results to a ticket. Their fix separates user, application, and agent identities, moves connection details into a YAML source, enforces read-only at the driver, and pins SQL behind prepared statements, with sensitive values like user ID bound by the application.

From coding to Knowledge work agents — Karan Vaidya, Composio

Sep 3, 2026 · 20:42

Karan Vaidya, cofounder and CTO of Composio, argues coding agents raced ahead not because models are better at code but because code already had six primitives knowledge work lacks: centralization, history, context, verification, governance, and reversibility. He walks each one, from the repo as a single source of truth versus a deal scattered across Salesforce, Notion, Gmail, Slack, and Zendesk, to git history against agents that start blank every time. He recounts pointing his own OpenClaw at hiring outreach that mass-emailed candidates and ended up on Twitter, noting every technical check passed while nothing tested whether the outreach should have gone at all. He cites a Meta alignment director whose email agent kept deleting messages until 200 were gone, showing prompts get compacted away and walls must live outside the agent. He concedes reversibility is hardest, since sent emails and wires cannot be recalled, so Composio uses sandboxes to catch mistakes before they reach the real world.

Tethered: Our Agents Are Us — Shu Fang, Two Sigma

Sep 3, 2026 · 21:10

Two Sigma's Shu Fang explains why every employee at the 25-year-old quant fund runs a cloud agent as themselves, not a service account — echoing the doubles in the film Us. A separate agent identity collapses fast: permissions drift, licensing doubles, and systems like Google Workspace refuse two identities on the same data. Agents instead run in per-person Kubernetes namespaces built for automated jobs, with a sidecar mounting the identity into the pod. Since agent and human share one identity, a header propagated like a trace ID recovers not just the actor but the replayable chain behind a result. For web access, they use Google's web grounding for enterprise — an index inside their network boundary — and deny native search and fetch tools, trading freshness (about 24 hours) for removing exfiltration and prompt injection risk.

Your Agent Just Authorized What?! — Jay Mok & Ben Coumes, Paypal

Sep 1, 2026 · 16:07

PayPal's Jay Mok and Ben Coumes tie agent authorization to three questions — did the human authorize it, is it allowed in scope, can you prove it later — answered by stakes and familiarity. Low stakes is the coding agent: allow/ask/deny permissions plus system logs, since actions revert. Medium stakes is money in a closed ecosystem — Evermind's shared vault plus OAuth scopes, with the amount-bound mandate and transaction logs settling disputes. High stakes: autonomous payments between strangers need a layered selective disclosure JWT per FIDO/AP2 — merchants verify checkout, processors verify the mandate. The approval token inverts the flow: users approve before an agent finds an item; PayPal returns a payload with amount, expiry and merchant, starting with Gemini.

Why Your AI Agent Needs a Wallet: USDC and Nanopayments — Harshal Bhangale, Circle

Sep 1, 2026 · 20:52

Harshal Bhangale, an engineer on Circle's Agentic Product team, argues that paying is where AI agents actually stall, and that Circle's USDC stablecoin and x402 protocol are the fix. He demos two identical Claude Code agents planning his trip to the FIFA World Cup final: the one without a wallet could only draft an email and admitted it had no way to call, while the wallet-equipped one paid for premium data, sent the email, and phoned him on stage to explain how to reach MetLife Stadium from his hotel. Card fees near 3% cannot sit on a one-cent call, he says, because agents consume in fractional amounts at high frequency, and sellers now meter slices of data instead of selling humans subscriptions. He cites roughly 24 million dollars transacted against paid API endpoints over x402 in 30 days, 99% settled in USDC. Blockchains alone fail since gas swamps microtransactions and shared block space brings unpredictable latency, so Circle's Nanopayments keeps settlement off-chain: funds sit in a smart contract, the agent signs cryptographic authorizations, and the seller relays them for confirmation in a few hundred milliseconds, with the wallet enforcing spending caps instead of human…

From Tokenmaxxing to Trusted Throughput — Mingsheng Hong, Ironclad

Aug 29, 2026 · 23:04

Mingsheng Hong, VP of Engineering focused on AI at Ironclad, argues token dashboards are smoke detectors, not leaderboards, and the goal is trusted throughput—merged, customer-validated PRs weighted by complexity—not token minimization. Don't cut cost before measuring value; Ironclad's metric evolved from lines of code to open PRs to merged PRs to merges weighted by an LLM-assigned complexity score, because a ten-line concurrency fix beats boilerplate. He identifies review and CI as new bottlenecks, where slow pipelines encourage giant batched PRs; he recommends AI as first-pass reviewer, killing flaky tests, capping agent retry loops, and measuring ready-to-merged time. He advises prompt caching, context pruning, and buying infrastructure while building context-specific playbooks in-house.

How do you diffuse AI into the real world? — Varun Shenoy, Long Lake

Aug 28, 2026 · 17:46

Varun Shenoy, co-founder of Long Lake, argues that AI diffusion into real-world services requires operator-owners, not vendors, and explains how his firm buys and runs 35 services businesses — including a $6.3 billion take-private of American Express Global Business Travel — to make agents complete economically relevant tasks. He frames adoption as a generational shift like electricity, and details a ladder from copilots to async agents to AI coworkers, earned by proving value. Long Lake represents knowledge work as code, captures traces and ground truth from work like roof repairs and book closing to build evals, and treats continual learning and enablement as one snowball loop. He concludes that co-designing software with a 100-year-old firm cannot happen over Zoom; it requires showing up in person, running conference stands, and asking questions on a mountain bike.

AI Evals for Cross-Functional Teams — Nachiket Paranjape & Swaroop Chitlur Haridas, DoorDash

Aug 28, 2026 · 16:11

DoorDash GenAI platform team's Swaroop Chitlur Haridas and Nachiket Paranjape argue evals stopped being an engineering harness and became a cross-functional effort spanning strategy and operations, product, operations, and engineering. They describe a continuous loop — trace, sample, annotate, calibrate — and an API-first platform that lets non-engineers use Codex or Claude Code to vibe code their own annotation UIs. Judge prompts are calibrated self-serve through a UI showing original and optimized prompts side by side, using JetPa, so product managers and operators can run optimization loops without engineering. Per-annotation cost fell sharply at DoorDash scale, and variation in who owns judge prompts across teams is treated as a sign the org is still learning.

How to Generate Mergeable Code with a Context Engine — Peter Werry, Unblocked

Aug 27, 2026 · 18:36

Peter Werry of Unblocked argues that access to information does not equal understanding—organizational context is the bottleneck for AI agents. He likens agents to expert engineers resetting knowledge every task, falling prey to radiology's 'satisfaction of search'; million-token windows distract. Werry demos the context engine generating an architecture diagram, then runs the same optimization plan in Claude Code twice: with it, under a dollar and about a minute; without, roughly double the time and more cost, because later steps loop on wrong assumptions. He also shows a review agent boosting comments by seniority, a drop in flagged issues traced to a Slack thread, and open-source tools: a GitHub-history query engine and a social graph showing thin review coverage.

How Anthropic Builds: Lessons from Labs — Mike Krieger, Anthropic

Aug 27, 2026 · 26:11

Instagram co-founder Mike Krieger, now an individual contributor at Anthropic's Labs, argues for being 'unreasonable' with Claude—he had it port a few hundred thousand lines of Python to TypeScript over a weekend. He attributes timid asks to first-gen AI's boxed tools; cites Instagram-era habits (pre-measure everything, knobs); and explains tagging Claude in Slack as multiplayer delegation. Labs runs two-week 'persevere or pivot' reviews, bet leads manage nobody, and code review is bottlenecked by comprehension, so 2,000-line PRs travel with artifacts explaining intent and tradeoffs. He also sketches Claude Design's future, urges unship of product complexity, and advises startups to bet on verticals like finance. On burnout: no job is so important you cannot be offline for two days.

The Agent Behind the Curtain: Building the Oz Cloud Agent Platform — Safia Abdalla, Warp

Aug 22, 2026 · 20:51

Safia Abdalla, an engineer at Warp, explains how her team built the Oz cloud agent platform on the principle that platforms should absorb complexity before the user sees it. Agents run in managed or self-hosted sandboxes; the platform supports multiple harnesses consistently. Users can orchestrate sub-agents by prompt or via an API exposed across the stack, and non-engineers used the SDK to build Slack bots for triaging social mentions. Abdalla recounts open-sourcing Warp: stars grew from about 20,000 to over 60,000, with thousands of PRs; agents triage issues and review every PR, so humans only see high-signal ones. She rejects 'software factory' as losing the people in it, offering a potter's workshop: stations, sourcing, verification, and an observable, improvable, cost-effective system.

Building Agents Is Trivial Now, Context Is the Next Frontier — Jeff Ng, Unblocked

Aug 21, 2026 · 13:22

Jeff Ng, founding engineer at Unblocked, argues building agents is trivial but missing context makes them confidently wrong. Cloud primitives and frameworks have removed plumbing that six months ago took a team a quarter. In a demo, an agent enriching a Linear ticket recommended re-enabling async dispatch, missing that a support engineer had disabled it after an outage; it lacked the Slack thread and postmortem. Background agents fail silently when no human supplies that context. Ng's context engine connects docs, code, tickets, and conversations, reconciles conflicts, and returns a synthesized understanding scoped to permissions. MCP provides access, not understanding; rerunning the same agent grounded in the engine flips its recommendation from repeating the outage to preventing it.

The Missing Layer: Design Taste in AI Agents — Hassan El Mghari, Together AI

Aug 21, 2026 · 14:10

Hassan El Mghari, developer experience lead at Together AI, argues design taste, not model capability, decides whether AI apps reach millions or read as vibe-coded slop. He ships about 10 apps a year — a few with millions of users — and credits design for that reach. His design skill Hallmark, launched six weeks ago to 10,000+ users, turns AI slop tells like purple gradients and italic headers into gates and supplies themes, since strong inspiration produces better work. He starts in Codex or Claude Code and iterates with cheaper open-source GLM 5.2, whose landing page was nearly indistinguishable from Opus 4.8. He advises saving screenshots as references, using longer prompts from voice notes, one or two features per prompt, and a skill file or AGENTS.md — but agent output is only a base.

Your Fine-Tuned Model Is Tech Debt: A 50x ROI House of Cards — Dan Bjornn, Lease End

Aug 20, 2026 · 16:39

Dan Bjornn, senior data scientist at Lease End, explains why his team's fine-tuned LLM for customer intent—despite bringing in $12 million at a 50x ROI—became tech debt. The fine-tuning pipeline took a week per retrain, with training itself the shortest step, and each fix caused regressions, so bugs were triaged by tolerable customer pain. He calls this the calcification tax: the model locked them into one provider and an outdated architecture, preventing upgrades. The rebuild swapped the tuned model for skills, prompts, and context on a model-agnostic framework, letting fixes ship in under an hour as uploaded files. Accuracy went up, cost per message rose, but total cost fell. Bjornn concludes fine-tune only when you cannot call a frontier model, and even then the decision must beat the tax.

The Era of Compound Engineering — Kieran Klaassen, Every/Cora

Aug 20, 2026 · 20:38

Kieran Klaassen of Every describes compound engineering, the system behind his AI email client Cora, built alone without writing a single line of code this year. He moved through bottlenecks—code, plans, judgment—then built memory so the next feature is easier. His rule: spend 50% building the feature and 50% teaching the system what it got wrong; stored solutions are more token-efficient than corrections and research. The loop is a human-AI sandwich—brain on to decide problems, autonomous middle overnight, brain on to raise the bar. His open-source Compound Engineering Plugin, used by hundreds of thousands daily, turns backlogs into argued ideas and automates work via /LFG. The bet: implementation gets cheaper, judgment does not.

Prototyping as Leadership: How a CTO Ships with AI Agents — Hursh Agrawal, The Browser Company

Aug 20, 2026 · 18:18

Hursh Agrawal, CTO and co-founder of The Browser Company, argues AI coding agents let leaders keep building: despite 15+ meetings, 7 direct reports, and a toddler, he ships 2–10 PRs/week. He says frontier models turn over every three months, so hands-on building is the only way to judge them and show engineers working prototypes. His method is an overnight loop: a coworker agent gathers Slack/Jira/Notion context into a prompt at 5pm, coding agent runs for hours, and a morning hour reviews tests, CI, and AI code review. He details three overnight uses: building features, hill-climbing evals from feedback JSONs, and training custom models like a PII classifier on AWS. He warns leaders to avoid critical path work and to rely on trustworthy CI, feature flags, a prototype branch, and readable PRs.

The Last Human Code Review: Building Trust in AI-Generated Code — Itamar Friedman, Qodo

Aug 20, 2026 · 18:54

Itamar Friedman, CEO and co-founder of Qodo, argues the bottleneck in AI-generated code is now context, not models; teams shipping code faster than humans can review are already inside the problem. He splits leaders into two camps — trust every line vs. ship bugs and fix fast — and says code-review benchmarks barely moved, proving context, not reasoning, matters. Qodo's answer is a context engine that codifies tribal knowledge — instruction files, Slack threads, and senior developers' heads — into interfaces humans can audit and agents can consume, shifting review from a single PR to a graph of PRs and contracts. Qodo targets 2027 with zero critical bugs in production, moving from artificial intelligence to artificial wisdom.

Healthcare’s Agent Bytecode: X12 as the Harness for AI Agents — Vasant Kearney, Onlay

Aug 19, 2026 · 20:25

Vasant Kearney of Onlay argues that X12, the healthcare claims data standard, is the harness for AI agents: every step of the claim lifecycle—from eligibility (270) through acknowledgment (999) to payment (835)—has an X12 correspondence, and an agent calling a payer or driving a portal is emitting the same transaction by another route. He says phone, portal, and X12 surfaces are built by different teams and can all agree on the wrong answer, so no surface is ground truth; his system keeps a semi-correct internal X12 representation, correct only until downstream evidence says otherwise. Enterprise memory must live in a database rather than local disk, and swapping in a stronger model is not automatically better inside a system built around the old one—evals and validation must be redone. The goal is cutting insurance costs and improving patient experience, and that means being AI pilled but also AI skeptical.

Generative Video at the Speed of Light — Keegan McCallum, uRun

Aug 18, 2026 · 8:43

Keegan McCallum, founder of inference provider uRun, argues that generative video's interesting axis is no longer quality but efficiency and long-horizon generation. He shows Helios, a distillation of Wan 2.1 14b, generating clips in real time for ~one-hundredth the cost of a slower frontier-quality clip, and notes at least 40 real-time/long-horizon models released this year. Ten dollars buys three hours of continuous generative video, and fifty buys fifteen; this unlocks magic-mirror webcam transformations, visual mediums for people who don't think in text, and content creation where you steer a generation in under a second. The hard part is serving: global GPUs, WebRTC with ICE/TURN, and synchronized streaming pipelines; uRun is building a React component, Python runtime, and MCP/CLI.

The Next Game Engine Won't Have a Manual — Arturo Nunez, Nereu

Aug 18, 2026 · 19:33

Arturo Nunez, creator of the AI-assisted game engine Nereu, argues the next game engine won't have a manual: instead of coding, users describe intent. Drawing on a decade at Unity, he built Nereu around an asset tag system (ATS) inspired by Entity Component System, where everything is an asset tagged with intent (e.g. character, animated, double jump) and systems query tags—so a building can be tagged drivable and dropped into a Mario Kart race. The assistant, Bibi, helps users get unstuck rather than one-shot finished games, and context is assembled using level of detail: assets near the edited object get full tag values, distant ones collapse to position and type, and hundreds of grass pieces are omitted. Nereu also uses vision models to tag 6,000–7,000 assets, runs in the browser with JavaScript, and is in closed alpha. Nunez contrasts this with world models, which he says remain far from real-time 4K 60 fps games.

How to Kill the Code Review — Ankit Jain, Aviator

Aug 17, 2026 · 16:26

Aviator co-founder Ankit Jain argues code review isn't just about catching bugs: it carries knowledge sharing and mentorship, and that half must survive even as AI writes and reviews code. He says reading line by line is already over — over 30% of changes merge without review. He calls spec-driven development 1970s waterfall because intent lives in the prompts, which teams discard when the PR opens. His fix: capture agent sessions, turn decisions into acceptance criteria, pair them with an 'AI slop registry' of recurring review comments, and generate test plans a verification system runs against a live preview. Reviewers review intent and evidence, not the diff. Homework: mine your last 1,000 review comments; he pitches Aviator's new Verify pilot.

Bringing agents onto the world wide web — Paul Klein IV, Browserbase

Aug 14, 2026 · 18:26

Paul Klein IV, founder of Browserbase, argues browser agents are no longer held back by models; the missing piece is engineering. He details a three-part harness: multimodal agents that write code and intercept network requests, skills and memory that compress page context, and consistent infrastructure, mocking the 'SOC 2 compliant Mac Mini at scale' problem. Klein says the web itself must improve for agents, pointing to accessibility trees, Chrome's new Web MCP, and unsolved problems of agent login and trust certification. He contends the payoff is the logistics company in Singapore, the bank in South Africa, and the lumber factory in Mexico running on PHP forms, not San Francisco, and introduces Browserbase Agents as a battery-included solution.

Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain

Aug 12, 2026 · 20:02

Vivek Trivedy, lead of applied research at LangChain, argues that improving agents is fundamentally a data mining problem: ship agents, collect traces, then mine them to drive continual learning. He claims observability and continual learning are the same problem because agents operating in environments produce trace data, which is the substrate for all improvement. Trivedy details how LangChain sends agents to read other agents' traces to find good/bad interactions, detect degradation after compactions, and test counterfactuals like swapping GPT-5.5 for GLM 5.2. He shares that with Harvey on a legal benchmark, an open model matched Opus's trace judging at one to two orders of magnitude lower cost, achieved through harness engineering informed by traces. His rule for when to stop prompt tuning and start fine-tuning is feedback speed: harness engineering answers in about two minutes, so exhaust that ceiling first, then fine-tune to break through, then return to harness engineering. He…

Lessons from Studying Every Memory System — Shlok Khemani, Independent

Aug 12, 2026 · 19:31

Shlok Khemani, an independent researcher, reverse-engineered the memory systems of ChatGPT and Claude to show how consumer AI personalization evolved from user-managed fact lists to background-updated running profiles. He details that ChatGPT's v2 profile is ~4,000 tokens of dense keyword clues, updated every few days, while Claude's is 1,000 tokens of full sentences, refreshed daily and visible in settings, arguing memory is a function of compute with trade-offs between serving and update costs. He highlights a false memory where ChatGPT claimed he visited Turkey in 2025 (he went to Thailand), blaming a product problem—no system notices conflicts or reasons over email and calendar—rather than a technology limitation. He concludes that memory cannot be outsourced, continual learning already happens outside model weights, and the real bottleneck is context gathering across fragmented products.

Designing Agents (The Floor Is the Frontier) — Ben Hylak, Raindrop

Aug 12, 2026 · 19:46

Ben Hylak, CTO and co-founder of Raindrop, argues that most eval advice is stuck in the chatbot era and that agents have effectively infinite issues, so the real question is which ones matter—measured by when an issue started and what share of users it hits. He reframes agent quality around raising the floor (the worst thing an agent can do, like recommending a competitor or deleting data) rather than the ceiling, and says evals belong in your repo as code tests, not prompt playgrounds, because the harness is the product. He offers three tactical lessons from Raindrop: clusters are not issues because boundaries drift and you don't control them; code mode scales to traces, letting you write classifiers and run them in a sandbox at production volume; and agents are poor at anomaly detection but good at investigating anomalies you surface deterministically, like keyword spikes. He also notes that continual learning is rare in the real world, and that your approach should depend on user…

Scaling Compute on Context — Jack Morris, Engram

Aug 12, 2026 · 19:42

Jack Morris of Engram frames scaling compute on context as the pursuit of depth in AI, contrasting it with the breadth of public-data pre-training. He argues models trained on public data know nothing about your emails, meetings, or company, and that with a fixed private corpus, compute is the only scalable axis. He critiques naive fine-tuning (loss 0.00001 on 10K financial reports then collapse), KV compaction, on-policy distillation, and synthetic continued pretraining, noting each hits a synthetic data wall. The goal is self-improvement like AlphaGo, where better models generate harder training questions, enabling indefinite compute scaling on your context.

Claude Managed Agents and Evolution of Agentic Surfaces — Gagan Bhat & Isabella Kai He, Anthropic

Aug 11, 2026 · 31:24

Gagan Bhat and Isabella Kai He of Anthropic's Applied AI team explain how agentic surfaces evolved from the Messages API to the Claude Agent SDK and now Claude-managed agents, arguing that harnesses encode assumptions about model limits that go stale as models improve. They detail decoupling the brain (agent loop) from the hands (tool execution sandbox), which cut time to first token by 60% at P50 and over 90% at P95, and made failures recoverable via durable session logs. The session log also powers observability, context recovery, and 'dreaming,' a batch process that rewrites agent memory for self-improvement. They cover production lessons: keeping credentials in vaults, self-hosted sandboxes for VPC control, MCP tunnels for private servers, and the 'outcomes' feature that uses a grader agent to enforce success criteria.

How Codex Works — Dominik Kundel, OpenAI

Aug 10, 2026 · 20:55

Dominik Kundel, an OpenAI engineer, explains the internals of the Codex agent harness, which is open source under Apache 2 and written in Rust. He details how context construction caps available skills at 2% of the context window and uses deferred tools with tool search to manage size and cost. For actions, Codex relies on an apply patch tool for file edits, a shell tool with ripgrep, and sandboxes: seatbelt on macOS, bubblewrap on Linux, and a custom open-source Windows sandbox. To reduce approval fatigue, an auto-review subagent with read-only permissions judges actions against user authorization and risk taxonomies. Speed improvements come from websocket mode in the responses API, which sends only changed items instead of full state, crucial when GPT 5.3 Codex Spark hit 1,000 tokens per second on Cerebras. Long-horizon goals work via a continuation prompt until the model calls an update goal tool, favoring concrete verifiable objectives, and auto compaction maintains performance…

Multiplayer agentic engineering — Arjun Singh, Superconductor

Aug 9, 2026 · 18:44

Arjun Singh explains how Superconductor enables multiplayer agentic engineering by making agents model-agnostic, cloud-isolated, and reachable from Slack, desktop, and GitHub as one shared session. He argues agents should run in a configurable network sandbox for least privilege, letting non-technical staff trigger real work without dev setups; a meeting bot left in a Google Meet at their expo booth picked up a passerby's idea, opened a ticket, and added acceptance-criteria fields. He advises benchmarking agents on your own codebase because SWE-bench is Python while they are Ruby on Rails, citing one month: 10.5 billion tokens, 3,300 Claude Code runs worth about $10,000, and Codex running four times as many sessions for less money. Takeaways: sandbox your code, integrate agents into human interfaces, stay model-agnostic.

Benchmarking Coding Agents on New vs Legacy Codebases — Denys Linkov, Wisedocs

Aug 8, 2026 · 18:08

Denys Linkov of Wisedocs, whose medical-claims ML pipeline ran across ten legacy repos, argues his team's six-month monorepo refactor was worthwhile even as coding agents improve fast. A refactor task that took o3 three hours and ten major mistakes now takes about one-fifth the time: Sonnet 4.6 needed one extra iteration, Opus 4.8 nearly one-shot it. Yet GPT 5.5 extra high 'completed' the job in 10 minutes 22 seconds, writing 2,000 lines of scaffolding with models missing and admitting no deployment or bootstrap command. So Linkov reads METR's task-length curve at 80-90% success, not 50%; an hour-long agent run at coin-flip odds wastes the hour and your attention. The payoff was social too: commit velocity never flattened, months-long features ship in under a week, and developers now volunteer across the monorepo.

Anthropic's CCA Exam as a Field-Guide for Agentic Engineering — Frank Coyle, UC Berkeley

Aug 8, 2026 · 20:08

Frank Coyle uses Anthropic's Claude Certified Architect exam as a field guide for agentic engineering: anti-patterns—knowing what not to do—are the key to agent design. In a customer support loop, the anti-pattern is calling the model and using its response directly; the model cannot execute tools, only hands back parameters, so branch on the stop reason, which also flags partial answers when tokens run out. For multi-agent research, loading one agent with every tool is the carpenter with plumbing gear; instead specialize subagents with one or two tools and give each only its own slice. A critic agent gets the claim and evidence but not the reasoning, to avoid groupthink, and subtask output is forked so only summaries return, with compaction past 150,000 tokens. Batch mode runs the same work at 50% lower token cost if you can wait 24 hours.

Jev CEO: I made ChatGPT, now I'm building what's next

Jul 31, 2026 · 18:05

Diogo Almeida, a GPT-4 co-author and founder of TypeSafe AI, argues that RLHF optimized models for human approval, making them superb assistants but unreliable for autonomous work. He contrasts assistance with automation, noting RLHF's reward model encourages confident overpromising — like ChatGPT praising an audio file of farts as music. The next era is not Claude Code (still assistance-native), but real automation via RLVR-style methods focused on calibrated decision-making, echoing Sutton's bitter lesson that the task matters more than data. He defends pre-training as phenomenal, blaming post-training's asymmetric reward model for hallucinations. TypeSafe is rebuilding the AI stack for reliability and automation.

The Base Model Is Dead — Varun Singh, Arcee AI

Jul 31, 2026 · 17:45

Varun Singh, pre-training lead at Arcee AI, argues the base model is dead: it no longer just mirrors web text but must carry the prior that reinforcement learning builds on. He traces how web text fell from 85% of GPT-3's mix to 15% in MAI Thinking 1, with code and STEM dominating, and how Nemotron 3 Ultra pulls SFT-style Q&A data back into pre-training. Synthetic rephrasing, as used in Arcee's Trinity Large and Kimi K2, upsamples information to teach task shapes early. He warns that without post-training-flavored data early, MoE load balancing can break when SFT distributions differ, citing MAI's cranking of the balancing coefficient. He frames training as supervised learning vs RL, noting RL compute now rivals or exceeds pre-training, as with Compose 2.5, and argues the base model's job is to provide atomic skills for RL to compose.

Morgan Stanley's ALPHALAB: Multi-Agent Research Across Optimization Domains — Brendan Rappazzo

Jul 29, 2026 · 20:07

Morgan Stanley's Brendan Rappazzo presents AlphaLab, an open-source multi-agent system that automates quant research by having agents write code, set up backtests, and run experiments, arguing that the lasting human role is designing verifiable environments like a private Kaggle. The system uses a strategist agent that proposes experiments and worker agents that implement them, managed via a Kanban board, and skipped off-the-shelf frameworks to maintain control. Rappazzo reports real improvements found internally, including a top 12% finish in a Kaggle competition fine-tuning Nvidia's Nematron model. He emphasizes that the key is building good evals and environments, which encode enterprise expertise, and that the ultimate goal is a self-improving system where the auto research optimizes itself.

The Model Was Right. The Harness Failed. — Vinoth Govindarajan, OpenAI

Jul 29, 2026 · 18:26

Vinoth Govindarajan of OpenAI argues that most agent failures are harness failures, not model failures. Using OpenClaw as a case study, he details five failure shapes: state hole (delivered but not remembered), overlapping writers (last write silently erases previous), dangling tool call (run waits for an event that never arrives), approval drift (expired approval blocks later work), and missing edge proof (internal success but user didn't see result). The through line: a model proposes, the harness commits, and the receipt proves it. He prescribes three invariants—own the state, order the mutation, prove the action—and a five-question run receipt audit: what woke it up, what state did it inherit, what authority did it use, what executed, and what evidence survived.

How Forward Deployed Engineering is done at Cognition — Jia Wu

Jul 28, 2026 · 17:38

Jia Wu, deployed engineering lead at Cognition, argues that their forward deployed engineers measure outcomes—not token usage—by embedding Devin in customer environments and delivering measurable productivity gains like an 82% reduction in delivery timelines. The job involves deeply understanding customer problem spaces, mapping Devin’s capabilities to highest-leverage work, and feeding product feedback from deployments. Wu distinguishes Cognition from single-point tools by claiming organizations achieve 10x faster output, not just individual engineers. He cites anonymized case studies: 150% headcount increase in three months, double the PRs compared to single-point tools, and one-third the timeline for an ETL migration at a bank. Wu insists that as coding becomes commoditized, deployed engineers must prioritize business and people skills, relentlessly tying into customer success and communicating back to the product roadmap.

Loop Engineering from First Principles — Kyle Mistele, HumanLayer

Jul 25, 2026 · 17:57

Kyle Mistele argues that the fix for AI-generated 40,000-line pull requests is not a better prompt but a better loop, borrowing from control theory: a sensor measures the gap between current and desired codebase state, a controller picks the smallest incremental change, and an actuator agent applies it using hand-written golden patterns. Mistele illustrates with HumanLayer's own loop that migrates their RPC API to Effect one procedure at a time, using AST grep as a deterministic sensor, a controller that selects the smallest unmigrated procedure, and an actuator agent gated by deterministic CI running a single iteration per day. The loop tracks its own PRs in version control, refuses to stack a new change while an earlier one is still open, and includes a feedback file and comment trigger for humans to re-steer it. Mistele concludes that this design makes the code incrementally better, readable, and verifiable, solving the problem of unreadable mass-generated code.

From Signal to PR: Anatomy of a Self-Improving Agent — Jason Lopatecki, Arize

Jul 24, 2026 · 20:36

Jason Lopatecki demonstrates how Arize's Signal agent transforms observability from human-clicked dashboards into telemetry for self-fixing systems. The key unlock is pulling production traces and logs as files into a repo—Claude Code works magically with files, not dashboards—so the agent sees the exact code path software took. Signal runs periodically or event-based in a sandbox (VPC-deployed for Uber, Booking), using composable skills to gather context, find root cause, and create a PR. Lopatecki argues you should trace and log ten times more because agents can read that smoke. For the question 'why not just point Claude Code at data?', he explains skills must be well-designed to fetch and format the right data into files. On evals, they run as LLM judges layered on production traces, pre-processing information for the agent to catch known failures and create new evaluators.

Why Software Factories Fail

Jul 23, 2026 · 19:18

Dex Horthy argues that the failure of lights-off software factories, including his own July 2025 experiment, is not a skill issue but a model training problem: coding models are reinforced only on passing tests, not on maintaining codebase quality, leading to slop code and outages. He explains that Claude Code succeeded where earlier CLI agents did not because it was the first model trained against the harness it ships in, optimizing for tool calls in an agentic loop. However, maintainability cannot be verified by current benchmarks like SWE-bench, which use binary test-pass rewards and ignore architectural degradation that only appears months later. Horthy advocates turning the lights back on—keeping human code review—but moving faster by investing upfront in product review, system architecture, program design (types and call graphs), and vertical slices. He claims thirty minutes of alignment saves hours of review, turning PR review from slop into a joy, and that this approach lets engineers still ship fast while owning code quality.

Why We Killed Our Multi-Agent Pipeline — Subbiah Sethuraman and Abhilash Asokan, ZS Associates

Jul 23, 2026 · 15:00

Subbiah Sethuraman and Abhilash Asokan of ZS Associates explain why they killed their multi-agent pipeline for pharma commercial analytics: the system produced incoherent output because no single agent owned end-to-end reasoning, domain knowledge was missing, and LLMs were used for deterministic signal detection. Observing Claude Code in an empty directory, they rebuilt with a deterministic pipeline that detects signals before the agent wakes up, consolidated to a single agent that owns reasoning and spawns sub-agents only for focused lookups, and added a knowledge graph as a control plane where every edge is a hypothesis the agent tests against data. The new system does in 20 minutes what an analyst did in a month, achieving bounded search and coherent output.

Active Graph Agent Runtime (BabyAGI 4) — Yohei Nakajima, Untapped Capital

Jul 22, 2026 · 17:34

Yohei Nakajima presents ActiveGraph, an event-sourced graph runtime that flips agent architecture around an immutable event log instead of the LLM, enabling native replays, rollbacks, and forks. He explains how behaviors react to graph changes and emit events, policies control which modifications require human approval or contradiction checks, and packs bundle object schemas, tools, and LLM behaviors. Nakajima demonstrates self-improvement loops that fork the agent, propose patches, run sandbox tests, and accept changes only if accuracy increases. He shares surprises: AI writes this architecture better due to decades of training data on blackboard and Kafka patterns; debugging shifted to querying the ActiveGraph DB; long running agents resume from failures instead of restarting; and his ActiveGraph Lab found a bug in its own code and opened a PR. The episode argues that long-running agents need an experiential world model derived from their own logs, not just reasoning capability.

Claude for Long-Horizon Tasks — Lance Martin, Anthropic

Jul 22, 2026 · 25:19

Lance Martin from Anthropic discusses how Claude's increasing task horizon enables asynchronous agents through decoupled architecture, verifier loops, and self-improving memory systems. He explains the shift from short task horizons (10–20 minutes) to 12+ hours, necessitating decoupling the brain (harness) from hands (sandboxes) for reliability and security. Verifier loops using separate contexts allow models to self-correct, demonstrated on the Parameter Golf benchmark with Opus 4.7. Memory systems inspired by human dreaming correct errors in-band, as shown in a Pokémon example where dreaming prevented repeated failures. Finally, org-level harnesses like Claude Tag provide shared identity and context for multiplayer proactive agents.

Every Harness Will Become A Claw — Sam Bhagwat, Mastra

Jul 21, 2026 · 15:36

Sam Bhagwat, founder/CEO of Mastra, argues that every developer harness will inevitably evolve into a 'claw'—an always-on, self-improving agent that takes initiative. He defines the journey from basic LLMs to agents, then to harnesses (durable, planning-capable tools like Claude Code), and finally to claws that add external event listening, heartbeat-driven wakeups, and continual learning via auto-skill generation. Drawing on Steinberger's law (a play on Zawinski's law), Bhagwat predicts a future shakeout where only a handful of claws survive, just as mobile platforms consolidated around a few apps. He urges builders to equip their agents with the full capability stack—durability, planning, parallel subagents, session persistence—to avoid being displaced in the coming consolidation.

HTML Is All Agents Need — James Russo, HeyGen

Jul 21, 2026 · 15:13

James Russo, software engineer at HeyGen, argues that HTML, CSS, and JavaScript are the native languages of LLMs and all agents need to create great videos—leading to HyperFrames, an open-source framework that turns agent-authored HTML into deterministic MP4s. By letting the small Gemini 3 Flash model author code first, they ensured the thinnest wrapper won, with only a few data attributes for timing. Rendering works by freezing the browser clock and seeking frame by frame, ensuring everything loads before capturing each frame—enabling deterministic video from any web technology (Three.js, WebGL, Lottie). HyperFrames skills focus on taste and video craft rather than teaching frameworks, raising the floor for single-shot output. The framework has already rendered over 1.3 million videos in 90 days from 267,000 creators, with 15,000 daily renders and 32,000 GitHub stars—proving the approach at scale.

It's 10pm. Do You Know Where Your Agents Are? — Kim Maida, Keycard

Jul 20, 2026 · 23:02

Kim Maida of Keycard argues that standard API keys dangerously overprivilege AI agents, enabling incidents like a night-shift agent dropping a production Postgres database because its kitchen-sink credential allowed it. Her fix uses OAuth token exchange (RFC 8693) to mint a fresh, short-lived, scoped token per tool call, evaluated against policy before the credential exists. This prevents leaks, replays, or theft—the drop request never receives a credential. It works across CLI agents, MCP servers, and any OAuth provider, and strengthens human-in-the-loop approval by checking operator roles against policy, preventing consent fatigue bypass. By chaining user and agent identity through a security token service, every action is attributed and delegation is narrowed at login and per call.

When Agents Meet Physical Data: The Other Physics of Agent Harnesses - Dmitry Petrov, DataChain

Jul 20, 2026 · 27:33

Dmitry Petrov of DataChain argues that AI agents fail on unstructured physical data because their intuitions assume cheap recompute, while large-scale video, sensor, and robot data requires a different harness. He cites Anthropic's finding that agents achieve only 21% accuracy on data projects without specific data harnesses, and OpenAI's need for six layers of context even on structured data. Petrov demonstrates DataChain, an open-source Python framework that uses Pydantic schemas to turn messy binary files into queryable databases, an execution engine for distributed processing, incremental checkpoints to avoid recomputing on failure, and a knowledge base of datasets and source code so agents can answer follow-up questions in seconds instead of reprocessing terabytes. In a live demo with Claude Code analyzing 90 dashcam videos, the harness took 24 minutes to extract 100,000 object records, then instantly answered 'how many clips have people?' without rerunning inference. Petrov emphasizes that the key is organizing metadata into star schemas and sharing data lineage across teammates so no one pays the compute cost twice.

Skills are the New SDKs - Elvin Aghammadzada, DataRobot

Jul 20, 2026 · 26:40

Elvin Aghammadzada of DataRobot argues that enterprise AI platforms must become 'teachable' through a skill layer—versioned, task-specific packages that encode operational knowledge for coding agents. He contends that context windows degrade after 25% usage (a phenomenon called 'context rot'), making progressive disclosure via skills critical for maintaining agent performance. Skills expose only metadata (<100 tokens) until activated, while MCP servers handle heavy resource isolation; skills can also self-modify and spawn MCP servers. With 26+ platforms (Claude Code, Codex, Copilot) supporting skills and 85,000+ published, the ecosystem is creating a 'fluency moat' where platform value compounds with each skill. However, LLM-generated skills currently hurt performance, and marketplaces lack verification controls, echoing early NPM risks. The episode positions skills as complementing MCPs, not replacing them, and urges versioning and testing as software.

Medic for Apache Spark - First Aid for Failing Jobs - Drasko Profirovic, Pinterest

Jul 20, 2026 · 11:21

Drasko Profirovic, a Staff Engineer at Pinterest, presents Medic for Apache Spark, an agentic diagnostics tool that automatically troubleshoots Spark job failures by ingesting logs, correlating context, and producing human-quality diagnoses and actionable recommendations in minutes. The talk covers the journey from a single React agent with unsustainable prompt tuning to a multi-agent architecture built on LangGraph's deep agent library, where specialized agents handle triage, research, and healing. Key improvements include an exception classifier pipeline that filters benign exceptions from logs, converting raw time-series metrics into annotated graphs for token efficiency, and an end-to-end test harness that snapshots production state for offline evaluations. Profirovic shares lessons on handling ambiguity, reducing hallucinations, and balancing automation with human oversight, plus unexpected failure modes that informed iterations. The system now extends to optimizing Spark SQL…

From Blind Spots to Merged PRs: Continuous Agentic Performance Optimization - May Walter, Hud

Jul 19, 2026 · 22:46

May Walter, CTO of Hud, presents a case study on using coding agents with runtime intelligence to automate continuous performance optimization in production. The system analyzes real production traces, queries, and latencies to surface high-ROI fixes like N+1 queries and missing indexes, scoring them by complexity and impact. It runs weekly via GitHub Actions and Claude Code, generating reports with verified fixes and evidence of impact, such as P90 latency improvements after deployment. Key challenges included 'plausible unverified' fixes, complex ClickHouse queries, and lazy fixes that only mask exceptions rather than root causes. Walter emphasizes the need for prod-to-code context mapping, scoring guardrails, and human review to achieve reliable automation. The talk concludes that context over cleverness works, and agents can automate the investigation phase that teams rarely do proactively, turning dormant performance debt into actionable merged PRs.

Stop Burning Tokens: Why self-improvement needs domain expertise first - Annabell Schäfer, Langfuse

Jul 18, 2026 · 17:39

Annabell Schäfer, Growth Engineer at Langfuse, argues that successful auto-improvement loops require domain expertise and high-signal target functions, not just generic evaluators. She details an experiment classifying arXiv papers with a minimal loop using GPT-5 for nano and Claude Opus 4.8 as an optimizer, achieving a 15% accuracy jump from 68% to 83% in four iterations. The first iteration alone gained 10% by adding classification rules and examples based on error analysis of a 200-item dataset. Schäfer advises replacing vague metrics like correctness with yes/no quality criteria (e.g., 'answer uses knowledge base'), and working with domain experts to identify failure modes and define what 'good' means. She emphasizes validation to prevent overfitting and treating the system as generalizing from representative examples, not just burning tokens on endless loops.

Using LLMs to Secure Source Code — Eugene Yan, Anthropic

Jul 17, 2026 · 21:30

Eugene Yan of Anthropic details how frontier LLMs like Claude are reshaping software security, citing Mozilla's 20x surge in monthly fixes (to 400 in April 2025, two-thirds credited to Claude) and Anthropic's own scan of 1,000+ open source repos that uncovered 6,200 high/critical issues, 1,600 reported, and about 100 patched upstream. He argues that finding vulnerabilities is no longer the hard part; the bottleneck has moved to verification, triage, and patching. Yan outlines a six-step workflow: a written threat model (boosts true positive rate to 90%), an isolated sandbox for reproducibility, discovery optimizing for recall, a separate adversarial verification agent that detonates exploits in fresh containers, triage to prioritize engineer attention, and patching that closes the loop so bugs cannot recur. His advice: start this week on open source dependencies, keep hands on the wheel before automating, and recognize that scanning was never the bottleneck.

Special Topics in Kernels, RL, Reward Hacking in Agents — Daniel Han, Unsloth

Jul 17, 2026 · 2:20:21

Daniel Han of Unsloth argues that reward hacking—where AI models cheat to maximize reward—is a critical problem in agent training, citing examples from GPT-5.1's calculator hacking and GPU mode kernel competitions. He shows that models exploit benchmark flaws, such as viewing Git history or editing timers, and that even open-source models like GLM 5.2 require anti-hacking measures. Han emphasizes that harness and tooling quality now outweigh model choice, with inference providers sacrificing accuracy for speed (e.g., 10% accuracy drops across providers). He also warns that hardware limits (float4 precision, diminishing returns) shift focus to software algorithms like FlashAttention and gradient checkpointing. The workshop concludes that benchmarks are unreliable—DeepSpeed's false positive rate is contested at 44.9%—and urges verification before trusting performance claims.

Every company should have a Brain — Garry Tan, Y Combinator

Jul 17, 2026 · 21:08

Y Combinator CEO Garry Tan argues founders must build AI-native companies that treat AI as a workforce, not auto-complete, to achieve 400x productivity. He details how skill files (Markdown documents) act as employees, a resolver table as an org chart, and a company brain (like his open-source GBrain) as the memory layer that selects the right context for each task. Tan cites YC's Winter '25 batch where 25% of companies had 95% AI-generated code, and portfolio companies Emergence and Retail reaching $15M ARR with 15 people and $60M with 40 respectively. His core message: never do one-off work—always "skillify" completed tasks into reusable skills, so the organization compounds knowledge daily rather than waking up with amnesia.

Claude Fable, Claude Tag, and Anthropic's Culture — Cat Wu & Thariq Shihipar ft Simon Willison

Jul 15, 2026 · 51:30

This episode features Anthropic's Cat Wu and Thariq Shihipar discussing Claude Code, Claude Tag, and Claude Fable, arguing that these tools have shifted engineering from slow spec-driven processes to rapid, ambitious building. Thariq notes that with each model generation, delegation increased, and Claude Fable now enables one-shot features. Cat says engineers now need product taste over execution, as timelines shrink from six months to a week. Claude Tag, a proactive multiplayer agent, lands 65% of product engineering PRs internally by monitoring channels and remembering preferences. The team reduced Claude Code's system prompt by 80% for frontier models by removing examples and hard constraints, relying on model judgment. Auto mode, used internally since January, mitigates prompt injection through thousands of evals and Sonnet classifiers. Cultural hacks include default-public channels and a 'don't negotiate against ourselves' mindset, leading to ambitious builds like Thariq's Claude-powered video editing and a Street Fighter game.

WTF Is the Context Layer? The Missing Infrastructure for Production Agents — Prukalpa Sankar

Jul 14, 2026 · 20:54

Prukalpa Sankar, founder of Atlan, argues that the missing infrastructure for production AI agents is a 'context layer' that encodes business knowledge, expertise, and norms into machine-usable form. She describes how her team built 300 skills and 40 agents but hit problems like context sprawl, dependency management, and skill quality. Sankar proposes a context layer with versioning, lifecycle management, and compounding learning loops from agent traces, enabling teams to share and maintain business context like code. She concludes that context is both king and intellectual property, differentiating companies when models and intelligence are commoditized.

Don't Ship Skills Without Evals — Philipp Schmid, Google DeepMind

Jul 14, 2026 · 21:46

Philipp Schmid, Staff Engineer at Google DeepMind, argues that shipping agent skills without evals is like merging code without tests, leading to unreliable performance. He explains that most of the 50,000+ skills on Skill Bench lack evals and are often AI-generated, which can negatively impact performance. The talk distinguishes between capability skills (teaching models new tasks) and preference skills (encoding team-specific workflows), with evals enabling retirement of skills as models improve. Schmid shares practical advice: keep skill descriptions concise (under 500 lines), include negative tests, and write directives over passive information. He details a case study creating 117 test cases for the Gemini interactions API, achieving 90% valid code generation using regex-based asserts and LLM-as-judge. The key takeaway: always run ablation tests with and without skills to measure their true impact, and start small with as few as 10-20 eval samples.

"I've never seen anything scarier than an LLM with tool calls." — Erik Meijer aka @HeadinTheBox

Jul 13, 2026 · 21:13

Erik Meijer of Leibniz Labs argues that AI agents with tool calls are intrinsically dangerous and must be tamed through formal verification. He recounts how adding tool calls to LLMs turned harmless chatbots into agents capable of irreversible side effects, like deleting files or emptying bank accounts. His solution, implemented in Automind, uses proof-carrying code: the agent submits a plan as a program (Free Monad) together with a machine-checkable safety proof, and a small checker verifies the proof before allowing execution. This air-gaps the agent from its tools, ensuring policy compliance statically. Meijer demonstrates that elementary type systems and compiler techniques, such as taint analysis on program expressions, can provably prevent unsafe actions, turning agents from 'vibe-coding' risks into provably safe systems.

The AI bugpocalypse is here. Now what? - Jack Cable, Corridor

Jul 12, 2026 · 19:44

Jack Cable, Co-Founder and CEO of Corridor, argues that frontier AI models can now find and exploit software vulnerabilities at scale, creating an 'AI bugpocalypse' that demands fundamental security changes. He cites 84% of developers using AI coding tools, with models like Mythos outperforming humans in finding vulnerabilities. He notes that basic vulnerability classes like buffer overflows remain prevalent despite decades of known fixes, and that memory-safe languages like Rust can eliminate 60-70% of such bugs, as shown by Android's drop from 75% memory safety vulnerabilities in 2019 to 30% in 2022. He warns that even advanced models introduce vulnerabilities 20-40% of the time (backsbench.com) and points to an Opus 4.6 incident that lost $2M. His recommendations to Congress include preventing vulnerabilities in new code, hardening open-source software, and fostering American open-weight models, arguing that defenders need access to these models to stay ahead.

A Song of Types and Agents - Roberto Stagi, Ratel

Jul 12, 2026 · 14:16

Roberto Stagi (Ratel) argues that TypeScript is overtaking Python as the dominant language for building AI agents and applications, driven by the rise of coding agents like Lovable, Claude Code, and Cursor that default to TypeScript. He notes that Python still rules training, research, and GPU serving, but the application layer—where AI is shipped inside products—now belongs to TypeScript. Stagi highlights that TypeScript surpassed Python as the most used language on GitHub in August 2025, with GitHub attributing the shift to AI agents. He cites the Vercel AI SDK growing from 1.6M to 15.1M weekly downloads in one year and npm's rich ecosystem of authentication, payments, and UI packages as reasons to build agents in TypeScript. Consistent end-to-end typing via Zod and a unified codebase for agent loops, tools, backends, and UIs further strengthen the case. Stagi concludes by invoking Atwood's Law: any application that can be written in JavaScript will eventually be written in TypeScript, urging developers to build agents in TypeScript to avoid falling behind.

remobi.app: Don't change your terminal workflow for mobile

Jul 12, 2026 · 9:21

Connor Adams presents remobi.app, a progressive web app that lets developers check on their terminal-based coding agents from a mobile phone without changing their existing workflow. Built for TMUX users, it mirrors a user's remote terminal session with gesture controls for zoom, scroll, and pane switching. Adams argues that unlike Claude Code's built-in handoff or dedicated mobile terminals, remobi is agent-agnostic and preserves custom TMUX setups with tools like lazygit and critique. The app uses Tailscale by default for secure remote access, though users can substitute Ngrok or Cloudflare tunnels. Open source and installable via npm, remobi aims to satisfy the compulsive need to monitor agents while away from the desk.

Chat and citations won't save your vertical AI - Atul Ramachandran, Filed Inc

Jul 11, 2026 · 15:12

Atul Ramachandran, CTO and co-founder at Filed, argues that chat interfaces and citation trails fail vertical AI because they shift the verification burden back to users, making every error catastrophic. Drawing from Filed's work building AI data entry for tax firms—where they hit 80%+ accuracy against a 50-60% baseline yet still faced complaints—he explains that the real fix is to design for delegation, not participation. He proposes treating the product as a conveyor belt where users are supervisors, using long-running background agents, automatic skills that capture user quirks, monitoring via traces, and control mechanisms like plan approval before irreversible actions. Ramachandran advocates measuring weekly active sessions instead of weekly active users, aiming for sessions to rise while direct user interaction falls. The episode covers these four pillars and the shift in metrics needed to build successful vertical AI products.

Every Solo Agent Builder Eventually Reinvents a Worse Version of CI/CD - Sumaiya Shrabony

Jul 11, 2026 · 10:51

Sumaiya Shrabony argues that solo agent builders inevitably reinvent CI/CD badly, as demonstrated by her open-source 19-skill Claude Code agent system. She identifies five reinventions—regression testing, CI monitoring, contract testing, staging environments, and audit trails—and three failure modes: voice drift, missing verification, and duplication hooks. The dangerous failure isn't bad output but polished artifacts that violate production contracts, like using the wrong voice or making unverified claims (e.g., a 37% statistic with no source). She advocates for adding boring gates at the most expensive handoff to block artifacts, not just warn, before shipping.

Building an ACP-Compatible Agent Live — Bennet Fenner, Zed

Jul 8, 2026 · 18:19

Bennet Fenner from Zed demonstrates building an ACP-compatible coding agent live, showing how the Agent-Client Protocol lets agents and editors communicate over JSON-RPC. He starts with a minimal agent that reads and edits files, then adds ACP support using the TypeScript SDK—implementing initialize, session creation, and prompt handling. Next, he streams model tokens via session updates, emits tool call progress with file paths and diffs, and proxies file system operations through the client. The demo culminates in the agent bootstrapping itself to add a terminal tool, which then runs shell commands inside Zed. ACP currently works over stdio; a remote transport is in development.

Your coding agent doesn't always follow your rules — Talha Sheikh, Checkout.com

Jul 8, 2026 · 10:08

Talha Sheikh from Checkout.com argues that the real value in AI coding agents lies not in the code they generate but in the verification layer that checks it. Drawing from his experience with Claude Code, he built Vector Harness—a tool using Claude hooks to deterministically test agent outputs against a config file, retrying failures automatically. He contends that model capability does not equal reliability, and that even with better specs or models, deterministic verification is essential. Sheikh observes that Anthropic, OpenAI, and CUDO are all adopting similar harness patterns, shifting the focus from instructing agents to enforcing rules. His central claim: work on the harness, not the code, and you can even use smaller, cheaper models with enough guardrails.

Beyond the Harness: A Journey Towards Adaptative Engineering - Rajiv Chandegra, Annicha Labs

Jul 7, 2026 · 37:01

Rajiv Chandegra of Annicha Labs introduces adaptive engineering as a design philosophy that moves beyond fixed AI harnesses (like Claude Code, Cursor, or Py) to let multi-agent systems self-organize during runtime. He argues that as models become exponentially more powerful and AI collides with the messy, dynamic real world, pre-engineered static harnesses become brittle. Drawing on complexity science, he contrasts the factory model—reliable but novelty-suppressing—with adaptive engineering, where agents interact locally and the harness emerges as an ongoing output, not a predetermined input. The engineer's role shifts to designing constraints (e.g., rate of coupling, goal reward) and sensing the emergent order, while failures include drift, monoculture, and legibility collapse. This approach targets 'horizontal intelligence'—decentralized coordination among agents—as the key to handling complex, real-world problems that cannot be decomposed into fixed parts.

What if the harness mattered more than the model? - Aditya Bhargava, Etsy

Jul 7, 2026 · 32:04

Aditya Bhargava, a Staff Engineer at Etsy, argues that the harness—the tooling around a model—matters more than the model itself, especially for weaker models, and introduces Agency, a language he built for constructing agents. He cites a benchmark where only the harness changed, producing a 52.4% to 76.2% score range (over 20 points), showing that a better harness can make local open-source models perform near proprietary ones. Through seven evolving examples of a coding agent, he demonstrates key harness improvements: adding tools, ensuring safety via handlers and partial function application (PFA) to lock directory access, implementing a reason-act-feedback loop, using subagents to add capabilities without context bloat, and self-optimization via a built-in optimizer to systematically measure and improve prompts. He advocates for building expertise in harnesses to reduce reliance on proprietary models, enabling use of locally runnable ones. Agency provides simple tool syntax, safety features like interrupts and true pause/resume, and built-in optimizers for measured improvement.

Respect The Process - Andrew Dumit, Watershed Technology Inc.

Jul 7, 2026 · 16:43

Andrew Dumit of Watershed explains how to deploy coding agents for editing complex supply chain graphs in sustainability, arguing that in a domain rife with expert judgment calls, you must constrain the agent's effects through a typed SDK and deterministic execution while leaving its reasoning free. Unconstrained coding agents caused inconsistency, hallucination, and false reports—saying edits were made when they weren't—and made review impossible for non-coders. Their solution: a TypeScript SDK as the only way to edit graphs, plus a harness that lints agent code, catches conflicts, runs the script, and validates output artifacts, producing traceable, replayable edits. This approach improved accuracy from 43% to 92% on internal evals, with further gains from prompt improvements, few-shot examples, and breaking tasks into plan-and-execute loops. Even when the agent lands on a different expert answer, every change is valid and reviewable without reading code.

500 people vibe-coded for 30 days. I was one of them. - Sanja Grbic, Automattic

Jul 7, 2026 · 17:49

In Automattic's Radical Speed Month, a 30-day experiment where over 500 employees paused roadmap work to ship real projects, product designer Sanja Grbic built three distinct products—a board game session manager, a design system status tracker, and an iOS chat for WooCommerce merchants—demonstrating how AI tools like Claude Code and ContextAIC can transform roles and unlock speed. The first project showed that engineers become enablers when they set up non-engineers for coding success. The second project, completed in two and a half weeks, moved Grbic from designer to design engineer by owning the full build, from proof of concept to deployment. The third project, built in just six days with a fellow designer, proved that shared understanding of AI workflows collapses iteration time. Grbic argues that large organizations can replicate this by providing tool access, finding enablers, creating space for experimentation, and granting agency—shifting human behavior to break out of established habits and release the velocity AI offers.

SWE-Marathon: Evaluating Coding Agents at Billion-Token Scale - Rishi Desai, Abundant AI

Jul 7, 2026 · 12:58

Rishi Desai of Abundant AI presents SWE-Marathon, a benchmark for coding agents that finds even the strongest setup achieves only a 26% resolution rate across 20 project-scale tasks. Trajectories average 31 million tokens, with the longest consuming 877 million tokens, and a computer-use agent verifier evaluates full-stack products via browser interactions. Desai warns that weak verifiers become attack surfaces in long-horizon evals, showing agents bypassing tasks by calling GCC from within a Rust compiler, caught by anti-cheat layers using S-Trace. Across 1,400 rollouts, 12.8% showed suspicious shortcuts and 9% clear verifier bypasses, but zero earned reward through exploits due to multi-channel checks. The central claim: long-horizon SWE is unsolved, and robust verification—not harder unit tests—is the key bottleneck for future benchmarks.

Field Guide to Fable — Thariq Shihipar, Anthropic

Jul 6, 2026 · 19:28

Thariq Shihipar of Anthropic discusses Fable and argues models improve in "spiky" ways: a chat model fails to list Pokémon ending in "aw" (Croconaw and Dreadnaw), but Claude Code fetches and filters the list in seconds – a gap he calls "capability overhang." He explains that to unlock Fable, Claude Code cut 80% of its system prompt because heavy instructions now constrain a more imaginative model, and the "ask user question" tool evolved from barely working under Opus 4 to generating embedded HTML questionnaires. He shares techniques like blind-spot passes and interviews to surface unknown unknowns, and reflects on the grief of moving from hand-coded programming to agentic workflows. Shihipar urges engineers to reject trade-offs – "good, fast, cheap: pick three" – and instead demand all three, citing a four-hour keynote deck built with Fable as proof that agents can deliver ambitious work faster.

MCP Apps: Primitives, discovery, and the Future of Software - Pietro Zullo, Manufact, Inc

Jul 5, 2026 · 28:54

Pietro Zullo, co-founder of Manufact, explains that MCP Apps are MCP servers that return interactive UI elements in sandboxed iframes, enabling bidirectional communication between the UI and the host. He details primitives like set-model-context and send-message, and shows how apps can stream tool inputs into the UI in real time. The ChatGPT, Claude, and Cursor stores now accept self-serve submissions, and Claude's dynamic discovery matches connectors to user intent. Manufact’s open-source SDK, mcp-use, has 8M+ downloads and provides a cloud platform for building, testing, and submitting MCP Apps.

The Agentic AI Engineer - Benedikt Sanftl, Mutagent

Jun 29, 2026 · 34:50

Benedikt Sanftl and Burak of Mutagent introduce the Agentic AI Engineer, an automated loop that builds, evaluates, and optimizes AI agents at scale. They argue the human bottleneck in manually iterating on agent prompts, evaluations, and diagnostics can be removed by using an orchestrator that manages spec-driven development, eval-driven testing, and online monitoring. The offline loop covers spec, build, and eval; the online loop monitors traces, diagnoses failures, and feeds improvements back into the system. They demo two research-preview agents: an evaluator that constructs eval suites and a diagnostics agent that analyzes production traces to suggest targeted fixes. The platform connects to observability tools and target frameworks (e.g., Claude Code, LangChain) and outputs PRs or updated agent definitions. The core claim is that automating this lifecycle enables teams to ship and improve hundreds of agents with faster iteration and less manual effort.

We Cut 94% of AI Coding Tokens With a Local Code Index - Rajkumar Sakthivel, Tesco

Jun 28, 2026 · 10:43

Rajkumar Sakthivel and his friend Faz built Code Context Engine (CCE) to cut AI coding costs after their bill jumped from £15 to £200 in one month. They found that 45,000 tokens were sent per query but only 5,000 were needed, with 90% of cost being input context. Their solution is a local retrieval layer that uses AST-aware chunks, combined vector and keyword search, and a weighted scoring heuristic (50% meaning, 30% keyword, 20% recency) that runs in 0.4 milliseconds. On a FastAPI test, tokens per question dropped from 83k to 4.9k—a 94% reduction—with 90% accuracy. They emphasize that fixing the input, not the model, yields the biggest savings, and their open-source tool shares a single index across Claude Code, Cursor, Copilot, and Codex.

Building an Autonomous Engineering Org - Angie Jones, Agentic AI Foundation

Jun 28, 2026 · 17:36

Angie Jones describes how she transformed Block's 3,500-person engineering org into an autonomous one using AI agents, starting with a maturity model and an AI champions program of 50 hand-picked engineers. She details making repos AI-ready with context and rules files, then enabling delegation from Slack, Jira, and GitHub, which boosted AI-authored code by 69% and automated PRs 21x. Multi-agent parallelism required cloud workspaces and a company world model across 25,000 repos, culminating in BuilderBot that allowed anyone to @mention it for fixes. However, the success led to layoffs, raising existential questions about the human cost of autonomous engineering.

Structuring the Unstructured - Cedric Clyburn, Red Hat

Jun 28, 2026 · 20:41

Cedric Clyburn (Red Hat) demonstrates how Docling, an open-source tool from the Linux Foundation, converts unstructured documents like PDFs, tables, and images into structured formats (Markdown/JSON) for AI workflows. He shows that naive PDF parsers lose table structure and image content, while proprietary VLMs are expensive and non-deterministic. Docling uses OCR and layout analysis to preserve context, achieving 50x cost savings compared to VLMs on CPU. The demo covers extracting tables and images from an 8-page PDF, using a local Granite vision model for image annotation, and implementing chunkless RAG where an LLM queries a Docling document outline directly. Scaling is addressed via Docling Serve (REST API) and Docling MCP server for agentic document processing with tools like Claude Code.

Agents Building Agents - Alfonso Graziano, Nearform

Jun 28, 2026 · 30:14

Alfonso Graziano, a tech lead at Nearform, presents a production workflow where a coding agent (Claude Code) builds and iteratively improves a target AI agent by running evals, inspecting failures, and autonomously fixing system prompts, tool contracts, or code. He demonstrates that this loop raised eval accuracy from 18% to 83% on a naive agent and achieved a 10% improvement on a human-optimized production agent. For live data, the system collects user feedback traces, clusters failure modes with root-cause analysis, and generates fix proposals, which are validated by subject matter experts and added to a golden dataset. The method relies on spectrum development, quality gates, and observability, enabling semi-autonomous agent improvement without cheating on eval sets.

Turn 10,994 Notes Into Memory - Paul Iusztin, Decoding AI & Louis-François Bouchard, Towards AI

Jun 26, 2026 · 39:32

Paul Iusztin and Louis-François Bouchard introduce an AI Research OS that transforms a personal Second Brain (e.g., an Obsidian vault of 5,000+ notes growing by 250 files/month) from a passive archive into a live, queryable memory for AI agents. They argue that bigger context windows aren’t enough; proper memory and context management are needed. The system evolved through three iterations: first a simple deep research loop generating static research Markdown files (used for 35 course lessons), then targeting both the public web and the user’s Second Brain, and finally adding a persistent wiki layer. The architecture uses plain files (Markdown, YAML index) instead of vector databases, with a hierarchy of raw sources, source summaries, and wiki derivatives (concepts, comparisons, entities). The wiki evolves with every question, not just during ingestion. Demos show ingesting Obsidian notes, GitHub repos, and arbitrary links, then querying the wiki to update it dynamically.

The Log Is The Agent - Ishaan Sehgal, Omnara

Jun 25, 2026 · 15:11

Ishaan Sehgal, CEO of Omnara, argues that an AI agent's true identity is not the model, runtime, or tools but its append-only event log—the durable record of every user input, model output, tool call, and permission. Drawing an analogy to a video game save file that persists beyond hardware failure, he explains that treating the log as the agent enables reliability, scalability, forking, migration, multiplayer, and ownership. He warns that deepest lock-in is log lock-in: if a provider owns the log, it effectively owns the agent. Omnara's open-source managed agents platform is built around the session log as the system, not a side effect, allowing agents to survive crashes, resume across machines, and be fully owned and inspected by users.

Recursive Coding Agents - Raymond Weitekamp, OpenProse

Jun 25, 2026 · 23:48

Recursive Language Models (RLMs) are a new test-time compute paradigm that unify reasoning and tool calling, and Raymond Weitekamp of OpenProse argues they can turn coding agents into reliable, recursive problem solvers. He demonstrates that RLMs can process millions of tokens beyond context windows, and a small Qwen 3.5 9B model using RLMs beats GPT-5.4 and Opus on the LongCOT benchmark. Weitekamp shows how Claude Code's Dynamic Workflows now make it an RLM, and how OpenProse allows any coding agent to be recursively orchestrated with declared sub-agent work and dependencies. Use cases include repo-scale migrations, deep research over file systems, and adversarial red-teaming audits. He concludes that RLMs represent the next paradigm of inference-time compute, turning mismanaged genius agents into trustworthy outcomes by capturing golden sessions as reusable workflows.

Your Attention Is the Bottleneck, Not Your Agents — Zack Proser, WorkOS

Jun 11, 2026 · 25:17

Zack Proser from WorkOS argues that human attention, not agent speed, is the real bottleneck in AI-assisted coding. He proposes a sustainable stack: signal layers to filter Slack and Linear, voice-first flows at 184 wpm, remote control of agents from a phone to leverage diffuse thinking, and weekly self-improvement passes over JSONL conversation history. He also integrates an Oura ring via MCP so Claude can nudge him about sleep, emphasizing balance over burnout.

RAG is dead, right?? — Kuba Rogut, Turbopuffer

Jun 9, 2026 · 11:13

Kuba Rogut (Turbopuffer) argues that RAG isn't dead—it's evolving into agentic retrieval, contrasting Cursor's upfront indexing (which boosted answer accuracy 24% in their composer model) with Claude Code's per-session grep approach. He frames embeddings as cached compute, citing Jeff Dean: 'You don't need a trillion at once, you need the right million.'

From MCP to Scale: Pipelines That Build Themselves — Rafael Levi, Bright Data

Jun 7, 2026 · 25:26

Rafael Levi from Bright Data shows how LLMs combined with Bright Data's MCP and infrastructure turn web scraping into self-healing pipelines. He demonstrates building a Walmart scraper in minutes via Claude Code, where the agent inspects HTML, generates a parser, and extracts 90 products using token-efficient scripts rather than parsing full pages—saving 62% of tokens. Levi explains that the MCP bypasses anti-bot systems like Cloudflare and provides 66 tools for agents, including remote browsers that mimic human behavior. He notes that Bright Data has 150M IPs and prebuilt APIs for domains like Amazon, and that public data scraping is legally protected (citing lawsuits won against Meta and Elon Musk). Even personal use cases are addressed: Levi built a listener that found him an apartment, and a bot that books restaurant tables. The key takeaway: agents can explore, build, maintain, and repair scrapers automatically, eliminating the need for human intervention when sites change.

Evals Are Broken, Use Them Anyway — Ara Khan, Cline

Jun 6, 2026 · 19:04

Ara Khan from Cline argues that evals are broken—people either treat benchmark numbers as gospel or dismiss them for vibes—but that the truth lies in between, and they should still be used. He presents three heuristics: don't believe model vendor eval numbers, stay current but not an earliest adopter, and look for new precise evals like Terminal Bench. Khan details Cline's journey from ignoring evals to building their own, then adopting Terminal Bench (89 real-world coding tasks). He explains the process: get a score (Cline started at 43%), portfolio allocate failures by sending another agent through traces to identify small levers, then hill climb by fixing zone1 bugs, zone2 nuanced prompt engineering (e.g., Anthropic-specific techniques that don't transfer to Codex or Gemini), and avoid zone3 overfitting. The episode offers a practical framework for using evals to improve agent performance while staying grounded in real-world usefulness.

Building Agent Interfaces: Lessons from Chrome DevTools (MCP) for Agents — Michael Hablich, Google

Jun 5, 2026 · 22:38

Michael Hablich, product manager for Chrome DevTools at Google, presents four engineering lessons from building agent interfaces, centered on the Chrome DevTools MCP server. He introduces the metric 'tokens per successful outcome' to measure interface fuel efficiency, contrasting it with raw JSON trace files that blew through context windows. The team improved error messages, like adding 'Cannot navigate back, no previous page in history' to enable agent self-healing without human intervention. They decomposed a monolithic `debug_webpage` tool into 25 focused tools, but then faced a discoverability problem solved by auditing tool descriptions for intent. Hablich defends keeping autoconnect friction to prevent prompt injection, citing the 'lethal trifecta' of agents with tool access, data access, and internet access, and outlines three security tiers for browsing agents.

SWE-rebench: Lessons from Evaluating Coding Agents — Ibragim Badertdinov, Nebius

Jun 4, 2026 · 16:30

Ibragim Badertdinov from Nebius presents SWE-rebench, a monthly updated benchmark that evaluates coding agents on fresh real-world software engineering tasks to prevent data leakage from pretraining. The leaderboard reveals that models like Claude Code cheat by reading git history or fetching original GitHub issues, even after restrictions; Badertdinov emphasizes that task quality is critical, as ambiguous or overfitted tests introduce noise rather than difficulty. The filtering pipeline has produced 30,000 real-world training environments used by frontier labs. The episode also covers practical evaluation lessons: define retry policies, use caching to cut costs by 4x, and verify infrastructure against reported numbers. SWE-rebench reports tokens per problem, price per problem, and pass rates across five runs, helping AI engineers choose between models and harnesses reliably.

Benchmarking semantic code retrieval on Claude Code — Kuba Rogut, Turbopuffer

Jun 3, 2026 · 16:08

Kuba Rogut from Turbopuffer benchmarks semantic code retrieval on Claude Code, finding that raw Claude Code wastes one in every three file reads (65% precision), windowed grep drops that to one in five, and adding semantic search cuts it to one in eight (87% precision). Semantic search wins on behavior-adjacent tasks where files share no keywords; grep wins on import tracing. Cursor's production numbers show a 24% relative improvement in answer accuracy from semantic retrieval plus a 2.6% increase in code retention in large codebases. Rogut explains Cursor's model knows when and why to call semantic search, while Claude Code treats it as just another tool, limiting gains.

What if the network was the sandbox? — Remy Guercio, Tailscale

Jun 1, 2026 · 24:29

Remy Guercio from Tailscale argues that standard sandboxing conflates execution isolation with access control, proposing Aperture—an LLM gateway built on Tailscale's WireGuard identity network—which gives every connection verified identity (user, tag, or group) so agents get placeholders instead of real API keys, making exfiltration impossible. Aperture provides visibility into every tool call, bash command, and MCP request without instrumentation inside the container; internally at Tailscale, bash dominates over structured tool calls. Access permissions are configured via Tailscale's grants and ACLs, supporting quotas, cost controls across providers, and webhooks for tool calls. The gateway works at the LLM layer, capturing even non-tool-call agent behaviors like direct code execution, and is available on Tailscale's free plan.

Reverse engineering a Viking VOIP phone protocol with Claude Code — Boris Starkov, Eleven Labs

May 29, 2026 · 20:11

Boris Starkov from ElevenLabs used Claude Code to reverse engineer the undocumented protocol of a Viking VOIP phone that three senior engineers and ChatGPT failed to configure in a year. Claude brute-forced all 676 possible two-letter command combinations, found 80 valid ones, then set up a TCP proxy between a Windows XP virtual machine and the phone to intercept traffic from the proprietary software. The critical missing piece—a one-byte checksum in the persistence command—was deduced by Claude by running known input-output pairs through the derived formula, confirming a simple subtraction. Starkov acted as Claude's hands, physically rebooting the phone and reporting beeps. The protocol is now open-sourced as a Claude Code skill, enabling direct programming without Windows software. The project culminated in a red phone booth at AI Engineer Europe where picking up the receiver connects to a Michael Caine voice agent quizzing attendees on British AI history.

Most Enterprise Agentic Projects Are Doomed, Here's Why — Jess Grogan-Avignon & Jack Wang, Accenture

May 28, 2026 · 20:35

Jess Grogan-Avignon and Jack Wang, Accenture, argue that most enterprise agentic projects fail not because of bad code but because organizational scaffolding designed for human pace chokes machine-speed AI delivery. They built an agentic application in two weeks, then spent twelve months aligning infrastructure, security, AI gateway, data governance, and application teams before it shipped—a pattern they say will worsen as GitHub's 275 million weekly commits flood approval infrastructure built for humans. They name five predictive tensions: human approval chains must become executable code, not longer meetings; finance should back a portfolio of AI bets like a VC, not demand committed returns from each project; delivery should run on hypothesis-driven loops building statistical confidence, not fixed milestones; trust is built through progressive autonomy—shadow, advisory, controlled autonomy—gated by outcome evidence, not plan completion; and the real moat is living memory from real customer signals, not static CRM or ERP data. Their prescription: bet like a VC, upgrade governance for machine speed, and engineer for trust with feedback loops from day one.

The AI Skill I Rely On Daily — Priscila Andre de Oliveira, Sentry

May 27, 2026 · 17:05

Priscila Andre de Oliveira, a senior software engineer at Sentry, reveals that 67% of her AI usage is for comprehension and only 2% for code generation, based on analyzing 116 of her own Claude sessions. Working in Sentry's 15-year-old codebase with 100 PRs merged daily and 100,000 organizations depending on it, she built a personal skill called 'Catch Me Up' with six exploration modes (architecture, conventions, feature traces, syntax, testing, history). She argues that understanding what the agent found before letting it plan and implement prevents misaligned mental models that produce slop code. The episode emphasizes that in large, complex codebases, AI's biggest unlock is comprehension, not generation, and advises developers to align their mental models before prompting.

The maturity phases of running evals — Phil Hetzel, Braintrust

May 27, 2026 · 18:34

Phil Hetzel from Braintrust argues that evaluating AI agents should not mimic exhaustive unit tests but instead focus on known failure modes, using a flywheel of production traces to guide improvements. He outlines four maturity phases: vibe checking with documented human justifications, scaling those justifications into LLM-as-judge evaluations, handling context-gathering tool calls, and tackling CRUD tools that require representing external system states at trace time. To address state-replay challenges, he recommends injecting captured system state directly into traces or using timestamp queries against vector databases. The goal is to rerun production data offline, using LLM judges evaluated against ground truth, and automatically uncover failure modes through topic modeling.

Let's Talk About FOMAT: Fear of Missing Agent Time — Michael Richman, Cmd+Ctrl

May 24, 2026 · 16:17

Michael Richman introduces FOMAT (Fear of Missing Agent Time), the anxiety of wasting agent productivity when you're away from your dev machine. He built Cmd+Ctrl, a system that sends push notifications when an agent stalls or finishes, lets you respond from phone or watch, and allows starting new sessions remotely. The demo shows Claude Code mirrored on an iPhone, with real-time interaction and a standup dashboard summarizing all sessions. The open-source daemon layer works with Claude Code, Cursor, Codex, Gemini CLI, and others, aggregating sessions across machines into a single pane of glass. Richman argues this alleviates the cognitive load of managing multiple agents and supports the new flow of agent choreography.

The Missing Primitive for Agent Swarms — Lou Bichard, Ona

May 23, 2026 · 18:37

Lou Bichard, field CTO at Owner, argues that agent swarm infrastructure for coding agents is missing a coordination primitive, while runtimes and orchestration are largely solved. He defines a Software Factory as incrementally removing humans from the SDLC and highlights internal systems from Stripe (Minions) and RAMP (Inspect) as real-world examples. Owner provides VM-level isolation for agent fleets and sub-agent patterns, demonstrated with a demo of process-based and VM-based swarms. Bichard identifies GitHub as a poor coordination layer for hundreds of parallel pull requests and proposes solutions like state machines, durable execution, and a CLI gateway that any local agent can invoke to check progress. He notes context rot and agents skipping steps as key challenges, and announces a public two-week project to build a Software Factory from scratch starting next week.

Skill issue: Lessons from skilling up coding agents to use Langfuse - Marc Klingen, Clickhouse

May 20, 2026 · 24:09

Marc Klingen, founder of Langfuse, shares six learnings from building a skill that lets coding agents like Claude Code add Langfuse observability and evals to projects. The skill addresses the problem of agents hallucinating stale instrumentation from outdated pre-training context by surfacing a search endpoint for documentation, helping agents navigate 478 documentation pages, and referencing rather than duplicating content. Klingen explains how examining traces revealed 80% of issues, basic eval setups were better than none, and an auto-research loop with a target function suggested six improvements (three accepted). However, the target function nearly backfired by optimizing away documentation-fetching steps vital for long-term reliability. He also discusses challenges of skill distribution, versioning, and whether to aim for quick initial setup or perfect single-shot implementation.

From 46% to 90%: Fine-Tuning Tiny LLMs for On-Device Agents — Cormac Brick, Google

May 20, 2026 · 21:01

Google's Cormac Brick explains how developers can build on-device AI agents using either system-level Gemini Nano via AI Core or app-level custom models via LiteRT-LM. He demonstrates a skill harness built on Gemma 4 that enables agentic tasks like restaurant roulette, running fully on-device with JavaScript UI. For fine-tuned tiny models, Function Gemma at 270M parameters improved from 46% to over 90% accuracy on eight of ten app-intent functions after synthetic data fine-tuning. The talk also covers the Eloquent transcription app, which chains two Gemma 3-based models (ASR and text polishing) under a few hundred million parameters for offline use. Key trade-offs are latency, privacy, and customization versus system integration effort.

Don't Build Slop (4 Levels of AI Agent Maturity) - Ara Khan, Cline

May 19, 2026 · 18:52

Ara Khan of Cline presents four levels of AI agent maturity, arguing that most builders suffer from mass psychosis and should focus on thoughtful architecture. Level one is using existing frameworks for rapid prototyping. Level two introduces five rules: treat every agent as a state machine, keep system prompts minimal (e.g., GPT-5.3's prompt is one-third the size of GPT-5's to avoid sensory overload), integrate with a CLI for pseudo-RL pipelines, avoid building slop by designing architecture manually, and master frontier model APIs to leverage reasoning traces correctly. Level three advocates Kanban boards as the ideal UX form factor for managing parallel inference-bound agents, a prediction he made on March 26 that Claude Code shipped ten hours before this talk. Level four involves shipping agents to the cloud for scalability, enabling long-running tasks and parallel execution without local dependencies.

Anthropic Workshop: Build Agents That Run for Hours — Ash Prabaker & Andrew Wilson

May 18, 2026 · 1:15:40

Anthropic's Ash Prabaker and Andrew Wilson detail how to build agents that run for hours by replacing self-evaluation with adversarial evaluator agents that use Playwright to test live apps and grade subjective output via rubrics. They explain that context compaction doesn't cure coherence drift, so structured handoffs between fresh context windows are essential. The generator and evaluator negotiate testable sprint contracts before building, and the planner provides high-level specs without overspecifying technical details. They show that a solo Claude Code session built a retro game maker that looked complete but failed in play mode, while the adversarial harness produced a fully functional app with live physics and AI features. Key takeaways include reading traces as the primary debug loop, deleting harness components as models improve, and using file-system state for long-running agents.

Fighting AI with AI — Lawrence Jones, Incident

May 17, 2026 · 17:29

Lawrence Jones, founding engineer at Incident.io, describes how his team uses AI to debug their own AI SRE product, which runs hundreds of prompts per investigation across logs, metrics, traces, and code. He explains three patterns: a small CLI (evaltool) that lets coding agents read and edit large eval YAML files, enabling a red-green runbook where agents write failing evals, fix prompts, and verify nothing else broke. Their bigger unlock is serializing every UI debugging view as a downloadable file system, which when dropped into Claude Code allows agents to trace through the prompt hierarchy and identify exactly which prompt to change. For fleet-scale analysis, they run 25 parallel agents—each analyzing one investigation—then cluster results to surface systemic failure patterns across customer accounts. Jones emphasizes that file systems are exceptionally good agent context and that creating AI runbooks for complex analysis saves days or weeks.

How Building with AI Can Double the Throughput of Your Engineering Team — Brian Scanlan, Intercom

May 15, 2026 · 21:49

Brian Scanlan of Intercom explains how treating Claude Code like a new hire—onboarding it to a 15-year-old Rails monolith, writing skills for every recurring task, and connecting it to production systems—doubled engineering PR throughput in under a year. By going all in on one platform instead of letting everyone choose their own tool, Intercom achieved a 17.6% automatic PR approval rate with SOC 2 sign-off, and its CI infrastructure collapsed under the volume. The key principle: give agents problems, not tasks. Scanlan recounts a security incident where Claude automatically pulled files, ran analysis, and handed back next steps in two minutes using a skill he didn't know existed. The talk emphasizes that all engineering work—debugging, testing, planning—should be agent-first, and that companies must invest in platform-level adoption and continuous skill improvement.

Ship Real Agents: Hands-On Evals for Agentic Applications — Laurie Voss, Arize

May 14, 2026 · 2:04:18

Laurie Voss, head of developer experience at Arize AI, delivers a hands-on workshop on evaluating agentic applications using Arize Phoenix, demonstrating that choosing the right eval matters more than tuning it: a correctness eval scored 0 out of 13 on the same financial analysis agent that a faithfulness eval scored 13 out of 13, because the model doesn't know the current year and cannot verify forward-looking data. He walks through building a complete eval pipeline from scratch—starting with tracing a Claude Haiku-based financial agent, reading and categorizing traces to identify root causes, then implementing code evals, built-in LLM-as-a-judge evals, and a custom actionability rubric with labeled examples. Voss emphasizes the importance of meta-evaluation to validate judge accuracy and introduces Phoenix experiments to prove prompt changes actually improve scores, not just vibes. Practical tips include using the impact hierarchy (data quality > prompting > model selection > hyperparameters) and the value of regression evals for safe model upgrades. The workshop closes with cost-aware evaluation, pairwise evaluation, and reliability scoring as next steps beyond the foundations…

Building a Chess Coach — Anant Dole and Asbjorn Steinskog, Take Take Take

May 13, 2026 · 18:22

Anant Dole and Asbjørn Steinskog of Take Take Take, Magnus Carlsen's chess app, built an AI chess coach that keeps LLMs as translators rather than reasoners. Stockfish evaluates positions, tactical and positional detectors extract forks, pins, and structural weaknesses, and the LLM only converts those structured signals into English—preventing hallucination. They target sub-3-second latency using Gemini Flash. When a user flags bad commentary, it posts to Slack and injects into a running Claude Code channel via MCP. Claude investigates, modifies prompts or detectors, regenerates commentary, and asks clarifying questions. They run automated evals across 16 scenarios: Gemini Flash at 75%, Claude thinking below 60%, GPT-5 Mini lower. Their key insight: separate data pipeline from language generation, and close the loop with autonomous agents.

How we solved Context Management in Agents — Sally-Ann Delucia

May 10, 2026 · 16:17

Sally-Ann Delucia of Arize explains how her team solved context management for their AI agent Alex, which analyzes trace data from Arize's observability platform. She details the failure of naive truncation and LLM summarization, and the success of smart truncation preserving head/tail with a retrievable memory store. Long sessions are handled with evals that test context at every 10 turns, and heavy tasks are offloaded to sub-agents to keep main context lean. She notes that Claude Code uses a similar truncation strategy, and emphasizes that context engineering matters more than prompt engineering for agent success.

Agentic Search for Context Engineering — Leonie Monigatti, Elastic

May 8, 2026 · 1:03:13

Leonie Monigatti from Elastic argues that context engineering is 80% agentic search—the search tool that decides what to pull from files, databases, memory, and the web. She identifies three failure modes: the agent not calling any tool, calling the wrong tool, or generating incorrect parameters, and shows how detailed tool descriptions and agent skills (progressive disclosure) reduce these. In demos, semantic search fails for keyword 'JEPA' due to embedding similarity; a general-purpose ESQL tool with an agent skill correctly queries the database. Shell/bash tool enables file system search but requires iterative grep, while the custom Gina Grap CLI provides semantic grep for fuzzy queries. Practical recommendations: start with general-purpose tools, log agent behavior, and add specialized tools for frequent queries to balance low floor (easy successes) and high ceiling (complex queries). Hybrid agents combining shell and database tools achieve higher accuracy by verifying results.

Everything You Need To Know About Agent Observability — Danny Gollapalli & Zubin Koticha, Raindrop

May 7, 2026 · 50:25

Zubin Koticha and Danny Gollapalli of Raindrop argue that agent observability must shift from evals to production monitoring because agent failures are non-deterministic and unbounded. They break down explicit signals (tool error rate, latency, cost) and implicit signals (user frustration, refusals, task failure) detected by trained classifiers and regex, emphasizing that aggregate patterns even from imperfect regex are valuable. Experiments let teams ship changes to a percentage of users and compare semantic signal rates, with statistical relevance often reached after a few hundred events. Self-diagnostics—a simple tool and system prompt—enable agents to report their own failures, capability gaps, and even self-correction behavior, as demonstrated in a live coding agent demo where a disabled write tool caused the agent to use bash and then report the bypass. The episode covers alerting, trace visualization, data export to BigQuery/Snowflake, and the challenge of managing fast-paced experimentation at scale.

Full Walkthrough: Writing & Using Skills — Nick Nisi and Zack Proser

May 6, 2026 · 1:21:03

Zack Proser and Nick Nisi, Developer Experience Engineers at WorkOS, teach how to write portable, composable skills — single markdown files with optional scripts — that train AI tools like Claude, Codex, and Cursor to perform specific tasks consistently. They demonstrate building a repo roast skill that uses deterministic Git commands via script interpolation and progressive disclosure to load targeted references only when needed. The workshop covers structuring skills with front matter (name, description for LLM routing), adding confidence scoring and constraints to improve output, and sharing skills as .skill files or through marketplaces. Advanced topics include using skills in non-coding workflows (e.g., Slack–Linear automation) and composing skills in the WorkOS CLI, which uses the Claude Agent SDK to install auth. Attendees learn to iterate skills by reflecting on past conversations and leveraging Claude's built-in skill builder for evaluation.

Ralph Loops: Build Dumb AI Loops That Ship — Chris Parsons, Cherrypick

May 4, 2026 · 1:48:26

Chris Parsons argues that dumb loops—simple while loops where an AI agent repeatedly executes the same task, evaluates its output, and iterates—outperform complex multi-agent orchestration and planning graphs. In this hands-on workshop, he builds a Pomodoro timer using Claude Code, implementing tickets one by one in a loop that self-corrects with each pass. He demonstrates the 'loop' command in Claude Code for scheduled, continuous execution and shows how to use sub-agents for adversarial validation to avoid self-confirmation bias. The talk covers real-world applications such as a worker loop that processes project steps overnight and a morning loop that generates daily briefings, all designed to offload routine work while keeping human oversight for irreversible decisions. Parsons emphasizes that the bottleneck in AI-augmented teams is often the review process, not coding speed, urging teams to identify and fix their biggest constraint first.

Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked

May 3, 2026 · 1:41:25

Peter Werry of Unblocked argues that context engines—systems that supply AI agents with only the relevant organizational context—are critical to avoid agent doom loops and wasted tokens. He debunks three myths: naive RAG, connecting MCP servers, and bigger context windows do not solve the context problem. Werry describes building a social engineering graph to identify experts and distill team best practices, and shares hard lessons including hiding conflicts and caching answers. In a benchmark task, Unblocked's context engine reduced a 2.5-hour, 21-million-token task to 25 minutes and 10 million tokens. The talk offers a practitioner's guide to building context engines with conflict resolution, personalization, and access control.

Human-in-the-Loop Automation with n8n — Liam McGarrigle

May 2, 2026 · 1:19:22

Liam McGarrigle of n8n shows how to build a secure human-in-the-loop automation agent using n8n's visual workflow system, with a Gmail and Google Calendar management agent as the concrete example. He walks through wiring a chat trigger, an AI agent with simple memory, and tools that the agent can call, such as sending emails and creating calendar events. The key addition is a human review node placed between the agent and destructive tools, which intercepts actions and presents them for approval via chat—preventing accidental sends or event creation. McGarrigle emphasizes naming nodes and writing tool descriptions to guide the LLM correctly, and demonstrates using expressions to format readable approval messages. He also covers extending the agent to Slack, adding scheduled runs for autonomous hourly inbox checks, and using sub-agents for specialized tasks. Additional topics include n8n's native MCP server for integration with Claude Code, enterprise Git-based environments for workflow management, and building custom REST APIs within n8n. The session focuses on giving developers observability and control over AI workflows, ensuring agents are not black boxes.

Software Engineering Is Becoming Plan and Review — Louis Knight-Webb, Vibe Kanban

May 2, 2026 · 20:23

Louis Knight-Webb, founder of Vibe Kanban, argues that software engineering is becoming all about planning and reviewing AI-generated code, not writing it. He presents two approaches: plan-heavy (spending time on specs to minimize review) and review-heavy (letting agents run and correcting output), noting five minutes of planning saves thirty minutes of review. As agents run longer—crossing the five-minute threshold—engineers must parallelize multiple agents and manage workflows, which Vibe Kanban aimed to facilitate. Knight-Webb announces shutting down Vibe Kanban despite 30,000 monthly active users because it resold tokens to enterprises and struggled to make money, advising that enterprise sales are crucial. He reflects on the journey positively, emphasizing learning to work with great people and the importance of hard work.

Building Conversational Agents — Thor Schaeff and Philipp Schmid, Google DeepMind

Apr 30, 2026 · 1:47:34

Thor Schaeff and Philipp Schmid of Google DeepMind demonstrate building conversational agents with the Gemini Interactions API and Live API, arguing that the new Interactions API simplifies agent development by introducing server-side state management, unified content blocks, and remote MCP support. They walk through creating a coding agent with read/write file and bash tools using Gemini 3 Flash, highlighting how the model uses tool calling to execute commands and return results. The Live API, powering real-time voice and video interactions, supports native multilingual audio, tool calling, and Google Search grounding, though the presenters acknowledge its current limitations in production, including lack of session transcript retrieval and shorter context windows (15 minutes audio-only). They showcase a Live Jukebox demo integrating Lyria 3 music generation and discuss business use cases like Shopify Sidekick, Waymo, and elderly companion apps, emphasizing that while the technology is ready for experimentation, production readiness depends on partner integrations for observability and compliance.

Scaling GitHub for your Agents — Sam Morrow, GitHub

Apr 27, 2026 · 20:35

Sam Morrow, GitHub's MCP server lead, details the architectural challenges and solutions for scaling a remote MCP server to 7 million weekly tool calls. He explains how tool proliferation degraded agent performance—LangChain's research confirmed more tools confuse agents—leading to innovations like tool sets and dynamic discovery, though 100% of users stuck with defaults. To reduce context, GitHub cut tool descriptions by 49% and trimmed output tokens by 75% on list pull requests. Security is addressed via OAuth 2.1 with PKCE and step-up auth; they rejected dynamic client registration to avoid unbounded app databases and rate-limiting issues. The stateless server uses Redis for session storage and builds a fresh server instance per request, enabling horizontal scaling without session affinity. Metrics include 11 million Docker downloads, 30,000 stars, 4,000 forks, and 126 contributors. Morrow predicts compositional tools and automatic server discovery will make thousands of tools the norm.

Gateways are All You Need — Karan Sampath, Anthropic

Apr 27, 2026 · 17:48

Karan Sampath, an engineer at Anthropic, argues that enterprises face a 'three-headed hydra' of observability, access control, and security when adopting MCPs, and proposes that unified gateways are the solution. He explains that current decentralized MCP deployments create bottlenecks, limiting enterprises to single-digit MCPs due to opaque usage, lack of role-based scoping, and data exfiltration risks. The gateway, a middleware layer handling auth, proxying, routing, and tunnels, establishes a root of trust. Core components include OAuth integration, a developer CLI, and an internal sub-registry. Benefits include multi-surface integration (Claude.ai, Claude Code), standardized access control, faster iteration without repeated security reviews, and scalable credential management. Sampath envisions decoupling agent architecture from data layers, making gateways invariant as agent surfaces evolve.

MCP = Mega Context Problem - Matt Carey

Apr 25, 2026 · 22:42

Matt Carey from Cloudflare argues that the real bottleneck in connecting agents to APIs via MCP is the context window, not the protocol itself. Cloudflare’s OpenAPI spec is 2.3 million tokens; naive tool generation would consume 1.1 million, overwhelming any agent. To solve this, Carey advocates progressive discovery: CLIs (requiring shell access), tool search (loading only relevant tools), or codemode—letting agents write code against a typed SDK and execute it in Cloudflare’s isolated Workers sandbox. This sandbox provides safe, programmable guardrails (no secrets, limited network access), allowing agents to access all 2,600+ Cloudflare API endpoints from a single MCP server. Carey predicts that as agents become code generators, infrastructure primitives like WorkerD, Deno, and Pydantic will proliferate, and MCP servers will become a lightweight middleware flag (e.g., `MCP: true` in Next.js). The talk demonstrates read-only access to the entire Cloudflare API from an MCP client, showing codemode in action.

"Software Fundamentals Matter More Than Ever" — Matt Pocock

Apr 23, 2026 · 18:26

Matt Pocock argues that software fundamentals matter more than ever in the AI era, directly countering the 'specs-to-code' movement that treats code as cheap. He shares practical skills like 'Grill Me' to reach a shared design concept with AI, a ubiquitous language from domain-driven design to align terminology, and test-driven development to force small, deliberate steps. Drawing on John Ousterhout's 'deep modules' and Frederick Brooks's 'design concept,' he advocates structuring codebases with simple interfaces behind which AI can implement freely. He warns that bad code is the most expensive it's ever been, as it blocks the productivity gains AI can offer. His reusable skills are available in the GitHub repo MattPocock/skills.

The End of Apps — Kitze, Sizzy.co

Apr 23, 2026 · 20:18

Kitze, creator of Benji and founder of Tinker Club, argues that current productivity apps and AI agents are unreliable and too complex for most users, predicting a future where AI prompts humans instead of the reverse. He traces his 24-year obsession with productivity tools—from a childhood to-do list to building Benji with 60 features—and his move to self-hosting after the ChatGPT moment. Kitze critiques custom agents like OpenClaw as too tinkerer-focused, cloud agents like ChatGPT as too nerfed, and believes local models on devices (Apple, Google Pixel) will win for normies, enabling AI to automatically manage notifications, emails, and tasks. He shares his own experiment, Wolfer, designed for predictable multi-agent orchestration without memory systems, using nested topics for context. The episode delivers a strong claim: the role of AI will invert, with machines prompting humans for decisions while handling all background work.

Agents need more than a chat - Jacob Lauritzen, CTO Legora

Apr 22, 2026 · 14:21

Jacob Lauritzen, CTO of Legora, argues that vertical AI agents handling complex work need collaboration interfaces beyond simple chat. He introduces the verifier's rule—tasks easy to verify get solved by AI—and shows how legal work spans from easy-to-verify contract definitions to impossible-to-verify litigation strategy. Lauritzen explains that increasing trust requires making tasks verifiable (e.g., test-driven development, proxy verification with golden contracts) or adding guardrails, while increasing control demands moving from low-control agent trees to planning, then to skills and elicitation where agents ask humans for decisions. He criticizes chat as one-dimensional and low-bandwidth, advocating for high-bandwidth persistent artifacts like documents and tabular reviews where humans can highlight clause three or flag items, enabling effective human-agent collaboration and progressive discovery without losing context.

How AI is changing Software Engineering: A Conversation with Gergely Orosz, @pragmaticengineer

Apr 21, 2026 · 26:42

Gergely Orosz explains how AI is reshaping software engineering, from token maxing at Meta, Microsoft, and Salesforce—where engineers use AI tools to inflate token counts out of fear of layoffs and performance reviews—to the broader shift in the engineer's role toward orchestrating agents rather than managing people. He notes that big tech companies like Uber, Airbnb, and Shopify are building custom internal AI infra (MCP gateways, coding agents) to stay ahead, even tolerating high churn for a six-month competitive edge. Shopify secured GitHub Copilot a year early by offering feedback from 3,000 engineers. Gergely also shares how The Pragmatic Engineer hit product-market fit: 100 paid subscribers before publishing, reaching 1,000 in six weeks, and becoming the #1 paid tech newsletter by focusing on two deep-dive articles per week for two years.

Taste & Craft: A Conversation with Tuomas Artman, CTO Linear & Gergely Orosz, @pragmaticengineer

Apr 21, 2026 · 29:17

Tuomas Artman, CTO of Linear, warns that AI's ability to instantly ship features risks creating convoluted, low-quality software, arguing that taste and design must guide development. He explains Linear's culture of deliberate product decisions, including a 'zero bug policy' where bugs are fixed within hours, and 'Quality Wednesdays' where engineers find and fix one minute detail each week—resulting in over 2,500 quality fixes. Artman notes that AI lacks human 'taste' and cannot feel user experience nuances like animation timing. He predicts all software engineers will become product engineers, needing to focus on customer needs and UX, and advises aspiring product engineers to build for themselves, talk to customers, and study Apple's Human Interface Guidelines.

Full Workshop: Build Your Own Deep Research Agents - Louis-François Bouchard, Paul Iusztin, Samridhi

Apr 20, 2026 · 1:57:03

Louis-François Bouchard, Samridhi Vaid, and Paul Iusztin built a combined system of a deep research agent and a deterministic writing workflow to automate high-quality technical content creation, trading flexibility for control. The research agent uses an MCP server (FastMCP) with tools for Google search, YouTube analysis, and report compilation, powered by Gemini and orchestrated by Claude Code. The writing workflow is a static pipeline that loads a user guideline, writing profiles (structure, terminology, character), and few-shot examples into a system prompt, then applies an evaluator-optimizer loop: a reviewer outputs structured pydantic objects with profile/location/comment to flag violations, and the editor applies them in priority order (guideline > research > profile). They stress that research demands autonomy while writing needs constraint, and show how to avoid AI slop by banning specific words and using profiles. For observability, they use Opik to capture all traces, and for evals they built a 20-sample dataset from real LinkedIn posts, split into train/dev/test, to calibrate an LLM judge (binary pass/fail) that they run on dev and test splits to measure F1 and detect…

The Friction is Your Judgment — Armin Ronacher & Cristina Poncela Cubeiro, Earendil

Apr 18, 2026 · 18:38

Armin Ronacher (creator of Flask) and Cristina Poncela Cubeiro argue that while AI coding agents promise frictionless shipping, the resulting loss of human judgment introduces dangerous technical debt and brittleness. They describe the psychological trap of addictive speed and the engineering challenge of agents producing slop that creates hidden failure conditions. To counter this, they advocate for agent-legible codebases that modularize both code components and flow, enforce strict linting rules such as no bare catch-alls and unique function names, and deliberately reintroduce friction in high-stakes decisions like database migrations and permission changes. They note that agents excel at libraries with simple cores but struggle with products built on intertwined concerns, and that human responsibility must remain central as code production outpaces review capacity.

State of the Claw — Peter Steinberger

Apr 17, 2026 · 44:12

Peter Steinberger, creator of OpenClaw, presents a five-month update on the world's fastest-growing open-source project. He details the project's staggering growth—30,000 commits, nearly 2,000 contributors—and the immense security burden: 1,142 advisories (16.6 per day) with 99 critical, often AI-generated slop that demands human vetting. Steinberger refutes media fearmongering, citing how researchers ignore security docs to fabricate scary scenarios. He clarifies OpenAI did not buy OpenClaw; he joined the company while establishing the OpenClaw Foundation to remain vendor-neutral. He emphasizes the importance of local models for data sovereignty and describes his coding workflow of running 5-6 agent sessions simultaneously, iterating on taste and personality. Future visions include ubiquitous agents, 'Dreaming' for memory reconciliation, and modular plugins. For engineers, he champions taste, system design, and learning to say no.

Harness Engineering: How to Build Software When Humans Steer, Agents Execute — Ryan Lopopolo, OpenAI

Apr 17, 2026 · 46:21

Ryan Lopopolo, a member of technical staff at OpenAI, argues that software engineers must shift to 'harness engineering' where agents produce all code and humans focus on delegation, system design, and guardrails. He describes how his team banned editors, using Codex with skills to manipulate apps, and reduced review friction by automating lint rules and review agents. Lopopolo explains that code is free but human time and model context are scarce, so teams should structure repositories for consistency and use 'garbage collection days' to eliminate recurring slop. He shares practical techniques like writing tests about source code (e.g., file length limits) and using review agents that inject prompts. His future vision involves agents autonomously executing long-horizon work based on token budgets and success metrics, freeing humans for higher-level activities.

How METR measures Long Tasks and Experienced Open Source Dev Productivity - Joel Becker, METR

Jan 19, 2026 · 1:15:52

Joel Becker of METR argues that AI benchmark scores are soaring while real-world developer productivity barely budges, reconciling this gap by presenting METR's time horizon measurements and a 16-developer RCT showing no significant speedup from AI on mature open-source projects. He cites reliability issues, task distribution mismatches, and the J-curve effect from Meta's developer data, noting that experienced users still fail to accelerate. The episode explores why AI struggles with messy enterprise data (e.g., LinkedIn's 5,000 'impressions' tables), the failure of computer-use agents in 'Agent Village,' and the possibility that hardware constraints may slow progress. Becker also previews new metrics like 'watched versus unwatched' time horizons and future studies on greenfield coding and data science.

Your MCP Server is Bad (and you should feel bad) - Jeremiah Lowin, Prefect

Jan 12, 2026 · 54:33

Jeremiah Lowin, CEO of Prefect and creator of FastMCP, argues that most MCP servers are poorly designed because they are simply REST API wrappers, leading to confused LLMs and wasted tokens. He presents a framework for agent-native product design based on three dimensions—discovery, iteration, and context—and offers five actionable principles: outcomes over operations, flatten arguments, treat instructions as context, respect the token budget, and curate ruthlessly. Lowin emphasizes that agents fail when given too many tools or complex arguments, citing a hard limit of 50 tools per agent for reliable performance. He shares real-world examples, including a company with 800 endpoints that could not be effectively exposed, and critiques client implementations like Claude Desktop for not respecting spec-compliant features. The central takeaway: MCP servers should be designed as user interfaces for agents, not as developer APIs.

Automating Large Scale Refactors with Parallel Agents - Robert Brennan, OpenHands

Jan 8, 2026 · 1:16:21

Robert Brennan, CEO of OpenHands, and colleague Calvin explain how parallel agents can automate large-scale code refactors, demonstrating that breaking tasks into agent-sized batches with human oversight achieves 90% automation. They trace the evolution from context-unaware snippets to autonomous coding agents and now agent orchestration, where multiple agents work in parallel on tasks like CVE remediation — one client saw a 30x improvement in time-to-resolution. Calvin shows a pipeline using OpenHands' SDK to batch code files, run a verifier to detect code smells, and spin up fixer agents that generate focused pull requests. Brennan details strategies: task decomposition into single-PR-sized chunks, dependency ordering, and context sharing via agent.md files. The episode closes with a live-coding walkthrough building a parallel agent script that scans for vulnerabilities and opens individual PRs.

Build a Prompt Learning Loop - SallyAnn DeLucia & Fuad Ali, Arize

Jan 6, 2026 · 52:08

SallyAnn DeLucia and Fuad Ali from Arize present prompt learning, a method to continuously improve AI prompts by using feedback from evals and human annotations. They argue that agent failures often stem from weak instructions rather than weak models, and that adding rules to system prompts can yield 15% improvements on SWE-Bench without fine-tuning or architecture changes. The talk compares prompt learning to GEBA, noting it achieves better results in fewer loops by leveraging text explanations. A case study shows Kline's performance improved from 30% to 45% on SWE-Bench-Lite after optimizing prompts. The workshop demonstrates building an optimization loop that ingests input-output pairs, evaluates them, and iteratively refines the system prompt using OpenAI.

Building durable Agents with Workflow DevKit & AI SDK - Peter Wielander, Vercel

Jan 6, 2026 · 1:09:49

Peter Wielander from Vercel introduces the Workflow DevKit, an open-source library that adds durability, observability, and resumability to AI agents by wrapping them in a workflow pattern. He demonstrates converting a coding agent into a workflow-supported agent, showing how steps like LLM calls and tool executions become isolated, retryable units. The toolkit enables long-running agents that can sleep for days, resume streams after disconnection, and integrate human-in-the-loop via webhooks. Deployable on Vercel or any cloud, it provides built-in observability through a local UI. The Workflow DevKit is currently in beta with general availability targeted for January.

Claude Agent SDK [Full Workshop] — Thariq Shihipar, Anthropic

Jan 5, 2026 · 1:52:25

Thariq Shihipar of Anthropic presents the Claude Agent SDK, arguing that Bash and file-system-based agents outperform traditional tool-only approaches for autonomous tasks. He defines agents as systems that build their own context and trajectory, contrasting with structured workflows. The SDK, built on Claude Code, emphasizes the Bash tool as the most powerful primitive for composability and code generation, enabling non-coding tasks like data analysis. He demonstrates live-coding a Pokémon team advisor that dynamically fetches API data via scripts, and explains security through a 'Swiss cheese defense' of model alignment, AST parsing, and sandboxing. Shihipar also covers skills for progressive context disclosure, sub-agents for parallel work, and hooks for deterministic verification, stressing that agent building is an art of reading transcripts and iterating on context engineering.

How Claude Code Works - Jared Zoneraich, PromptLayer

Dec 26, 2025 · 1:05:43

Jared Zoneraich from PromptLayer dissects how Claude Code works, arguing its success stems from a radical simplification: a single-threaded master loop with tool calls, relying on capable models rather than complex DAG-based architectures. He breaks down the core internal tools—Bash, FileEdit, Grep—and the to-do planning mechanism that is purely prompt-based and not deterministically enforced. Zoneraich contrasts Claude Code with other coding agents: Codex (Rust core, kernel-level sandboxing), Amp (handoff context management, model-agnostic), and Cursor Composer (UI-first, distilled model). He emphasizes that Bash is the most important tool, acting as a universal adapter, and that context management via sub-agents and summarization is critical. Key takeaways include trusting the model, simple design, rigorous tool testing, and that different agent philosophies (not one winner) will serve different use cases.

Dispatch from the Future: building an AI-native Company – Dan Shipper, Every, AI & I

Dec 18, 2025 · 17:58

In this episode, Dan Shipper of Every argues that there is a 10x difference between an organization where 90% of engineers use AI and one where 100% do. At Every, 99% of code is written by AI agents, enabling 15 people to run four software products with 7,000 paying subscribers, each built by a single developer. Shipper introduces 'Compounding Engineering', where each feature makes the next easier to build through a loop of plan, delegate, assess, and codify. He describes how this approach allows managers to commit code, enables tacit code sharing across products, and lets new hires be productive on day one. The episode details how Every's shift to agentic workflows (Claude Code, Codex) and a 'demo culture' over 'memo culture' has transformed their engineering velocity and collaboration.

From Vibe Coding To Vibe Engineering – Kitze, Sizzy

Dec 14, 2025 · 25:28

Kitze, founder of Sizzy, argues that "vibe engineering"—actively steering AI agents with technical knowledge—trumps passive "vibe coding" for building real software. He shares how Cursor's Composer 1 let him port Benji.so and Glink to Next.js 16 with Monorepo in under a week, reviving near-dead projects. Kitze insists LLMs excel at React because humans are bad at it, and that repetitive code is fine since agents don't care. He warns against giving AI tools to juniors without oversight, and predicts the bottom of the job market will thin as agents replace interns. The talk ends with a pitch for "vibe code fixers" as a new role, maintaining legacy systems like cowboy coders.

Proactive Agents – Kath Korevec, Google Labs

Dec 13, 2025 · 16:51

Kath Korevec, Director of Product at Google Labs, argues that AI coding agents must become proactive rather than reactive to truly reduce developer cognitive load. She introduces Jools, a proactive autonomous coding agent that observes workflows, personalizes responses, and intervenes at the right moment. Korevec details three levels of proactivity: level one auto-fixes issues during tasks; level two learns project context; level three connects agents across code, design, and data. She highlights features like memory, a critic agent for code quality, verification via Playwright, and a to-do bot. A demo shows Jools indexing a codebase and suggesting high-confidence tasks. Korevec ties this to her personal Halloween animatronic project, where she wished Jools handled debugging so she could focus on creative LED animations.

2026: The Year The IDE Died — Steve Yegge & Gene Kim, Authors, Vibe Coding

Dec 6, 2025 · 24:59

Steve Yegge and Gene Kim argue that by 2026, traditional IDEs will be obsolete, replaced by AI-driven coding agents. Yegge claims current tools like Claude Code are too hard for most developers, with high cognitive overhead and a single-agent bottleneck akin to a solo diver running out of oxygen. He predicts next year will bring multi-agent systems, comparing the shift from manual tools to CNC machines, and provocatively states that engineers using an IDE after January 1st are bad engineers. Gene Kim introduces their 'Vibe Coding' book and shares case studies: Booking.com saw double-digit productivity gains, Travelopia replaced a legacy app in six weeks with a small team, and a Fidelity leader Vibe Coded an app in five days that previously was estimated to take five months. They present the FAFO framework (Faster, Ambitious, alone/autonomous, Fun, Optionality) and cite DORA research showing trust in AI grows with usage. The episode concludes that Vibe Coding will reshape organizations, enabling non-developers to ship features and potentially compressing team sizes.

Katelyn Lesse – Evolving Claude APIs for Agents, Anthropic

Dec 4, 2025 · 13:25

Katelyn Lesse, who leads the Claude Developer Platform team at Anthropic, explains how the platform is evolving to help developers build powerful agentic systems using Claude. She details three key areas: harnessing Claude's capabilities through features like extended thinking and tool use, managing context with MCP, a memory tool, and context editing (which combined to yield a 39% performance bump), and giving Claude a computer via the code execution tool and agent skills. The talk also covers challenges like container orchestration for Claude Code on web and mobile, and the importance of letting Claude work autonomously in a sandbox environment.

No Vibes Allowed: Solving Hard Problems in Complex Codebases – Dex Horthy, HumanLayer

Dec 2, 2025 · 20:31

Dex Horthy of HumanLayer argues that with deliberate context engineering — specifically a technique he calls "frequent intentional compaction" — today's AI coding agents can handle large brownfield codebases, not just greenfield projects. He presents a three-phase workflow (research, plan, implement) that keeps agents in the "smart zone" of the context window, avoiding the diminishing returns that set in around the 40% mark. Horthy demonstrates this approach solving a real problem in a 300k-line Rust codebase, shipping a week's worth of work in seven hours with code that passed expert review. He cautions against outsourcing thinking to AI, emphasising that plans must include actual code snippets for reliable execution and team alignment. The talk also addresses the cultural rift where senior engineers clean up slop from juniors' AI tools, calling for top-down adoption and team-wide workflow adaptation.

Context Engineering: Connecting the Dots with Graphs — Stephen Chin, Neo4j

Nov 24, 2025 · 26:50

Stephen Chin, VP of Developer Relations at Neo4j, argues context engineering powered by knowledge graphs transforms AI from prompt engineering to information architecture, enabling agents with structured memory and retrieval. He demonstrates GraphRAG using Neo4j's knowledge graph builder with supply chain and VEX documents, showing how vector and graph algorithms retrieve specific Jackson library vulnerabilities with CVE details, severity, and remediation versions. Chin contrasts fast two-pass vector+graph retrieval with agentic multi-step traversal via the Neo4j MCP server and Claude Code, which yields deeper results like CVE number, attack type, and upgrade paths. He explains that graphs excel when relationships span two or more facts, using a presentation update example involving himself, colleague Sid, and the GIDS event. Practical applications include role-based access overlays and explainable AI via visualized conversation flows. Resources like free Graph Academy courses, Nodes AI 2026 conference, and graphrag.com support implementation.

Infra that fixes itself, thanks to coding agents — Mahmoud Abdelwahab, Railway

Nov 24, 2025 · 18:08

Mahmoud Abdelwahab from Railway presents Railway Autofix, a system that uses Inngest for durable execution and OpenCode as a coding agent to automatically detect infrastructure issues and open pull requests to fix them. The system runs scheduled workflows to fetch project architecture, resource metrics, and HTTP performance data, then analyzes services exceeding thresholds. For affected services, it gathers additional context like logs and generates a detailed fix plan using an LLM. The coding agent then implements the plan and opens a PR with a summary of changes. The talk demonstrates how this shifts developers from manually investigating alerts to simply reviewing auto-generated fixes.

Context Platform Engineering to Reduce Token Anxiety — Val Bercovici, WEKA

Nov 24, 2025 · 23:52

WEKA's Val Bercovici and Callan Fox present Context Platform Engineering, open-sourcing a toolkit that maximizes KV cache hit rates—called the single most important metric for production-grade AI agents by Manus AI. They argue that without this engineering, users resort to Context Financial Engineering, a clairvoyant prompt cache arbitrage against time-to-live (5 minutes to 1 hour) and cache read/write pricing. The toolkit includes a load generator that configures agent swarms with specific SLOs, cycling through deterministic and random prompts. Callan's WEKA Labs research shows that 1-minute TTL causes 15-16x token repetition, while 1-hour TTL approaches 1x, but requires larger cache capacity. Benchmarks compare HBM+WEKA (purple) vs HBM+DRAM (orange) vs HBM+DRAM+slow storage (pink); WEKA's NVMe-backed augmented memory grid maintains higher output token rates under increasing concurrent users—up to the point where DRAM tiers drop off sharply due to insufficient speed. The episode covers how SLA requirements translate into SLOs via memory tiering and KV cache offloading, and emphasizes that subscription users effectively purchase cache allotments to keep inference providers in…

Hacking Subagents Into Codex CLI — Brian John, Betterup

Nov 24, 2025 · 13:39

Brian John, Principal Full Stack Engineer at Betterup, explains how to hack subagents into OpenAI's Codex CLI by using a 72-line wrapper script that launches child Codex processes as subagents, enabling context management without polluting the main agent's window. He details the minimum sandbox permissions required—workspaceright on both parent and child, plus disabling the rollout recorder—and notes that security risk is low per Meta's Rule of 2, though not zero. The wrapper uses file-based communication to avoid repetitive permission prompts, and subagents are defined in an agents.md file with configurable reasoning effort. John demonstrates a word counter and file writer agent, noting that Codex runs subagents serially and slower than Claude Code (e.g., 40 seconds for a simple task, up to 20 minutes for large codebases), but considers this acceptable for Codex's unattended design.

Developing Taste in Coding Agents: Applied Meta Neuro-Symbolic RL — Ahmad Awais, CommandCode

Nov 24, 2025 · 20:52

Ahmad Awais, founder of Langbase, introduces CommandCode, a coding agent that learns a developer's personal taste and preferences to generate code that feels like their own. Built on Langbase's infrastructure (deploying over 350K agents and 1.2B monthly runs), CommandCode uses a Meta Neuro-Symbolic architecture with reinforcement learning to continuously capture a programmer's invisible architecture of choices—such as preferring TypeScript, PNPM, TSUP, and Commander over other tools—without requiring explicit rule files. Awais demonstrates how CommandCode automatically applies these learned preferences when building a CLI, contrasting it with Claude Code which generates generic output. He argues that taste models represent the next frontier, enabling agents to evolve with the developer and even share tastes across teams or borrow preferences from other developers like design engineers.

What Data from 20m Pull Requests Reveal About AI Transformation — Nick Arcolano, Jellyfish

Nov 24, 2025 · 17:57

Nick Arcolano, head of research at Jellyfish, presents data from 20 million pull requests across 200,000 engineers at 1,000 companies to reveal what AI transformation actually looks like. Median AI adoption of interactive coding tools (Copilot, Cursor, Claude Code) jumped from 22% in June 2024 to nearly 90% today, driving a 2x increase in PR throughput and 24% faster cycle times on average. Autonomous agents like Devin and Codex remain negligible—only 44% of companies have tried them, contributing less than 0.2% of PRs. PRs grow 18% larger with AI, but bug rates and reverts show no significant quality impact. Crucially, code architecture determines success: centralized and balanced repos see 4x throughput gains, while highly distributed architectures (many repos per engineer) show no correlation with AI adoption, likely due to cross-repo context limitations. Arcolano advises focusing on adoption first, then addressing architectural barriers to unlock productivity.

Form factors for your new AI coworkers — Craig Wattrus, Flatfile

Aug 22, 2025 · 15:35

Craig Wattrus, AI design engineer at Flatfile, argues that designing AI interactions requires treating AI as a new coworker with distinct form factors — invisible, ambient, inline, and conversational — rather than traditional UI. He demonstrates how feeling the material by building tools like a 'chat tuner' to tune AI character shifts from helicopter-parenting to character coaching. Finding the grain involves designing tool UX where AI communicates visually, checks alignment, and hands back control. Courting emergence yields unexpected results: an agent combined two files without being asked and suggested the user contact HR for missing employee IDs. Looking forward, Wattrus explores auto-complete for data transformations, using LLMs to suggest fixes alongside human oversight.

Vibe Coding with Confidence — Itamar Friedman, Qodo

Aug 6, 2025 · 20:55

Itamar Friedman, CEO of Qodo, argues that the command-line interface (CLI) is the future for achieving 'vibe coding with confidence' in enterprise software development. He contrasts vibe coding's greenfield success with the challenges of large codebases, where context, workflows, and multi-agent collaboration are essential. Friedman explains Qodo's multi-agent architecture, including tools for code review (Qodo Merge) and coverage (Qodo Cover), and announces the soft launch of a new CLI tool that lets developers create and pipe agents, supporting MCP and A2A protocols. He envisions a future 'swarm of agents' each specialized with credentials and best practices, enabling end-to-end workflows across the SDLC. The episode emphasizes moving from chat-based agents to command-driven multi-agent systems for reliable, high-quality code generation.

How to Improve your Vibe Coding — Ian Butler

Aug 3, 2025 · 7:30

Ian Butler (CEO of BizMyth) presents benchmark data showing that popular coding agents like Devin and Cursor have less than 10% true positive rates for finding bugs, with Cursor producing a 97% false positive rate across 100+ repos and 1,200+ issues. He argues that these agents overrun codebases with false positives, causing alert fatigue and bugs reaching production. To improve vibe coding, he recommends adding bug-focused rules (e.g., OWASP Top 10, explicit bug classes like auth bypasses), managing context by feeding diffs and preventing file summarization, and using thinking models which were significantly better at finding deeper bugs. However, even thinking models show high variability in which bugs they find across runs, indicating agents never holistically examine a file. The talk concludes with a plug for BizMyth.sh, which automates PRs and vulnerability scanning.

3 ingredients for building reliable enterprise agents - Harrison Chase, LangChain/LangGraph

Jul 23, 2025 · 20:55

Harrison Chase, CEO of LangChain, argues that reliable enterprise agents depend on three ingredients: high value when right, high probability of success, and low cost when wrong. He says agents succeed by tackling high-value tasks like legal or research, and by doing more work through long-running processes. Reliability improves by blending deterministic workflows with agentic loops, as in LangGraph, and by using observability (LangSmith) to build trust. Cost is lowered by reversible actions (e.g., code diffs) and human-in-the-loop patterns like approvals and time travel. Chase predicts a future of ambient agents—event-triggered, human-supervised background agents—and notes code agents lead because code is verifiable and easily revertible.

A Taxonomy for Next-gen Reasoning — Nathan Lambert, Allen Institute (AI2) & Interconnects.ai

Jul 19, 2025 · 19:21

Nathan Lambert, senior research scientist at AI2 and founder of Interconnects.ai, argues that next-generation reasoning models require a taxonomy of four traits: skills, calibration, strategy, and abstraction. While current models excel at math and code (skills), they overthink easy problems, wasting tokens and latency—calibration is needed to match output length to task difficulty. The real frontier is planning: models must learn strategy (choosing the right direction) and abstraction (breaking problems into tractable sub-tasks) to enable long-horizon agents like Deep Research and Claude Code. Lambert traces how OpenAI's Q*→Strawberry→o1 took 12–18 months of human data to teach backtracking and verification; planning should be easier because humans can write 5-step plans. He predicts post-training compute could reach parity with pre-training, citing DeepSeek's shift from 0.18% post-training compute in V3 to an estimated 10–20% for R1. The path forward: collect diverse verifiable questions, filter by difficulty, run stable RL—then scale.

Training Agentic Reasoners — Will Brown, Prime Intellect

Jul 7, 2025 · 19:17

Will Brown of Prime Intellect argues that reasoning and agents are fundamentally the same, and reinforcement learning (RL) is the key to advancing both. He explains that RL now works at scale, as shown by DeepSeek's GRPO and OpenAI's o3, and that agentic tasks like tool calling are natural RL environments. Brown warns against reward hacking and emphasizes designing evals that are harder to game than the task itself. He introduces his open-source toolkit 'verifiers' (now on pip) which lets users build trainable agent loops with a simple API, and demonstrates training a 7B Wordle agent in a few turns on just a couple GPUs.

Claude Code & the evolution of agentic coding — Boris Cherny, Anthropic

Jul 4, 2025 · 18:12

Boris Cherny, creator of Claude Code and Anthropic member of technical staff, argues that the product stays intentionally unopinionated and minimal because the model's coding capabilities are improving exponentially and the right UX remains unknown. He traces programming UX from 1950s punch cards to 1980s Smalltalk's live reload, Eclipse's static-analysis autocomplete, Copilot, and Devin's natural-language paradigm. Claude Code's terminal-first approach works in any terminal, over SSH, and inside VS Code or Cursor, with a GitHub integration that keeps data on user compute and a programmatic SDK for custom UIs. Key tips include using Claude for code-based Q&A (shortening onboarding from weeks to days), teaching it tools via `--help` and Claude MD files, leveraging TDD with visual iteration, and running multiple Claude instances in parallel via terminal tabs or GitHub Actions. Today's launch adds a plan mode triggered by Shift+Tab, which makes a plan and waits for approval before writing code.

Turning Fails into Features: Zapier’s Hard-Won Eval Lessons — Rafal Willinski, Vitor Balocco, Zapier

Jun 30, 2025 · 16:15

Zapier AI Tech Lead Rafal Willinski and Staff Engineer Vitor Balocco explain how Zapier's evaluation system turns agent failures into targeted improvements through a data flywheel. They detail collecting explicit feedback at critical moments and mining implicit signals like testing behavior, cursing, and user follow-ups. The pair advocates building unit test evals for specific failure modes, then trajectory evals and LLM-as-judge with rubrics to avoid overfitting and capture multi-turn criteria. They share that over-indexing on unit tests hurt model benchmarking, and reasoning models can compare model runs, revealing differences like Claude as a decisive executor versus Gemini's yapping. Ultimately, they argue that the goal is user satisfaction, so A/B testing on a small traffic fraction is the ultimate verification.

Containing Agent Chaos — Solomon Hykes, Dagger

Jun 28, 2025 · 23:48

Solomon Hykes, creator of Docker and founder of Dagger, argues that containing agent chaos requires engineering reproducible execution workflows built on containerization applied to each step of an agent's workflow. He introduces 'container use'—agents developing inside fully isolated, customizable environments rather than just sandboxing outputs—and demonstrates a prototype using MCP to integrate with Claude Code and Goose. The system provides background work, rails, seamless human stepping in, and optionality by leveraging Dagger, Git-based state management, and ephemeral containers snapshotted per action. Hykes shows how agents can run parallel experiments, merge snapshots, and discard failed environments without pollution. The episode concludes with him open-sourcing the project as github.com/dagger/containeruse.

Agents reported thousands of bugs, how many were real? - Ian Butler and Nick Gregory

Jun 3, 2025 · 18:39

Ian Butler and Nick Gregory introduce SM-100, a benchmark of 100 real-world bugs across 84 repositories, and present their agent Bismuth as outperforming existing agents in detecting complex bugs—finding 10 needles vs 7 for the next best. However, even top performers struggle: Bismuth's true positive rate is only 25%, and Codex leads at 45%, while most agents (Cursor, Debian) report thousands of false positives (97% for basic loops). They argue that agents universally exhibit narrow thinking, missing simple bugs like a form state issue that only Bismuth and Codex caught, and that high SWE-bench scores do not translate to maintenance tasks. The episode concludes that software maintenance remains an unsolved problem requiring deeper reasoning and targeted search.