Diamond

Memory Harnesses for Long-Running Research Agents — Stefania Druga, Sakana.ai

Aug 12, 2026 · 13:04

Stefania Druga, a research scientist at Sakana AI in Tokyo, presents experiments on memory harnesses for long-running research agents running on local models like Qwen 27B and DeepSeek V4 Flash on an M3 Ultra. She frames memory as a write-manage-read control loop, not a database, and tests a recall ladder—no recall, vector RAG, a ranked decisions ledger, and an oracle—across 68 xbench questions. The ranked ledger performed best, beating even gating on whether memory is needed, while the oracle didn't hit max because giving the right memory doesn't force its use. When tasks fit in context, memory only added cost with no accuracy gain, but for long-horizon tasks where answers sit far outside the window, good recall policy became essential and cheaper. She urges treating recall policy as a first-class metric and highlights the broader memory technique landscape, including over 30 runnable cookbooks from Diamond, while noting local models run serially, which is why her Tokyo machine…

AI powered entomology: Lessons from millions of AI code reviews — Tomas Reimers, Graphite

Jul 22, 2025 · 10:21

Tomas Reimers, co-founder of Graphite, discusses the company's AI-powered code reviewer Diamond, arguing that while LLMs can effectively find bugs, they must be carefully prompted to avoid frustrating developers. After analyzing 10,000 comments from codebases, Graphite classified bugs into a quadrant based on what LLMs can catch versus what humans want to receive, identifying bugs, accidentally committed code, performance and security concerns, and documentation mismatches as high-value targets. Code cleanliness and best practice comments, though technically correct, are often unwelcome from AI. Graphite measures success via emoji reactions (less than 4% downvote rate) and the percentage of comments that lead to code changes, reaching 52% actionability in March—matching human-level effectiveness. The talk emphasizes continuous monitoring to ensure LLM comments stay within the desirable quadrant.

Don’t get one-shotted: Use AI to test, review, merge, and deploy code — Tomas Reimers, Graphite

Jun 27, 2025 · 5:45

Tomas Reimers, CPO and co-founder of Graphite, argues that AI-generated code accelerates the inner loop (development) but bottlenecks the outer loop (review, test, merge, deploy), necessitating AI-native tooling across the entire workflow. He presents Graphite's AI code reviewer, Diamond, which achieves a 52% acceptance rate on its comments—exceeding the 45–50% rate for human comments—while only 4% of its comments are dismissed. Reimers emphasizes that AI tools must deeply understand the codebase, integrate with CI/testing pipelines, and provide high-signal, low-noise feedback to handle the increasing volume of AI-assisted code. He cites that 46% of GitHub code is now written by Copilot, and predicts the outer loop will break without AI-driven prioritization, review automation, and optimized deployment. The talk advocates for self-driving code review that lets developers focus on product features rather than manual processes.