Sakana AI

Memory Harnesses for Long-Running Research Agents — Stefania Druga, Sakana.ai

Aug 12, 2026 · 13:04

Stefania Druga, a research scientist at Sakana AI in Tokyo, presents experiments on memory harnesses for long-running research agents running on local models like Qwen 27B and DeepSeek V4 Flash on an M3 Ultra. She frames memory as a write-manage-read control loop, not a database, and tests a recall ladder—no recall, vector RAG, a ranked decisions ledger, and an oracle—across 68 xbench questions. The ranked ledger performed best, beating even gating on whether memory is needed, while the oracle didn't hit max because giving the right memory doesn't force its use. When tasks fit in context, memory only added cost with no accuracy gain, but for long-horizon tasks where answers sit far outside the window, good recall policy became essential and cheaper. She urges treating recall policy as a first-class metric and highlights the broader memory technique landscape, including over 30 runnable cookbooks from Diamond, while noting local models run serially, which is why her Tokyo machine…

Navigating AI’s Frontier in 2025 - Grace Isford, Lux Capital

Mar 13, 2025 · 17:55

Grace Isford, partner at Lux Capital, argues that while 2025 is a 'perfect storm' for AI agents with reasoning models like O3 and R1, cheaper inference, and billions in infrastructure (e.g., Stargate, DeepSeek), agents still fail due to cumulative errors—decision, implementation, heuristic, and taste—exemplified by OpenAI Operator booking a flight incorrectly. She prescribes five strategies: curating proprietary and agent-generated data, building personalized evals for non-verifiable domains (e.g., seat preference), designing scaffolding that prevents cascading failures (citing Ramp's approach), treating UX as the moat (e.g., Codium, Harvey, TLDraw), and building multimodally with voice, smell via Osmo, and touch for embodiment. The talk, recorded at the AI Engineer Summit 2025 in NYC, closes with a call to reframe perfection through visionary product experiences.