It’s Tokens All The Way Down: How RLMs are Different — Kevin Madura, AlixPartners
Sep 9, 2026 · 20:48
Kevin Madura, director of advanced technology at AlixPartners, argues that recursive language models (RLMs) mark a fundamentally different way for LLMs to handle context: instead of attending to every token, the model treats its context as a variable in a Python REPL and delegates subtasks to sub-LMs, including itself. He traces RLMs to early work by Omar Khattab and Alex Zhang, citing the Oolong and BrowseComp benchmarks, where an RLM beats GPT-5 tool calling at lower cost, and the long chain of thought benchmark, where accuracy jumps from 2.6 to 45.4 percent. Unlike RAG, which stuffs the context window, or agents that pass JSON strings back and forth, an RLM keeps logic, execution, and results in one environment, avoiding context rot. Madura demonstrates a cohort retention analysis on three data frames where the model reasons in its own REPL and decides when to submit a typed answer. Case studies include Trampoline AI consolidating long invoices without chunking or embeddings, an AWS engineer surfacing patterns in raw logs, Halo optimizing an agent harness from its own traces, and a security report generated across 500,000 lines of code. His closing bet: models post-trained to…
