How long can your skills be before your agent forgets what you told it? — Laurie Voss, Arize AI
Sep 9, 2026 · 22:26
Laurie Voss, head of developer relations at Arize AI and co-founder of npm, argues the 200-instruction ceiling for skills files collapsed 10x in a year. Rerunning the IFScale benchmark, which asks a model to write a report containing exact words and counts how many appear, he replicated last year's 200-to-300 drop, then watched current models ace it. He had to raise the test from 500 words to 10,000 before anything broke: the boundary now sits near 2,000 instructions, closer to 5,000 for the best. Failures diverged: DeepSeek forgets, Claude refuses when random words trip its safety filter, Gemini overthinks and answers nothing, and GPT-5.5 writes half a report then calls the request stupid. Compression is solved; verification—checking outputs with evals—is now the hard part.
