Context Engineering

Your agents lack context: Here's how to fix "You're absolutely right!" — Brandon Waselnuk, Unblocked

Sep 9, 2026 · 14:09

Brandon Waselnuk of Unblocked argues that AI-generated code should feel like it was written by someone who has been on your team for years, and that the gap is no longer intelligence but context. He traces how bad context compounds as teams move from tab completion to parallel and background agents, producing correction doom loops, wasted search tokens, and a review tax. Two common fixes stall: the curated context trap, where markdown files rot and someone must curate them for everyone, and the MCP plateau, where an agent may never call the server or stops at the first plausible answer due to satisfaction of search bias, missing last night's Slack correction. A context engine must resolve conflicts between an old architecture diagram and a fresh CTO Slack thread, personalize relevance, enforce permissions, and deliver token-optimized context. Running the same prompt with and without context cut tokens from 21 million to 10.8 million and finished about two hours sooner. He closes by demoing three open source tools, including a workshop that builds a relational context engine from scratch.

It’s Tokens All The Way Down: How RLMs are Different — Kevin Madura, AlixPartners

Sep 9, 2026 · 20:48

Kevin Madura, director of advanced technology at AlixPartners, argues that recursive language models (RLMs) mark a fundamentally different way for LLMs to handle context: instead of attending to every token, the model treats its context as a variable in a Python REPL and delegates subtasks to sub-LMs, including itself. He traces RLMs to early work by Omar Khattab and Alex Zhang, citing the Oolong and BrowseComp benchmarks, where an RLM beats GPT-5 tool calling at lower cost, and the long chain of thought benchmark, where accuracy jumps from 2.6 to 45.4 percent. Unlike RAG, which stuffs the context window, or agents that pass JSON strings back and forth, an RLM keeps logic, execution, and results in one environment, avoiding context rot. Madura demonstrates a cohort retention analysis on three data frames where the model reasons in its own REPL and decides when to submit a typed answer. Case studies include Trampoline AI consolidating long invoices without chunking or embeddings, an AWS engineer surfacing patterns in raw logs, Halo optimizing an agent harness from its own traces, and a security report generated across 500,000 lines of code. His closing bet: models post-trained to…

Why AI Agents Need Million-Token Context — Thomas Wolf & Olive Song, MiniMax

Sep 4, 2026 · 20:48

MiniMax's M3 pairs a functional one-million-token context window with coding and multimodal abilities in a 400-billion-parameter, 20-billion-activated model, explain Olive Song, RL Lead at MiniMax, and Thomas Wolf, co-founder of Hugging Face. Song says short context fails when agents handle multi-round tool responses; MiniMax Sparse Attention — an index branch selecting what matters plus a sparse branch computing on selected blocks — was designed by an intern. She argues multimodal training from the very first step beats post-hoc adapters that harm text performance and risk collapse; interleaved data and reward modeling solved this. Its apps reach over 300 million people in 200 countries; anyone can propose projects, and community feedback and internal agent harnesses now drive M3.1.

Agents' next frontier: agent-to-agent and network effects — Jean-Denis Greze, Town

Sep 3, 2026 · 21:17

Jean-Denis Greze, CTO of Town and former Plaid CTO, argues agent-to-agent is not a useful concept: every LLM system is a search problem, and what matters is whether the right information sits in the context window at the moment of a tool call. The ideal is a single agent with access to all the world's information; privacy, not context length, blocks it — his Coase framing: privacy is the transaction cost. He grades five strategies by how closely each approximates that impossible agent: a shared trust boundary like an HR agent scoped to the most junior access, custom tools that read everyone's mail but return only a connection score, shared silos fed by a sweeper agent, humans as the conduit, and a black box that searches every silo unasked, seeking approval only from the information's owner.

Agentic Sites: Building Hyper Personalized Websites — Carlos Sanchez, Adobe

Aug 29, 2026 · 20:43

Carlos Sanchez, principal scientist at Adobe, argues that agentic sites can deliver hyper-personalized web pages in real time by treating the whole site as a corpus and generating only specific blocks. He demonstrates a coffee-machine site that builds a personalized camping page in 1.64 seconds using Google's Gemma 4 on Cerebras at 2,300 tokens per second. Model choice must be evaluated per site for accuracy and speed, he says, showing 1.1 seconds against 4.6 for the runner-up, and this doesn't need a frontier model because work is choosing and arranging blocks. Marketers define personas in natural language, browsing signals feed an 'audience of one' loop for pre-generated recommendations, and a tool builds an agentic site for any URL in less than an hour.

How to Generate Mergeable Code with a Context Engine — Peter Werry, Unblocked

Aug 27, 2026 · 18:36

Peter Werry of Unblocked argues that access to information does not equal understanding—organizational context is the bottleneck for AI agents. He likens agents to expert engineers resetting knowledge every task, falling prey to radiology's 'satisfaction of search'; million-token windows distract. Werry demos the context engine generating an architecture diagram, then runs the same optimization plan in Claude Code twice: with it, under a dollar and about a minute; without, roughly double the time and more cost, because later steps loop on wrong assumptions. He also shows a review agent boosting comments by seniority, a drop in flagged issues traced to a Slack thread, and open-source tools: a GitHub-history query engine and a social graph showing thin review coverage.

KV Cache-Aware Routing and P/D Disaggregation on Kubernetes — Yuchen Fama & Ashish Kamra, Red Hat

Aug 27, 2026 · 21:48

Yuchen Fama and Ashish Kamra of Red Hat argue that public inference benchmarks hide the chaotic reality of agentic workloads, and show how KV cache-aware routing plus prefill/decode disaggregation in the open source llm-d framework tackles it. Red Hat's traces show agentic sessions running from a few turns to 3,000, cache hit rates over 90%, and input-output token ratios past 100:1, making a 10x cost gap between cached and uncached tokens. A live demo shows routing reusing cache on the same pod cutting time from 3 seconds to 1, while a fresh system prompt pays the full 3 again. For prefill/decode disaggregation, they report P99 inter-token latency dropping from roughly 900 milliseconds to about 100 across 16 H100s serving gpt-oss, but note it only wins in the middle concurrency band and requires RDMA or RoCE to move KV caches. Their closing case study runs GLM 5.2 on H200s with three prefill workers to one decode, achieving 4x faster time to first token and 60% more requests.

Building Agents Is Trivial Now, Context Is the Next Frontier — Jeff Ng, Unblocked

Aug 21, 2026 · 13:22

Jeff Ng, founding engineer at Unblocked, argues building agents is trivial but missing context makes them confidently wrong. Cloud primitives and frameworks have removed plumbing that six months ago took a team a quarter. In a demo, an agent enriching a Linear ticket recommended re-enabling async dispatch, missing that a support engineer had disabled it after an outage; it lacked the Slack thread and postmortem. Background agents fail silently when no human supplies that context. Ng's context engine connects docs, code, tickets, and conversations, reconciles conflicts, and returns a synthesized understanding scoped to permissions. MCP provides access, not understanding; rerunning the same agent grounded in the engine flips its recommendation from repeating the outage to preventing it.

The Last Human Code Review: Building Trust in AI-Generated Code — Itamar Friedman, Qodo

Aug 20, 2026 · 18:54

Itamar Friedman, CEO and co-founder of Qodo, argues the bottleneck in AI-generated code is now context, not models; teams shipping code faster than humans can review are already inside the problem. He splits leaders into two camps — trust every line vs. ship bugs and fix fast — and says code-review benchmarks barely moved, proving context, not reasoning, matters. Qodo's answer is a context engine that codifies tribal knowledge — instruction files, Slack threads, and senior developers' heads — into interfaces humans can audit and agents can consume, shifting review from a single PR to a graph of PRs and contracts. Qodo targets 2027 with zero critical bugs in production, moving from artificial intelligence to artificial wisdom.

Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI

Aug 17, 2026 · 1:03:26

Louis-François Bouchard, Omar Solano, and Samridhi Vaid of Towards AI test context engineering on their AI tutor: keeping full history beat every compaction technique on recall, cost, and latency because 97% of tokens were served from cache (up to 50x cheaper), making summarization a trap unless it shrinks context by more than 50x. Full history recovered specific details 95% vs 32% after summarizing, and distinctive facts survived 800k tokens. Local hardware changes it: a 32k window can't keep everything, and larger models don't widen context. Dense retrieval hit 0% recall at 400k tokens where BM25 got 100%, so they use hybrid retrieval. Rule: name the constraint before compacting.

The Rise of CaaS: Context-as-a-Service for Agentic AI — Omer Primor, Bright Data

Aug 14, 2026 · 22:20

Omer Primor of Bright Data argues that rented AI search and context-as-a-service (CaaS) lose to a self-built pipeline once query volume passes a tipping point. The web is context, not data, and it decays: social content goes stale within a day, news/finance/retail within 30 days, so context is never a one-time snapshot. His test enriching 100 sponsor companies across 25 fields found general search beat dedicated CaaS vendors on coverage, because CaaS only answers from data it already holds; costs were similar, but frequency is the real killer — every repeated query costs the same when nothing changed. He built scrapers for LinkedIn, jobs, and Crunchbase in a day, priced setup at $5,000, and put the crossover just over 15,000 entities. Owned context compounds while rented decays.

Scaling Compute on Context — Jack Morris, Engram

Aug 12, 2026 · 19:42

Jack Morris of Engram frames scaling compute on context as the pursuit of depth in AI, contrasting it with the breadth of public-data pre-training. He argues models trained on public data know nothing about your emails, meetings, or company, and that with a fixed private corpus, compute is the only scalable axis. He critiques naive fine-tuning (loss 0.00001 on 10K financial reports then collapse), KV compaction, on-policy distillation, and synthetic continued pretraining, noting each hits a synthetic data wall. The goal is self-improvement like AlphaGo, where better models generate harder training questions, enabling indefinite compute scaling on your context.

AI on Your Lakehouse: Context Comes in Shapes, Not Queries — Zach Blumenfeld, Neo4j

Jul 23, 2026 · 1:59:10

Zach Blumenfeld of Neo4j argues that AI agents need context in shapes rather than queries, building three reusable graph shapes on lakehouse data to solve agent hallucinations and missed connections. The shapes include a connection semantic layer on top of BigQuery (or Databricks/Snowflake) that helps agents navigate join paths across hundreds of tables, a deterministic table-of-contents tree that lets agents traverse document folders and links without vector search, and a Leiden community-detection theme shape that surfaces unknown patterns and documentation gaps. Blumenfeld demonstrates with an auto-repair chain scenario, showing how these shapes enable an agent to answer specific repair questions and estate-level questions like what documentation is missing or what failure patterns exist, by treating context as navigable structure rather than a single query.

From Systems of Record to Systems of Context — Omri Bruchim & Tomer Ast, monday.com

Jul 22, 2026 · 15:58

Omri Bruchim and Tomer from monday.com argue that AI assistants fail to understand users because the bottleneck is understanding, not retrieval—so they are building a 'Monday world model' that precomputes context before the user asks. The system uses two engines: a slow engine that mines weeks of activity into a durable user profile (knows you), and a fast engine that reads recent signals for urgent items (knows your day). This split mirrors neuroscience’s hippocampus-neocortex and data architecture’s lambda architecture. The context is served to their Sidekick assistant, which degrades gracefully by falling back to verified context and compounds as every new day sharpens the profile. The result: Sidekick can answer 'what should I focus on right now' with understanding, not just a list of disconnected bullets.

Enterprise Agents Have a Structure Problem - Ishita Daga, Tesla

Jul 20, 2026 · 12:08

Ishita Daga, a senior machine learning engineer at Tesla, argues that enterprise agents fail because of three structural problems — ambiguity, staleness, and preference — rather than needing bigger models or more RAG. For ambiguity, she proposes a hierarchy of sources of truth: a curated semantic layer (best for known KPIs), canonical tables (parametric queries for flexibility), and a database graph (full schema but hard to maintain). To solve staleness, she recommends a context lifecycle embedding live data sources (GitHub, CRM, semantic layers) and a feedback loop that logs events, evaluates agent performance, and updates context automatically. On preference, she notes that different teams calculate the same metric differently (e.g., average milestone time by start vs. completion) and that current solutions like semantic layers or agent memory still fail to capture user-level routing, calling this an open problem requiring further research.

RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI

Jul 12, 2026 · 17:27

Shashi from Superagentic AI explains how Recursive Language Models (RLM), a pattern from MIT, solve the context window problem in large codebases by externalizing context management into a programmable REPL where the model writes code to inspect, slice, and compute relevant chunks, and recursively delegates sub-questions via llm_query. The RLM loop loads the repo as data, produces bounded observations, and can recursively call another model for deeper insights. Shashi demonstrates RLM Code, an open-source independent implementation running locally and on Gemini with a Docker sandbox, showing two tool calls and a complete trajectory. He notes that similar RLM concepts are used in proprietary systems like Codex, Claude managed agents, and dynamic workflows, making it a practical pattern for AI engineers dealing with monorepos.

Semantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, Phaidra

Jul 12, 2026 · 16:25

Raahul Singh and Vanč Levstik of Phaidra argue that LLMs fail at scale with physical infrastructure data, a problem they call 'Semantic Blindness.' They show how deploying LLMs on 500,000 real sensors led to three failure modes: the topology trap (embeddings ignore physical causality), the illusion of scale (attention degrades with context), and the repetition kill switch (identical naming triggers model's repetition penalty). Their solution replaces raw context with a hierarchical tree, letting an LLM plan a search pattern and a deterministic resolver execute set operations via pre-indexed subtrees. This achieved 100% accuracy across 460,000 GPUs versus 30% with old approaches, cut tokens from 116 million to 390,000 per eval, and kept query cost flat at 9,000 tokens. They advocate starting with pure LLM for demos then migrating deterministic logic into code, inverting the traditional software evolution path.

How we taught agents to use good retrieval - Hanna Lichtenberg, Mixedbread AI

Jul 7, 2026 · 14:28

Mixedbread AI co-founder Amir and AI engineer Hanna Lichtenberg argue the main bottleneck for knowledge agents is not reasoning but retrieval, coining the 'Oracle Gap'—the performance drop when agents use default tools. On BrowsComPlus, Oracle accuracy is 93% but Codex drops 9 points; with Mixedbread's search they recover to within 3 points. They show models write keyword gibberish (e.g., 'Senator, women, questions, billionaires, not a company') because they were trained on code tools like grep. To fix this, they built a search agent with 4 tools (overview, semantic, filter, grep) using a harness that encourages natural query sentences. Training involved supervised fine-tuning from a larger teacher and reinforcement learning with a reward combining retrieval metrics and trajectory quality (e.g., judging if queries are natural sentences). Their agent achieved NDCG@10 of 0.4 on Oblique Congress (vs. 0.18 for GPT multihop) and topped Snowflake's MedQA benchmark with 93.4% accuracy using Gemini 3.5 Flash.

Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked

May 3, 2026 · 1:41:25

Peter Werry of Unblocked argues that context engines—systems that supply AI agents with only the relevant organizational context—are critical to avoid agent doom loops and wasted tokens. He debunks three myths: naive RAG, connecting MCP servers, and bigger context windows do not solve the context problem. Werry describes building a social engineering graph to identify experts and distill team best practices, and shares hard lessons including hiding conflicts and caching answers. In a benchmark task, Unblocked's context engine reduced a 2.5-hour, 21-million-token task to 25 minutes and 10 million tokens. The talk offers a practitioner's guide to building context engines with conflict resolution, personalization, and access control.

Ship Production Software in Minutes, Not Months — Eno Reyes, Factory

Jul 25, 2025 · 16:06

Eno Reyes, cofounder and CTO of Factory, argues that AI agents can orchestrate the entire software development lifecycle, moving beyond vibe coding to agent-native development where enterprises delegate planning, coding, testing, and incident response to autonomous droids. He explains that AI tools are only as good as the context they receive—missing context from meetings, whiteboards, or Slack is the primary cause of failure, not LLM quality. Factory's droids search codebases, leverage organizational memory, and question unclear tasks before executing, from generating PRDs and tickets to creating runbooks and RCAs from sentry alerts. Reyes demonstrates how an agent can convert user transcripts and ad-hoc notes into a full feature plan, then break it into parallel tickets for multiple code droids. For incident response, droids pull logs, historical runbooks, and team discussions to produce mitigation plans in minutes, cutting response times in half and shifting from reactive to predictive operations. He emphasizes that the future belongs to engineers who manage agents—thinking clearly and communicating effectively—rather than those writing every line of code.

Transforming search and discovery using LLMs — Tejaswi & Vinesh, Instacart

Jul 16, 2025 · 21:10

Vinesh Gudla and Tejaswi Tenneti from Instacart detail how they use LLMs to overhaul search and discovery. They address challenges with conventional search: broad queries suffer cold start, tail queries lack engagement data. Using LLMs with Instacart's domain knowledge—e.g., top-converting categories as context—they improved precision by 18 percentage points and recall by 70% for tail queries, and cut zero-result queries. For discovery, LLMs generate complementary/substitute items; but pure LLM suggestions like 'chicken' for 'protein' missed user intent, so they augmented prompts with behavioral data (top categories, subsequent queries) to boost engagement and revenue. They use a hybrid approach: pre-compute offline for head/torso, distilled Llama 8B for long-tail, and LLMs as judges for evaluation. Key takeaways: combining LLM world knowledge with domain-specific data is critical, and evaluation is as hard as generation.

Your LLM Ran Out of Knowledge — Now What?

Feb 22, 2025 · 12:51

Speaker 1 presents a technique for applying LLMs to low-knowledge domains like corporate negotiations and geopolitics, where structured training data is scarce. The method uses a parsing engine to identify the problem type and match domain-specific heuristics—binary rules like 'must prioritize agreements with highest combined value' for negotiations or 'must have three independent paths for critical resources' for geopolitics. After reformatting the input for consistency, the system sends it to a reasoning model (e.g., Anthropic's Claude with a world sim) that applies the rule set alongside powerful reasoning capabilities. The demo shows an intelligence estimate scenario where the model identifies geopolitics, reformats the query, and evaluates options against rules—eliminating those that breach heuristics (e.g., scenario five violating a rule on control thresholds). The approach enables generating dozens of scenarios in a fraction of the time a human expert would take, allowing practitioners to explore more options and find novel solutions while keeping a human in the loop for subject-matter oversight.

How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou

Jul 31, 2024 · 18:42

Kevin Hou, AI engineer at Codeium, argues that embedding-based retrieval is hitting a ceiling for AI code generation and that Codeium breaks through by throwing large language models directly at the problem via a system called MQuery. He outlines the limitations of long context, fine-tuning, and embeddings, noting that embeddings are cheap but fail to reason over multiple documents needed for real-world coding tasks. Codeium developed a product-driven benchmark using recall 50 and commit-message datasets to measure retrieval quality. Because Codeium is vertically integrated—training custom models and building its own infrastructure—it can afford to run thousands of LLM calls in parallel per query, giving users 100x more compute than competitors. MQuery retrieves context from thousands of files in seconds, resulting in more accepted completions and better chat satisfaction. Hou compares this shift to autonomous driving: early heuristics like embeddings are giving way to large models that handle more data directly.