Context Engineering

Context Engineering in 2026 — Louis-François Bouchard, Omar Solano & Samridhi Vaid, Towards AI

Aug 17, 2026 · 1:03:26

Louis-François Bouchard, Omar Solano, and Samridhi Vaid of Towards AI test context engineering on their AI tutor: keeping full history beat every compaction technique on recall, cost, and latency because 97% of tokens were served from cache (up to 50x cheaper), making summarization a trap unless it shrinks context by more than 50x. Full history recovered specific details 95% vs 32% after summarizing, and distinctive facts survived 800k tokens. Local hardware changes it: a 32k window can't keep everything, and larger models don't widen context. Dense retrieval hit 0% recall at 400k tokens where BM25 got 100%, so they use hybrid retrieval. Rule: name the constraint before compacting.

The Rise of CaaS: Context-as-a-Service for Agentic AI — Omer Primor, Bright Data

Aug 14, 2026 · 22:20

Omer Primor of Bright Data argues that rented AI search and context-as-a-service (CaaS) lose to a self-built pipeline once query volume passes a tipping point. The web is context, not data, and it decays: social content goes stale within a day, news/finance/retail within 30 days, so context is never a one-time snapshot. His test enriching 100 sponsor companies across 25 fields found general search beat dedicated CaaS vendors on coverage, because CaaS only answers from data it already holds; costs were similar, but frequency is the real killer — every repeated query costs the same when nothing changed. He built scrapers for LinkedIn, jobs, and Crunchbase in a day, priced setup at $5,000, and put the crossover just over 15,000 entities. Owned context compounds while rented decays.

Scaling Compute on Context — Jack Morris, Engram

Aug 12, 2026 · 19:42

Jack Morris of Engram frames scaling compute on context as the pursuit of depth in AI, contrasting it with the breadth of public-data pre-training. He argues models trained on public data know nothing about your emails, meetings, or company, and that with a fixed private corpus, compute is the only scalable axis. He critiques naive fine-tuning (loss 0.00001 on 10K financial reports then collapse), KV compaction, on-policy distillation, and synthetic continued pretraining, noting each hits a synthetic data wall. The goal is self-improvement like AlphaGo, where better models generate harder training questions, enabling indefinite compute scaling on your context.

AI on Your Lakehouse: Context Comes in Shapes, Not Queries — Zach Blumenfeld, Neo4j

Jul 23, 2026 · 1:59:10

Zach Blumenfeld of Neo4j argues that AI agents need context in shapes rather than queries, building three reusable graph shapes on lakehouse data to solve agent hallucinations and missed connections. The shapes include a connection semantic layer on top of BigQuery (or Databricks/Snowflake) that helps agents navigate join paths across hundreds of tables, a deterministic table-of-contents tree that lets agents traverse document folders and links without vector search, and a Leiden community-detection theme shape that surfaces unknown patterns and documentation gaps. Blumenfeld demonstrates with an auto-repair chain scenario, showing how these shapes enable an agent to answer specific repair questions and estate-level questions like what documentation is missing or what failure patterns exist, by treating context as navigable structure rather than a single query.

From Systems of Record to Systems of Context — Omri Bruchim & Tomer Ast, monday.com

Jul 22, 2026 · 15:58

Omri Bruchim and Tomer from monday.com argue that AI assistants fail to understand users because the bottleneck is understanding, not retrieval—so they are building a 'Monday world model' that precomputes context before the user asks. The system uses two engines: a slow engine that mines weeks of activity into a durable user profile (knows you), and a fast engine that reads recent signals for urgent items (knows your day). This split mirrors neuroscience’s hippocampus-neocortex and data architecture’s lambda architecture. The context is served to their Sidekick assistant, which degrades gracefully by falling back to verified context and compounds as every new day sharpens the profile. The result: Sidekick can answer 'what should I focus on right now' with understanding, not just a list of disconnected bullets.

Enterprise Agents Have a Structure Problem - Ishita Daga, Tesla

Jul 20, 2026 · 12:08

Ishita Daga, a senior machine learning engineer at Tesla, argues that enterprise agents fail because of three structural problems — ambiguity, staleness, and preference — rather than needing bigger models or more RAG. For ambiguity, she proposes a hierarchy of sources of truth: a curated semantic layer (best for known KPIs), canonical tables (parametric queries for flexibility), and a database graph (full schema but hard to maintain). To solve staleness, she recommends a context lifecycle embedding live data sources (GitHub, CRM, semantic layers) and a feedback loop that logs events, evaluates agent performance, and updates context automatically. On preference, she notes that different teams calculate the same metric differently (e.g., average milestone time by start vs. completion) and that current solutions like semantic layers or agent memory still fail to capture user-level routing, calling this an open problem requiring further research.

RLM: Recursive Language Models for Large Codebases - Shashi, Superagentic AI

Jul 12, 2026 · 17:27

Shashi from Superagentic AI explains how Recursive Language Models (RLM), a pattern from MIT, solve the context window problem in large codebases by externalizing context management into a programmable REPL where the model writes code to inspect, slice, and compute relevant chunks, and recursively delegates sub-questions via llm_query. The RLM loop loads the repo as data, produces bounded observations, and can recursively call another model for deeper insights. Shashi demonstrates RLM Code, an open-source independent implementation running locally and on Gemini with a Docker sandbox, showing two tool calls and a complete trajectory. He notes that similar RLM concepts are used in proprietary systems like Codex, Claude managed agents, and dynamic workflows, making it a practical pattern for AI engineers dealing with monorepos.

Semantic Blindness: 500,000 Sensors Confused an LLM - Raahul Singh & Vanč Levstik, Phaidra

Jul 12, 2026 · 16:25

Raahul Singh and Vanč Levstik of Phaidra argue that LLMs fail at scale with physical infrastructure data, a problem they call 'Semantic Blindness.' They show how deploying LLMs on 500,000 real sensors led to three failure modes: the topology trap (embeddings ignore physical causality), the illusion of scale (attention degrades with context), and the repetition kill switch (identical naming triggers model's repetition penalty). Their solution replaces raw context with a hierarchical tree, letting an LLM plan a search pattern and a deterministic resolver execute set operations via pre-indexed subtrees. This achieved 100% accuracy across 460,000 GPUs versus 30% with old approaches, cut tokens from 116 million to 390,000 per eval, and kept query cost flat at 9,000 tokens. They advocate starting with pure LLM for demos then migrating deterministic logic into code, inverting the traditional software evolution path.

How we taught agents to use good retrieval - Hanna Lichtenberg, Mixedbread AI

Jul 7, 2026 · 14:28

Mixedbread AI co-founder Amir and AI engineer Hanna Lichtenberg argue the main bottleneck for knowledge agents is not reasoning but retrieval, coining the 'Oracle Gap'—the performance drop when agents use default tools. On BrowsComPlus, Oracle accuracy is 93% but Codex drops 9 points; with Mixedbread's search they recover to within 3 points. They show models write keyword gibberish (e.g., 'Senator, women, questions, billionaires, not a company') because they were trained on code tools like grep. To fix this, they built a search agent with 4 tools (overview, semantic, filter, grep) using a harness that encourages natural query sentences. Training involved supervised fine-tuning from a larger teacher and reinforcement learning with a reward combining retrieval metrics and trajectory quality (e.g., judging if queries are natural sentences). Their agent achieved NDCG@10 of 0.4 on Oblique Congress (vs. 0.18 for GPT multihop) and topped Snowflake's MedQA benchmark with 93.4% accuracy using Gemini 3.5 Flash.

Mergeable by default: Building the context engine to save time and tokens — Peter Werry, Unblocked

May 3, 2026 · 1:41:25

Peter Werry of Unblocked argues that context engines—systems that supply AI agents with only the relevant organizational context—are critical to avoid agent doom loops and wasted tokens. He debunks three myths: naive RAG, connecting MCP servers, and bigger context windows do not solve the context problem. Werry describes building a social engineering graph to identify experts and distill team best practices, and shares hard lessons including hiding conflicts and caching answers. In a benchmark task, Unblocked's context engine reduced a 2.5-hour, 21-million-token task to 25 minutes and 10 million tokens. The talk offers a practitioner's guide to building context engines with conflict resolution, personalization, and access control.

Ship Production Software in Minutes, Not Months — Eno Reyes, Factory

Jul 25, 2025 · 16:06

Eno Reyes, cofounder and CTO of Factory, argues that AI agents can orchestrate the entire software development lifecycle, moving beyond vibe coding to agent-native development where enterprises delegate planning, coding, testing, and incident response to autonomous droids. He explains that AI tools are only as good as the context they receive—missing context from meetings, whiteboards, or Slack is the primary cause of failure, not LLM quality. Factory's droids search codebases, leverage organizational memory, and question unclear tasks before executing, from generating PRDs and tickets to creating runbooks and RCAs from sentry alerts. Reyes demonstrates how an agent can convert user transcripts and ad-hoc notes into a full feature plan, then break it into parallel tickets for multiple code droids. For incident response, droids pull logs, historical runbooks, and team discussions to produce mitigation plans in minutes, cutting response times in half and shifting from reactive to predictive operations. He emphasizes that the future belongs to engineers who manage agents—thinking clearly and communicating effectively—rather than those writing every line of code.

Transforming search and discovery using LLMs — Tejaswi & Vinesh, Instacart

Jul 16, 2025 · 21:10

Vinesh Gudla and Tejaswi Tenneti from Instacart detail how they use LLMs to overhaul search and discovery. They address challenges with conventional search: broad queries suffer cold start, tail queries lack engagement data. Using LLMs with Instacart's domain knowledge—e.g., top-converting categories as context—they improved precision by 18 percentage points and recall by 70% for tail queries, and cut zero-result queries. For discovery, LLMs generate complementary/substitute items; but pure LLM suggestions like 'chicken' for 'protein' missed user intent, so they augmented prompts with behavioral data (top categories, subsequent queries) to boost engagement and revenue. They use a hybrid approach: pre-compute offline for head/torso, distilled Llama 8B for long-tail, and LLMs as judges for evaluation. Key takeaways: combining LLM world knowledge with domain-specific data is critical, and evaluation is as hard as generation.

Your LLM Ran Out of Knowledge — Now What?

Feb 22, 2025 · 12:51

Speaker 1 presents a technique for applying LLMs to low-knowledge domains like corporate negotiations and geopolitics, where structured training data is scarce. The method uses a parsing engine to identify the problem type and match domain-specific heuristics—binary rules like 'must prioritize agreements with highest combined value' for negotiations or 'must have three independent paths for critical resources' for geopolitics. After reformatting the input for consistency, the system sends it to a reasoning model (e.g., Anthropic's Claude with a world sim) that applies the rule set alongside powerful reasoning capabilities. The demo shows an intelligence estimate scenario where the model identifies geopolitics, reformats the query, and evaluates options against rules—eliminating those that breach heuristics (e.g., scenario five violating a rule on control thresholds). The approach enables generating dozens of scenarios in a fraction of the time a human expert would take, allowing practitioners to explore more options and find novel solutions while keeping a human in the loop for subject-matter oversight.

How Codeium Breaks Through the Ceiling for Retrieval: Kevin Hou

Jul 31, 2024 · 18:42

Kevin Hou, AI engineer at Codeium, argues that embedding-based retrieval is hitting a ceiling for AI code generation and that Codeium breaks through by throwing large language models directly at the problem via a system called MQuery. He outlines the limitations of long context, fine-tuning, and embeddings, noting that embeddings are cheap but fail to reason over multiple documents needed for real-world coding tasks. Codeium developed a product-driven benchmark using recall 50 and commit-message datasets to measure retrieval quality. Because Codeium is vertically integrated—training custom models and building its own infrastructure—it can afford to run thousands of LLM calls in parallel per query, giving users 100x more compute than competitors. MQuery retrieves context from thousands of files in seconds, resulting in more accepted completions and better chat satisfaction. Hou compares this shift to autonomous driving: early heuristics like embeddings are giving way to large models that handle more data directly.