Scaling Compute on Context — Jack Morris, Engram
Aug 12, 2026 · 19:42
Jack Morris of Engram frames scaling compute on context as the pursuit of depth in AI, contrasting it with the breadth of public-data pre-training. He argues models trained on public data know nothing about your emails, meetings, or company, and that with a fixed private corpus, compute is the only scalable axis. He critiques naive fine-tuning (loss 0.00001 on 10K financial reports then collapse), KV compaction, on-policy distillation, and synthetic continued pretraining, noting each hits a synthetic data wall. The goal is self-improvement like AlphaGo, where better models generate harder training questions, enabling indefinite compute scaling on your context.
