NVFP4

Taking Reinforcement Learning Cross Datacenter — Nan Jiang, Modal

Aug 10, 2026 · 19:50

Nan Jiang from Modal explains how reinforcement learning post-training can run across datacenters by shipping sparse weight deltas instead of full checkpoints. He argues that less than 1% of rollout-visible weights change between versions because Adam steps are tiny relative to BF16 rounding boundaries, a mechanism he calls Adam absorption. Modal's implementation, Stitch, lets rollout engines sync via patches (e.g., 500 MB instead of 500 GB) and operate as an elastic fleet across regions and providers. He cites internal runs showing 0.15% weight changes initially, settling near 0.05% for GLM 4.7 Air in FP8, and notes gradients are dense but updates small. He also explores whether sparsity holds for Muon and async RL scalability.

Compression at the Edge — NVIDIA, Unsloth, HuggingFace, Ollama

Aug 7, 2026 · 46:01

NVIDIA's Chris Alexiuk, Unsloth's Daniel Han, NVIDIA's Asma Beevi, Hugging Face's Merve Noyan, and Ollama's Parth Sareen argue compression democratizes AI: GLM 5.2 shrinks from 1.5 terabytes to 250 GB, 86% smaller without being 86% dumber. Han says layers are unequal—first/last critical, middle near-useless, and one 'super weight' can make a model 20% dumber—so layer choice is a combinatorial search. Asma details NVFP4, 4-bit floats sharing an FP8 scale per 16 values, targeting under 1% accuracy loss and working out of the box above ~20B parameters. Benchmarks only verify tasks—Han uses KL divergence of BF16 vs quantized logits, Ollama tests quants in real harnesses—and linear-attention models can break heuristics before KV-cache compression pushes models to phones.

Running LLMs locally: Practical LLM Performance on DGX Spark — Mozhgan Kabiri chimeh, NVIDIA

Apr 10, 2026 · 10:16

NVIDIA’s Mozhgan Kabiri Chimeh demonstrates that running LLMs locally on the DGX Spark workstation, powered by the GB10 Grace Blackwell superchip and 128GB unified memory, achieves practical performance for models up to 14B parameters. Using a reproducible vLLM benchmarking methodology, she shows that the 14B NVFP4 quantized model delivers 20.19 tokens per second and a time-to-first-token 3.4× faster than the unoptimized 14B base model. The DGX Spark supports the same NVIDIA AI software stack as production environments, enabling local development, fine-tuning, and privacy-sensitive workloads before scaling to the cloud. NVFP4 quantization is highlighted as critical for balancing intelligence and throughput on single-system setups.