T4

Compression at the Edge — NVIDIA, Unsloth, HuggingFace, Ollama

Aug 7, 2026 · 46:01

NVIDIA's Chris Alexiuk, Unsloth's Daniel Han, NVIDIA's Asma Beevi, Hugging Face's Merve Noyan, and Ollama's Parth Sareen argue compression democratizes AI: GLM 5.2 shrinks from 1.5 terabytes to 250 GB, 86% smaller without being 86% dumber. Han says layers are unequal—first/last critical, middle near-useless, and one 'super weight' can make a model 20% dumber—so layer choice is a combinatorial search. Asma details NVFP4, 4-bit floats sharing an FP8 scale per 16 values, targeting under 1% accuracy loss and working out of the box above ~20B parameters. Benchmarks only verify tasks—Han uses KL divergence of BF16 vs quantized logits, Ollama tests quants in real harnesses—and linear-attention models can break heuristics before KV-cache compression pushes models to phones.

Text-to-Speech Data Preparation and Fine-tuning Workshop - Ronan McGovern

Jun 3, 2025 · 34:00

Ronan McGovern walks through fine-tuning Sesame's CSM-1B text-to-speech model on a specific voice, using a YouTube video as the data source. He explains token-based TTS models, including how audio is represented via codebooks and how CSM-1B uses a main transformer for zeroth tokens and a secondary transformer for 31 hierarchical tokens. The workshop covers data preparation: downloading audio with yt-dlp, transcribing with Whisper Turbo, manually correcting the transcript, and splitting audio into 30-second chunks (41 clips from a 30-minute video). Fine-tuning uses Unsloth with LoRA adapters (rank 32, alpha 16) on linear layers, training for one epoch with a batch size of 2 and virtual size of 8, reducing loss from ~6.34 to ~3.72. Evaluation compares zero-shot inference (random speaker), voice cloning (closer but imperfect), and fine-tuned plus cloning (best result, producing an Irish-accented voice with natural errors). McGovern recommends 50+ 30-second clips for noticeable effect and notes that combining fine-tuning with voice cloning yields good performance even with limited data.

Keynote: The AI developer experience doesn't have to suck – why and how we built Modal

Feb 22, 2025 · 21:38

Eric Bernhardson, CEO of Modal, explains why and how his company replaced Kubernetes and Docker with a custom container system to deliver sub-second cold starts for AI developers. Modal turns any Python function into a serverless function with a decorator, runs on thousands of H100s, and fans out to 10,000 parallel calls. To achieve fast startup, Modal built content-addressable storage for deduplication, lazy file loading with prefetching, and uses gVisor for CPU memory snapshotting, cutting Stable Diffusion startup to seconds. The company built its own scheduler and file system, and uses mixed integer programming to manage a global GPU pool across cloud vendors. Customers like Suno use Modal for AI-generated music inference. Modal offers $30/month free credits.