Apache Lucene

Serving 2 Million Models Without Melting: Scaling the Hugging Face Hub — Arek Borucki, Hugging Face

Jul 28, 2026 · 21:39

Arek Borucki, ML platform and database engineer at Hugging Face, explains how the Hugging Face Hub serves 14 million users and hosts 3 million public models and 1 million datasets while keeping search instant. The Hub uses MongoDB Atlas with Apache Lucene for full-text search, storing metadata separately from model artifacts in S3. Precomputed tokens and denormalized read collections optimize queries, while a seven-node MongoDB cluster distributes reads and reserves a hidden analytics node for heavy queries. Kubernetes autoscaling scales pods from 10 to 500 based on traffic, with CastAI adding nodes when capacity is exhausted, and they are migrating from HPA to KEDA for event-driven scaling on real application metrics. As the catalog grows, sharding will horizontally scale the database across multiple shards, each with its own replication.

How agent o11y differs from traditional o11y — Phil Hetzel, Braintrust

May 28, 2026 · 20:43

Phil Hetzel of Braintrust argues that agent observability differs fundamentally from traditional observability, which only answers 'is the system up?' Agents are non-deterministic, with individual traces exceeding a gigabyte and spans reaching 20 megabytes, packed with unstructured text that traditional tools like Datadog or Grafana cannot handle. Braintrust built a custom database from scratch—featuring a write-ahead log for instant visibility, analytical indexes for filtering, and a forked Rust-based Tantivy index for full-text search—to ingest and query this data in real time. Unlike traditional observability, agent observability involves non-technical experts (e.g., clinicians, lawyers) who grade traces and write justifications that become training signals for automated scoring functions. Braintrust also runs lightweight LLMs on traces to perform clustering and topic modeling, surfacing user intent, sentiment, and failure modes to accelerate the iteration loop between production issues and fixes. The episode also covers Braintrust's shift from ClickHouse to a custom database due to ClickHouse's limitations in text indexing.

Information Retrieval from the Ground Up - Philipp Krenn, Elastic

Jul 27, 2025 · 1:48:07

Philipp Krenn of Elastic demonstrates that vector search is only a feature of information retrieval, not its foundation, in this hands-on workshop on retrieval for RAG. He contrasts classic keyword search—using tokenization, stemming, stopword removal, and BM25 scoring with the inverted index—with sparse (Elser/Splade) and dense (OpenAI text-embedding-small) vector embeddings. Krenn explains why hybrid search, combining lexical and semantic methods via reciprocal rank fusion (RRF), outperforms any single approach, especially for brand or exact-match queries where keyword search remains superior. He also covers scoring normalization, chunking strategies, and the new `text_similarity_reranker` retriever, showing how Elasticsearch's retrievers API enables two-stage retrieval with a cross-encoder for re-ranking. The session emphasizes that retrieval quality depends on evaluating against a golden dataset or LLM-as-judge, and that 'it depends' is the correct answer for choosing between PgVector and dedicated search engines.