All episodes

Scaling Enterprise-Grade RAG: Lessons from Legal Frontier - Calvin Qi (Harvey), Chang She (Lance)

Jul 29, 2025 · 16:40

Calvin Qi of Harvey and Chang She of LanceDB detail how Harvey builds enterprise-grade RAG for legal use cases, tackling massive scale (tens of millions of documents per corpus), spikey workloads, and non-negotiable privacy. They explain why domain-specific reranking is essential for ambiguous, expert queries like 'What is the applicable regime to covered bonds issued before 9 July 2022 under Directive EU 2019/2062?' and stress that evaluation-driven development—from expert reviews to automated metrics—is key to accuracy. Chang shows how LanceDB's AI-native multimodal lakehouse, built on the open-source Lance format, enables low-latency, high-throughput retrieval across billions of vectors (indexing 3–4 billion in under 3 hours) while supporting offline analytics, GPU indexing, and compute-storage separation. The talk emphasizes that creative data understanding, iteration speed via good evals, and flexible infrastructure are critical for scaling RAG in domain-specific, privacy-sensitive environments.

Building Alice’s Brain: an AI Sales Rep that Learns Like a Human - Sherwood & Satwik, 11x

Jul 29, 2025 · 22:18

Sherwood Callaway and Satwik from 11x describe building 'Alice's brain' — a knowledge base that lets their AI sales development representative learn from seller materials as a human SDR would, using a full RAG pipeline. They outsourced parsing to vendors like LlamaParse (documents/images), Firecrawl (websites), and Cloud Glue (audio/video), converting resources into Markdown. Their chunking strategy splits on Markdown headers then sentences and tokens. Storage uses Pinecone vector database for similarity search, with bundled embedding models. For retrieval, they built a deep research agent with Letta that plans and executes context retrieval for email generation. A 3D UMAP visualization lets users inspect Alice's knowledge nodes. Key lessons: RAG is complex, get to production before benchmarking, and leverage vendors for expertise. Future plans include hybrid RAG with a graph database, hallucination tracking, and cost reduction.

Layering every technique in RAG, one query at a time - David Karam, Pi Labs (fmr. Google Search)

Jul 29, 2025 · 20:22

David Karam of Pi Labs (formerly Google Search) walks through layering every technique in RAG, from in-memory retrieval to planet-scale search with 70+ corpus mix of token, embeddings, and knowledge graphs, jointly retrieved and re-ranked at 160,000 queries per second in under 200msec. He advocates a Quality Engineering Loop: baseline with simplest methods, analyze losses, then apply incremental techniques based on complexity-adjusted impact. Karam explains why queries like "falafel" are notoriously hard due to ambiguous intent, highlights failures of chunking documents, and shows when BM25 suffices, when relevance embeddings are needed, and when custom embeddings or domain-specific signals (price, popularity, user clicks) become essential. He covers query orchestration via fan-out, supplementary retrieval across back ends, and distillation for cost optimization, concluding that at sufficient complexity, problems must be punted to LLM or UX for graceful degradation.

Building a Smarter AI Agent with Neural RAG - Will Bryk, Exa.ai

Jul 29, 2025 · 18:42

Will Bryk, CEO of Exa.ai, argues that traditional keyword search engines like Google are ill-suited for AI agents, which need semantic understanding, complex multi-paragraph queries, and comprehensive results. He explains Exa's neural search approach using embeddings to capture ideas and context, unlike keyword-based systems that miss nuances like negation. Bryk demonstrates a live-coded agent combining neural searches (e.g., finding personal sites of SF engineers who like information retrieval) with keyword searches (e.g., retrieving specific GitHub profiles), showing how Exa's API exposes toggles for date ranges, domains, and result counts. He emphasizes that AI agents require search engines that return exactly what they ask for, not what humans click on, and can handle thousands of results. The episode also previews Exa's new research endpoint for automated deep research.

[Full Workshop] Building Metrics that actually work — David Karam, Pi Labs (fmr Google Search)

Jul 29, 2025 · 40:28

Achim and David Karam, former Google Search leaders now at Pi Labs, argue that building reliable AI evaluations requires a multi-signal scoring system rather than single LLM-as-judge metrics. They walk through designing custom metrics starting with simple correlated signals, using a copilot to generate and iterate on dimensions like 'includes insights' or 'title length,' then calibrating them against ground truth thumbs up/down data from users. The workshop demonstrates applying this to a meeting summarizer, integrating the scoring system into Google Sheets to score 120 examples and produce a confusion matrix showing alignment with human feedback. They also show how to use the same scoring system for model comparison (e.g., GPT-1.5 vs 2.5) and online best-of-N sampling, which boosts response quality by generating multiple responses and selecting the highest-scored one.

Make your LLM app a Domain Expert: How to Build an Expert System — Christopher Lovejoy, Anterior

Jul 28, 2025 · 19:18

Christopher Lovejoy, a medical doctor turned AI engineer at Anterior, argues that building a domain-native LLM application requires a system for incorporating domain insights rather than relying solely on model sophistication. Anterior's Adaptive Domain Intelligence Engine uses domain expert clinicians to review AI outputs, generate metrics, define failure modes, and suggest improvements. This process improved Anterior's clinical reasoning tool from 95% to 99% accuracy on medical necessity reviews for health insurance providers covering 50 million lives. By enabling same-day iteration—production cases reviewed, failures categorized, domain knowledge added—the system solves the 'last mile problem' of giving models nuanced understanding of customer workflows. Lovejoy emphasizes that the limitation is not model reasoning but encoding domain-specific context, and the winning vertical AI team will build the best system for this translation.

Shipping Products When You Don't Know What they Can Do — Ben Stein, Teammates

Jul 28, 2025 · 19:34

Ben Stein, founder of Teammates, argues that product management for AI agents requires abandoning traditional specs and embracing uncertainty. He describes how his team discovered emergent behavior when a customer asked to tag an agent in a Google Doc comment—an unplanned feature that worked. Stein advocates thinking in affordances rather than features, using evals as the new spec, and vibe coding to feel interactions before committing. He notes that bugs become ambiguous when behavior is probabilistic, and customer meetings shift from selling vision to co-inventing the future. The episode offers a field guide for product teams navigating boundless surface areas built on unpredictable LLMs.

Shipping something to someone always wins — Kenneth Auchenberg (ex. Stripe, VSCode)

Jul 28, 2025 · 16:17

Kenneth Auchenberg, former VSCode and Stripe developer platform lead, argues that great AI products come from rapid iterative loops — shipping continuously viable 'skateboards' to real users rather than aiming for a perfect final product. He advocates for writing launch blog posts before building, working intimately with a few real customers (even texting them), and ignoring constraints like legal initially to design the best product. APIs are harder to change than UI, so early user feedback is critical. In AI, the fundamentals haven't changed: customer knowledge and iteration velocity matter more than ever, even as AI tools accelerate building. The goal should be to run a product feedback loop in under a day.

Why your product needs an AI product manager, and why it should be you — James Lowe, i.AI

Jul 28, 2025 · 18:37

James Lowe, Head of AI Engineering at the UK Government's Incubator for AI, argues that technical expertise makes AI engineers ideal candidates for the critical role of AI Product Manager, which balances business viability, technological feasibility, user desirability, and the core question of possibility. He shares three hard-won lessons from building products like CONSULT, Minute, and Redbox: resolve AI uncertainties early with evaluations and real-user tests (achieving 1,000x faster and 400x cheaper consultation analysis), experiment widely with features then cut back (streamlining a transcription tool from overwhelming options to a focused Justice Transcribe used for Prime Ministerial meetings), and pivot harder than ever as the landscape shifts (Redbox evolved from digitizing ministerial red boxes to a secure LLM chat client, then to an MCP-based tool provider when Microsoft Copilot Chat went free). The episode emphasizes that AI product management is a mindset, not just a title, and that engineers should step into this leadership gap to build impactful AI products.

Everything is ugly, so go build something that isn't — Raiza Martin, Huxe (ex NotebookLM)

Jul 28, 2025 · 25:15

Raiza Martin, former lead of Google's NotebookLM and founder of Huxe, argues that the current chaotic phase of AI product design is a once-in-a-career opportunity to rebuild from first principles, calling everything we use 'the ugliest that it will ever be.' Drawing from her experience forcing NotebookLM into existence against skepticism, she emphasizes that personal clarity of vision fuels product building, and purpose must be relentlessly focused on a single outcome—for NotebookLM, enabling users to upload 50 files and interact with them. She stresses earning trust by nailing deterministic behavior first, noting that 90% of first queries were summarization and failures drove users away forever, then layering on delightful probabilistic features like podcast generation. Finally, she warns against the 'kitchen sink' approach, citing her own Huxe app that did everything but users only used one feature, advocating restraint as an innovation multiplier and focus on one excellent outcome to avoid building ugly products.

Building the platform for agent coordination — Tom Moor, Linear

Jul 28, 2025 · 19:43

Tom Moor, Head of Engineering at Linear, explains how the company is evolving from an issue tracker into an operating system for engineering teams that treats AI agents as first-class teammates. He details Linear's pragmatic AI journey: starting with embeddings and PG vector, then moving to a hybrid search index using TurboPuffer and Cohere embeddings, leading to features like Product Intelligence (query rewriting, reranking, deterministic rules for suggestions), natural language filters, Slack-to-issue creation, and daily audio pulses. The core of the talk is Linear's agent platform, launched two weeks ago, where coding agents like CodeGen, Bucket, and Charlie integrate via OAuth, GraphQL, and new webhooks, allowing users to assign, mention, and interact with agents just like human teammates. Moor emphasizes best practices for builders: respond fast, inhabit the platform's language, move issues to 'in progress,' and clarify plans before acting, all while keeping interactions concise and value-adding. The episode argues that with this platform, engineering teams can build more, faster, and with higher quality by offloading grunt work to infinitely scalable cloud-based teammates.

What Is a Humanoid Foundation Model? An Introduction to GR00T N1 - Annika & Aastha

Jul 28, 2025 · 17:47

NVIDIA's Annika Brundyn and Aastha Jhunjhunwala introduce GR00T N1, an open-source Vision-Language-Action foundation model for humanoid robots. They argue that physical AI is key to addressing labor shortages in industries like healthcare and manufacturing, and that humanoid forms are needed because the world is built for humans. The model uses a data pyramid strategy combining limited real-world teleoperation data, synthetic simulation data, and internet video, with DreamGen for data multiplication. Its dual-system architecture, inspired by Kahneman's 'Thinking, Fast and Slow', has a System 2 planner for high-level reasoning and a System 1 for fast motor control at 120 Hz. The model is trained end-to-end via imitation learning and reinforcement learning, and features a generalist action decoder that enables cross-embodiment fine-tuning.

Real-time Experiments with an AI Co-Scientist - Stefania Druga, fmr. Google Deepmind

Jul 28, 2025 · 18:42

Stefania Druga, formerly at Google DeepMind, demonstrates a real-time AI co-scientist system built with under $300 in open-source hardware and two weeks of effort, capable of live scientific experimentation. Using a micro:bit with Jack DAC temperature sensor, ReCamera for object tracking, and a microscope, the system streams sensor data, images, and voice to Gemini via a React app that dynamically assembles multimodal context. Druga presents two home experiments: crystal growth, where she discovered that formation occurs in bursts once critical saturation is reached, and fermentation, monitoring CO2 and temperature to optimize dough rise. She references DeepMind's AI co-scientist paper, which replicated a 12-year gene transfer discovery in two days and identified novel drug targets for liver fibrosis that proved effective in wet labs. Her vision moves from async data analysis to real-time empirical hypothesis generation, and she highlights open-source platforms like Jubilee and open bioreactors for automating lab equipment, with future plans to use real-time data to inform realistic simulations.

Scaling AI Agents Without Breaking Reliability — Preeti Somal, Temporal

Jul 28, 2025 · 15:01

Preeti Somal, SVP of Engineering at Temporal, argues that the platform is the right choice for building reliable, scalable agentic AI applications. Temporal provides battle-tested orchestration through workflow abstractions, handling state, retries, and failure transparency so developers can focus on business logic. She cites customers like Dust and Gorgias running AI agents on Temporal in production, achieving up to 6x faster feature delivery. The platform uses signals and queries for human interaction, stores full workflow history for debugging and compliance, and offers Temporal Cloud for managed scalability. Somal emphasizes that agents are complex distributed systems requiring robust orchestration, and Temporal’s decade of production use in mission-critical applications proves its reliability.

Ship Agents that Ship: A Hands-On Workshop - Kyle Penfound, Jeremy Adams, Dagger

Jul 27, 2025 · 1:21:00

Kyle Penfound and Jeremy Adams from Dagger demonstrated building production-minded AI agents using Dagger, a programmable delivery engine that runs containerized workflows identically on laptops and in CI. They created a workspace submodule with tools — read file, write file, list files, and run tests — and wired it to an LLM (Claude 3.5) so the agent can edit code and validate changes through the project's actual test suite. The agent runs in isolated containers for determinism and traceability, with behavior visualized via Dagger Cloud. They extended the agent to respond to GitHub issues: a label triggers GitHub Actions to run Dagger, read the issue, hand the assignment to the agent, and open a pull request. The talk emphasized that Dagger's modularity lets developers give agents precisely scoped tools while reusing existing CI workflows, providing guardrails without sacrificing flexibility.

The AI Engineer’s Guide to Raising VC — Dani Grant (Jam), Chelcie Taylor (Notable)

Jul 27, 2025 · 34:17

Dani Grant (CEO of Jam) and Chelcie Taylor (Notable Capital) give AI engineers a tactical playbook for raising venture capital, arguing that you don't need revenue, a product, or even to leave your full-time job to raise a pre-seed or seed round. They explain that VCs bet on founders' vision and unique insights, not perfect technology or traction, and share real cold emails that led to investments—including one that simply referenced a blog post and another that didn't mention the startup at all. In pitch meetings, they advise focusing on 'why you, why now' over tech deep dives, making the conversation bidirectional, and ending by asking about next steps. Common mistakes include over-engineering the demo, failing to acknowledge competitors, and not having a prepared pitch deck to send afterward. Grant's original Jam pitch deck is still live at deck.jam.dev. Taylor's contact is ctaylor@notablecap.com.

Strategies for LLM Evals (GuideLLM, lm-eval-harness, OpenAI Evals Workshop) — Taylor Jordan Smith

Jul 27, 2025 · 32:28

Taylor Jordan Smith from Red Hat presents a hands-on workshop on evaluating large language models (LLMs) for production, using three open-source tools: GuideLLM for system performance benchmarks (latency, throughput), lm-eval-harness for factual accuracy via MMLU Pro, and promptfoo for safety and bias custom evaluations. He argues that traditional benchmarks are insufficient and advocates for a layered evaluation pyramid—starting with system performance, then factual accuracy, then safety/bias—taking an incremental approach similar to software testing (unit, integration, end-to-end). He demonstrates deploying an IBM Granite 2B model with vLLM on an L4 GPU, showing how to adjust input/output tokens for use cases like chatbots or RAG. The episode emphasizes that evaluations must be tailored to the specific system (RAG, agents, etc.) and continuously integrated into CI/CD pipelines to manage risk, cost, and reliability in production. Attendees gain actionable strategies for custom eval suites and human-in-the-loop feedback.

Why you should care about AI interpretability - Mark Bissell, Goodfire AI

Jul 27, 2025 · 21:11

Mark Bissell of Goodfire AI argues that mechanistic interpretability—reverse engineering neural networks—has moved from research to practical use cases that AI engineers can apply today, demonstrated through Goodfire's Ember platform for neural programming. He shows how Ember enables debugging and steering models at the neuron level, such as turning up a 'sensitive information' feature to make LLaMA refuse to reveal an email, or dynamically injecting a Coca-Cola recommendation when a beverage feature activates. For image models, the Paint with Ember demo (paint.goodfire.ai) lets users paint concepts like pyramid or lion face directly onto a canvas and steer sub-features (e.g., lion minus mane becomes tiger). Beyond interfaces, interpretability powers model diffs to detect unwanted changes after fine-tuning, guardrails for production systems, and scientific extraction: Goodfire works with the ARC Institute to uncover biological principles from the superhuman genomics model Evo2. Efficiency gains also emerge by pruning unnecessary weights for specialized tasks, making interpretability a critical tool for reliable AI engineering.

Information Retrieval from the Ground Up - Philipp Krenn, Elastic

Jul 27, 2025 · 1:48:07

Philipp Krenn of Elastic demonstrates that vector search is only a feature of information retrieval, not its foundation, in this hands-on workshop on retrieval for RAG. He contrasts classic keyword search—using tokenization, stemming, stopword removal, and BM25 scoring with the inverted index—with sparse (Elser/Splade) and dense (OpenAI text-embedding-small) vector embeddings. Krenn explains why hybrid search, combining lexical and semantic methods via reciprocal rank fusion (RRF), outperforms any single approach, especially for brand or exact-match queries where keyword search remains superior. He also covers scoring normalization, chunking strategies, and the new `text_similarity_reranker` retriever, showing how Elasticsearch's retrievers API enables two-stage retrieval with a cross-encoder for re-ranking. The session emphasizes that retrieval quality depends on evaluating against a golden dataset or LLM-as-judge, and that 'it depends' is the correct answer for choosing between PgVector and dedicated search engines.

Introduction to LLM serving with SGLang - Philip Kiely and Yineng Zhang, Baseten

Jul 26, 2025 · 43:42

Philip Kiely and Yineng Zhang introduce SGLang, an open-source fast serving framework for LLMs and VLMs, arguing it offers production-ready performance with day-zero support for new models like DeepSeek and Qwen, and a customizable codebase for contributions. Yineng, a core maintainer, traces SGLang's rapid growth from a December 2023 paper to nearly 15,000 GitHub stars and adoption by xAI, AMD, and Meituan. The workshop demonstrates deploying a first model via Baseten's trunk packaging, then tuning the CUDA Graph max batch size flag on an L4 GPU to maintain CUDA Graph-enabled decoding during higher concurrency, boosting generation throughput. They also cover Eagle 3 speculative decoding, where a draft model derived from the target model speculates tokens; users can benchmark different step and top-k configurations on representative prompts to find optimal settings for production. Finally, they invite contributions via GitHub's 'good first issue' tags and highlight Baseten's job openings.

Waymo's EMMA: Teaching Cars to Think - Jyh Jing Hwang, Waymo

Jul 26, 2025 · 17:28

Jyh-Jing Hwang of Waymo presents EMMA, an end-to-end multimodal model built on Gemini that directly processes camera inputs and route text into driving waypoints, achieving state-of-the-art open-loop planning on the NuScenes benchmark without LiDAR or HD maps. Adding chain-of-thought reasoning—identifying critical objects and meta-decisions—further surpasses specialized models like MotionLM on Waymo's 100K dataset. Co-training on tasks such as 3D detection and road graph estimation maintains competitive performance across domains. For evaluation, Waymo leverages generative video models (e.g., Veo2) to simulate diverse conditions like rain and night, testing EMMA's robustness. The work demonstrates how Gemini's generalization can help scale Waymo's autonomous driving to new cities by handling long-tail scenarios.

Robotics: why now? - Quan Vuong and Jost Tobias Springberg, Physical Intelligence

Jul 26, 2025 · 18:07

Physical Intelligence's Quan Vuong and Jost Tobias Springberg describe their mission to build a model that can control any robot to do any task, arguing that software intelligence is the main bottleneck in robotics. They explain Vision Language Action models (VLAs) as adaptations of vision language models that output robot actions instead of text. To train these models, they built a data engine from scratch, collecting 10,000 hours of successful episodes via teleoperation in six months. Their latest model, PAIO-5, achieves open-world generalization by training on data from multiple homes, matching or surpassing performance on held-out scenes. They demonstrate this with a policy that performs long-horizon tasks like cleaning an unseen bedroom for up to 10 minutes autonomously. They also highlight a remote coffee-making demonstration on a robot they never touched, showing model portability across hardware.

A2A & MCP Workshop: Automating Business Processes with LLMs — Damien Murphy, Bench

Jul 26, 2025 · 1:23:14

Damien Murphy presents A2A and MCP protocols for building multi-agent systems that automate business processes from webhooks, using a host agent that delegates tasks to sub-agents (Slack, GitHub, Bench) via A2A, with each sub-agent using MCP tools from Zapier or internal APIs. He demonstrates processing a meeting transcript to create a GitHub issue, send a Slack message, and research attendees, emphasizing that A2A handles remote agent discovery and opaqueness while MCP provides standardized third-party tool access. Murphy highlights benefits like context isolation—sub-agents absorb large tool outputs and keep the host's context small—and parallel processing, but notes limitations: A2A's early stage, MCP's silent failures (e.g., Zapier Slack missing channels), the non-determinism of LLM orchestration, and the challenge of prompt caching costs. He argues that A2A is best for third-party agents where complexity is hidden, while MCP is useful for extensible tool integration, but if you control the tools or agents, direct function calls are simpler and more reliable.

Piloting agents in GitHub Copilot - Christopher Harrison, Microsoft

Jul 26, 2025 · 59:07

Christopher Harrison (Microsoft) demonstrates GitHub Copilot's agent capabilities for AI Engineers, arguing context is key to effective pair programming. He explains Copilot's modes—completion, chat, edit, local agent, and coding agent—and how Model Context Protocol (MCP) enables access to external tools like databases. Harrison shows how Copilot-instructions.md files provide project-specific guidance, and how coding agent runs within ephemeral GitHub Actions environments with strict security (no external access, only write to its own branch). He emphasizes that AI doesn't change DevOps fundamentals: tests still run locally and via PR workflows, and code review remains essential. The lab covers assigning issues to Copilot, configuring MCP servers, and using .instructions files for reusable patterns, with a repeat session at 3:30 PM.

Ship Production Software in Minutes, Not Months — Eno Reyes, Factory

Jul 25, 2025 · 16:06

Eno Reyes, cofounder and CTO of Factory, argues that AI agents can orchestrate the entire software development lifecycle, moving beyond vibe coding to agent-native development where enterprises delegate planning, coding, testing, and incident response to autonomous droids. He explains that AI tools are only as good as the context they receive—missing context from meetings, whiteboards, or Slack is the primary cause of failure, not LLM quality. Factory's droids search codebases, leverage organizational memory, and question unclear tasks before executing, from generating PRDs and tickets to creating runbooks and RCAs from sentry alerts. Reyes demonstrates how an agent can convert user transcripts and ad-hoc notes into a full feature plan, then break it into parallel tickets for multiple code droids. For incident response, droids pull logs, historical runbooks, and team discussions to produce mitigation plans in minutes, cutting response times in half and shifting from reactive to predictive operations. He emphasizes that the future belongs to engineers who manage agents—thinking clearly and communicating effectively—rather than those writing every line of code.

Beyond the Prototype: Using AI to Write High-Quality Code - Josh Albrecht, Imbue

Jul 25, 2025 · 17:59

Josh Albrecht, CTO of Imbue, discusses Sculptor, an experimental coding agent environment that aims to move AI-assisted coding beyond prototypes to high-quality production software. He argues that the key is preventing errors through learning, planning, writing specs, and enforcing strict style guides, then detecting remaining issues with automated linting, generated test suites, and LLM-based code reviews. Sculptor forces agents to plan first, auto-fixes linting errors, and enables generating hundreds of happy-path and unhappy-path unit tests. Albrecht emphasizes writing functional, side-effect-free code, sandboxed execution, and focusing on integration tests and test coverage. He also notes that generated tests can be thrown away since AI can regenerate them easily, and that a well-stated problem is half solved.

Software Development Agents: What Works and What Doesn't - Robert Brennan, OpenHands

Jul 25, 2025 · 16:46

Robert Brennan, creator of the open-source coding agent OpenHands (formerly OpenDevin), argues that while coding is going away, software engineering remains about critical thinking, not typing. He explains that agents like OpenHands operate via a loop between an LLM and tools like a diff-based editor, terminal, and web browser, all sandboxed in Docker. Brennan advises starting small with rote tasks like merge conflicts (99% success) or PR feedback, being specific about frameworks and files, and always reviewing AI-generated code to avoid tech debt. He shares that 90% of his code now goes through the agent, with 10% requiring manual intervention, and warns against auto-merging without human accountability—OpenHands now assigns PRs to the human who triggered them. Key use cases include fixing failing tests, infrastructure changes, database migrations, and greenfield internal apps where vibecoding is acceptable.

Devin 2.0 and the Future of SWE - Scott Wu, Cognition

Jul 25, 2025 · 16:13

Scott Wu of Cognition AI describes an AI 'Moore's Law' for coding agents, with capability doubling every 70 days, propelling Devin from simple migrations to autonomous project work. He traces Devin's evolution through tiers: first repetitive migrations using Playbooks for instruction-following, then isolated bugs with VM snapshots, then multi-file changes with codebase intelligence, and finally backlog killing with self-testing. Each tier required solving different bottlenecks—memory, language server integration, iterative planning, and confidence calibration. Wu argues that the interface and key capabilities shift every 2-3 months, and predicts another 16-64x improvement in the next year.

Your Coding Agent Just Got Cloned And Your Brain Isn't Ready - Rustin Banks, Google Jules

Jul 25, 2025 · 13:40

Rustin Banks, an AI product manager at Google Labs, shares how Jules, Google's asynchronous coding agent powered by Gemini 2.5 Pro, enables developers to shift from sequential coding to orchestrating many parallel agents. He demonstrates real-world use on a conference schedule website, adding tests with Jest and Playwright simultaneously, implementing features like Google Calendar integration and AI summaries, and running accessibility audits. Banks argues that success requires clear verification criteria, creating 'disagreement' with agents to ensure correctness, and a robust merge/test framework. He notes that in its first two weeks, Jules generated 40,000 public commits, and emphasizes that agents-as-a-service in the cloud allow infinite scalability and development from any device.

Latent Space Paper Club: AIEWF Special Edition (Test of Time, DeepSeek R1/V3) — VIbhu Sapra

Jul 25, 2025 · 53:54

VIbhu Sapra presents DeepSeek's latest models and announces a new curriculum-based Test of Time Paper Club. DeepSeek's May 28th R1 update doubles reasoning tokens (12k→25k) and matches O3/Gemini 2.5 Pro, improving AIME 2024 from 70% to 87.5%. A new distillation of R1 traces into Qwen 3 8b achieves performance on par with Qwen's 235b thinking model, proving reasoning models distill efficiently. The Test of Time Paper Club will cover 50–100 foundational papers over six months in San Francisco and remote, targeting core AI engineering topics from attention and scaling laws to inference techniques. Sapra emphasizes that pure RL (GRPO) on verifiable math and code unlocks emergent reflection and aha moments, shifting scaling from pre-training compute to inference-time reasoning.

Human seeded Evals — Samuel Colvin, Pydantic

Jul 25, 2025 · 12:02

Samuel Colvin, creator of Pydantic, demonstrates how type safety and validation loops in Pydantic AI build reliable GenAI applications. He argues that type-safe frameworks enable confident refactoring, a necessity for evolving AI apps, and shows how returning validation errors to the model fixes simple mistakes like incorrect date formats. The agentic loop with tools and final result tools ensures structured extraction, while type-safe dependencies on tools prevent runtime errors. Colvin also highlights Logfire observability, which traces calls, shows pricing, and helps debug failures like failed memory retrieval. The talk covers agent definitions, validation error retries, and the importance of explicit exit conditions in agent loops.

Building AI Products That Actually Work — Ben Hylak (Raindrop), Sid Bendre (Oleve)

Jul 24, 2025 · 18:42

Ben Hylak (Raindrop) and Sid Bendre (Oleve) argue that building reliable AI products requires iterative real-world signals over traditional evals. Ben debunks eval myths—evals don't measure product quality, LLM-as-judge fails, and production evals are costly—and stresses tracking explicit signals (thumbs up/down, copy rate) and implicit signals (refusals, frustration) to identify issues. Sid introduces Trellis, a framework for scaling viral AI apps that uses discretization—breaking infinite output into intent buckets—prioritization by volume times negative sentiment times achievable delta, and recursive refinement. Starting with an MVP, teams classify user intents, convert them into semi-deterministic workflows, then repeatedly drill into sub-intents to engineer repeatable, attributable magic. Oleve's approach, powered by Raindrop, has scaled six viral products to $6M ARR profitably with four people.

Rise of the AI Architect — Clay Bavor, Cofounder, Sierra w/ Alessio Fanelli

Jul 24, 2025 · 18:55

Clay Bavor, cofounder of Sierra, and Alessio Fanelli discuss the rise of the AI Architect—a new role combining technology, brand, and business outcomes to build customer-facing AI agents. Sierra serves hundreds of millions of consumers this year. Bavor defines the AI Architect as wearing three hats: understanding AI capabilities, defining the agent's voice (e.g., Chubbies' irreverent Duncan Smothers), and driving business outcomes. Successful AI Architects embrace risk, start with narrow problems like processing a single return, and re-architect teams to coach the AI. On build vs. buy, Bavor warns of the "agent iceberg"—hundreds of hidden complexities like regression testing and model migration. He advises tracking model improvement in a Google Doc and anticipating future capabilities, predicting glasses as the ultimate interface for trusted personal AI.

AI That Pays: Lessons from Revenue Cycle — Nathan Wan, Ensemble Health

Jul 24, 2025 · 18:19

Nathan Wan, head of AI at Ensemble Health Partners, argues that Revenue Cycle Management (RCM) is a critical yet overlooked domain for AI disruption in healthcare, where 40% of hospitals operate at negative margins due to broken manual financial processes. He explains that RCM's vast structured and unstructured data, rule-based workflows, and direct financial impact make it ripe for AI, contrasting it with more publicized clinical AI applications. Wan details how Ensemble uses AI to predict and prevent claim denials—often caused by technical errors like missing data—and to accelerate clinical denial appeals via GenAI, achieving a 40% reduction in processing time and improved overturn rates. He emphasizes that Ensemble's end-to-end view of the revenue cycle, supported by their EIQ data platform, enables longitudinal error correction and agent-driven automation, turning upstream fixes into measurable ROI for providers facing increasing payer denial rates.

Structuring a modern AI team — Denys Linkov, Wisedocs

Jul 24, 2025 · 17:40

Denys Linkov, who leads ML at Wisedocs, argues that building a modern AI team hinges on identifying your company's bottleneck—shipping features, acquiring users, or scalability—rather than reflexively hiring AI researchers. He introduces Ampere's Wager: trading your entire domain-savvy team for five top-lab researchers is usually a losing bet. For early-stage AI strategy, generalists who blend model training, serving, and business acumen outperform specialists; Linkov lived this in 2021 building a custom MLOps platform for a conversational AI startup and again in 2024 using advanced open-source tools for medical record processing. He stresses reskilling existing teams through weekly learning cadences and moving domain experts from giving feedback to writing evaluations. Hiring should hold context and act on it, verifying trends like 'don't hire juniors' against YC's AI school drawing 2,000 young people.

The Rise of Open Models in the Enterprise — Amir Haghighat, Baseten

Jul 24, 2025 · 16:50

Amir Haghighat, CTO of Baseten, argues that enterprises are increasingly moving from closed frontier models like OpenAI and Anthropic toward open source models, driven by four specific cracks in the assumption that closed models will work indefinitely: quality for specialized tasks (e.g., medical document extraction), latency requirements (especially for voice), unit economics ballooning from agentic use cases where a single user action triggers 50 inference calls, and the desire for competitive differentiation. Drawing on conversations with over 100 enterprises, he explains that while most started with dedicated deployments on Azure/AWS for toying around in 2023, by 2024 about 40-50 had production use cases, and in 2025 the shift accelerated. However, adopting open models forces enterprises to build inference infrastructure, facing challenges like speculative decoding, prefix caching, guaranteeing four-nines reliability with hardware failures and VLM crashes, and scaling replicas—with one Fortune 50 soft drink company reporting an eight-minute spin-up time. Haghighat concludes by contrasting the simple API-call world with the complexities of mission-critical inference, where…

Mentoring the Machine — Eric Hou, Augment Code

Jul 24, 2025 · 19:06

Eric Hou, member of technical staff at Augment Code, argues that AI agents work best when treated as perpetually junior engineers requiring mentoring and context, not just prompts. He recounts a chaotic Tuesday where he parallelized three urgent tasks—a design system component, a staging emergency involving a gRPC library upgrade touching 12 services and 20,000 lines of code, and mentoring a new hire—using Augment's agent and Slack bot, completing in half a day what normally takes weeks. Hou quantifies industry inefficiencies: each interruption costs 23 minutes recovery, two-thirds of time spent maintaining code, totaling $300 billion annually. He prescribes a three-step organizational adoption: knowledge gathering (mapping Notion, Google Docs, GitHub; using meeting intelligence tools like Granola), gaining tool familiarity, and leaning in by sharing successful agent memories across teams. He demonstrates live creation of an agent personality during the talk, and concludes that solving knowledge infrastructure allows parallel exploration of prototypes, turning software development into a more scientific, data-driven process.

Building Applications with AI Agents — Michael Albada, Microsoft

Jul 24, 2025 · 15:50

Michael Albada, Principal Applied Scientist at Microsoft, explains how to build effective AI agent systems, defining agents as entities that reason, act, communicate, and adapt. He emphasizes that tool use requires exposing grouped, human-facing functions rather than one-to-one API mappings, noting a 254% increase in agentic Y Combinator startups. Orchestration should start simple with chains and trees before moving to fully agentic patterns, using deterministic logic for safety and external state management. For multi-agent systems, he recommends splitting tools into semantically similar groups to avoid overwhelming a single agent. Albada stresses investing in evaluation via open-source tools like Intel Agent, Pirate, Label Studio, and automatic optimization with Trace, TextGrad, and DSPY. He warns against common pitfalls: insufficient evals, poorly designed tools, excessive complexity, and lack of observability, while urging design for security with tripwires and fallback to human review.

AX is the only Experience that Matters - Ivan Burazin, Daytona

Jul 24, 2025 · 15:25

Ivan Burazin, co-founder of Daytona, argues that agent experience (AX) is the only experience that matters, as AI agents will soon outnumber human developers and tools must be built for agents to autonomously operate. He cites that 25% of YC startups say AI writes 95% of their code and 37% of the latest YC batch build agents as products. Burazin outlines three AX pillars—seamless authentication, agent-readable docs (like Stripe's .md and LLMs.txt), and API-first design—then introduces Daytona's agent-native runtime, which spins up sandboxes in 27 milliseconds and includes features like a declarative image builder, network-mounted volumes for large datasets, and parallel execution for agents to fork environments. He concludes that any tool requiring a human in the loop is built for the past, and if agents cannot use a product, no one will.

How to build Enterprise Aware Agents - Chau Tran, Glean

Jul 24, 2025 · 19:53

Chau Tran, a software engineer at Glean, contends that building enterprise-aware agents requires combining workflows and agents via workflow search instead of choosing one over the other. Workflows offer predictability for repetitive tasks, while agents bring open-ended flexibility for novel tasks. Their synergy lets workflows evaluate and train agents, and agents discover new workflows. For training, fine-tuning works well for generalized behaviors with stable data, while dynamic prompting with search excels at personalized tasks and rapid requirement changes—like building custom hardware versus writing software. Workflow search combines textual similarity with authoritativeness signals from knowledge graphs to pick the right workflow among many similar options.

Monetizing AI — Alvaro Morales, Orb

Jul 23, 2025 · 18:18

Alvaro Morales, CEO of Orb, argues that traditional subscription-based SaaS pricing fails for AI products due to rapid cost fluctuations, margin pressures, and customer demand for clear ROI. He presents three frameworks: decide between direct monetization (add-on, bundled) or indirect (free to drive upsells) using examples like GitHub Copilot, Notion AI, and Expedia; select a value metric from resource-based tokens (Vercel's v0) to task-based (Zapier) to outcome-based pricing (Intercom's Fin at $0.99 per resolved ticket); and embrace continual experimentation, as AI pricing cycles shrink from years to months. To reduce pricing guesswork, Orb built Simulations, a tool that lets companies backtest alternate pricing strategies on real usage data from closed betas before launch, helping teams like Replit and Perplexity optimize revenue without risking customer trust.

Does AI Actually Boost Developer Productivity? (100k Devs Study) - Yegor Denisov-Blanch, Stanford

Jul 23, 2025 · 18:12

Yegor Denisov-Blanch from Stanford presents findings from a study of nearly 100,000 developers across 600+ companies, showing AI boosts developer productivity by an average of 15-20%, but the effect varies widely. The study measures functionality delivered, not commits, and reveals that AI introduces significant rework—bug fixes from AI-generated code. Productivity gains depend on task complexity, codebase maturity (Greenfield vs. Brownfield), language popularity, and codebase size: low-complexity Greenfield tasks see 30-40% gains, while high-complexity Brownfield tasks see only 0-10%. AI can even decrease productivity for low-popularity languages like COBOL or Haskell, and context window limitations reduce performance on larger codebases. The study provides data-driven guidance for when AI helps or hinders, with a matrix showing 20% gains for low-complexity common languages and minimal gains for complex niche ones.

How agents will unlock the $500B promise of AI - Donald Hruska, Retool

Jul 23, 2025 · 16:22

Donald Hruska, engineering lead for Retool's Agents product, argues that AI agents will unlock the $500B promise of AI by moving enterprises beyond toy chatbots into production-grade systems. He explains that building a basic agent is easy (e.g., 100 lines of code using React) but getting it into production requires solving security, cost overruns, observability, and compliance. Hruska outlines four options—build from scratch, use a framework like LangGraph, a managed platform like Retool Agents, or verticalized agents—and advises building for core differentiators and buying for commodity workflows. He cites Retool customers like ClickUp saving over $200,000 in vendor costs and Descript saving hundreds of hours weekly, while noting inference costs dropped 99.7% from 2022-2024. Retool charges $3 per hour for its cheapest agent and supports on-prem deployment.

How Intuit uses LLMs to explain taxes to millions of taxpayers - Jaspreet Singh, Intuit

Jul 23, 2025 · 18:59

Jaspreet Singh, Senior Staff Engineer at Intuit, explains how TurboTax uses Anthropic's Claude and OpenAI models to generate personalized tax explanations for 44 million customers. The system, built on Intuit's GenOS platform, combines prompt-engineered static explanations with dynamic question-answering using RAG and graphRAG for tax-specific queries. Singh details their fine-tuning of Claude Haiku on AWS Bedrock, which improved quality but proved too specialized. A key focus is evaluation: manual reviews by tax analysts followed by automated LLM-as-a-judge scoring for accuracy, relevancy, and coherence, with a golden dataset and safety guardrails to prevent hallucinated numbers. He highlights challenges like latency spikes on tax day (3–10 seconds), vendor lock-in through expensive contracts, and the difficulty of upgrading models even within the same vendor. Singh emphasizes that evaluations are essential for launching any GenAI feature in a regulated domain like tax.

3 ingredients for building reliable enterprise agents - Harrison Chase, LangChain/LangGraph

Jul 23, 2025 · 20:55

Harrison Chase, CEO of LangChain, argues that reliable enterprise agents depend on three ingredients: high value when right, high probability of success, and low cost when wrong. He says agents succeed by tackling high-value tasks like legal or research, and by doing more work through long-running processes. Reliability improves by blending deterministic workflows with agentic loops, as in LangGraph, and by using observability (LangSmith) to build trust. Cost is lowered by reversible actions (e.g., code diffs) and human-in-the-loop patterns like approvals and time travel. Chase predicts a future of ambient agents—event-triggered, human-supervised background agents—and notes code agents lead because code is verifiable and easily revertible.

From Hype to Habit: How We’re Building an AI-First SaaS Company—While Still Shipping the Roadmap

Jul 23, 2025 · 19:29

Rossella Blatt Vital and Deepsha, AI leaders at Sprout Social, share a candid, real-time look at transforming a SaaS company into an AI-first organization, arguing that the journey is messy, non-linear, and requires evolving across strategy, ways of working, and people. They explain that being AI-first means shifting from adding AI to existing features to reimagining entirely new experiences, while managing the innovator's dilemma of balancing current customer needs with future investments. Key shifts include adopting ritualized discovery—making experimentation a repeatable process—and embracing 'smart velocity' (speed with purpose) over chaotic fast shipping. On the people side, they advocate for T-shaped talent (deep expertise plus broad vision) and building org-wide AI fluency so every team feels empowered to use AI. Despite the disruption, they emphasize that fundamentals like solving real customer problems, user experience, and trust remain non-negotiable. The talk provides a practical framework but no simple playbook, encouraging leaders to start with the right questions and the conviction to evolve.

Machines of Buying and Selling Grace - Adam Behrens, New Generation

Jul 23, 2025 · 19:37

Adam Behrens, CEO of New Gen, argues that AI will transform commerce from static websites to agentic interactions where buyer and seller agents negotiate via intent infrastructure. He traces the evolution from clerk-assisted stores to e-commerce, now to AI-natives where ChatGPT and Claude act as shopping interfaces. Behrens details three challenges: payment delegation (solved via Viza's delegated authentication), product discovery (a unified API akin to Plaid for merchants), and preference representation (two-sided, dynamic, with market-design solutions). He cites Samsung's adoption of an MCP server for chat clients and notes that AI-sourced users convert at higher rates. Behrens predicts revenue sharing via affiliate models and that agents may bypass credit cards for stablecoins, while merchants retain control by embedding transportable data into model providers' surfaces.

How to Build Planning Agents without losing control - Yogendra Miraje, Factset

Jul 23, 2025 · 15:58

Yogendra Miraje from FactSet explains how to build controllable planning agents using agentic workflows and planning by subgoal division, arguing they balance reliability and flexibility for enterprises. He distinguishes workflow agents (static) from agentic workflows (dynamic), and introduces a blueprint generator that pre-selects tools and provides high-level natural language plans to reduce planner cognitive load. He details an architecture using LangGraph nodes (blueprint generator, planner, executor, joiner) and emphasizes designing tools from an agent's perspective with clear purpose, description, and contracts. Miraje stresses evals using code-based and LLM-as-judge techniques, and advises against agentic workflows for fixed tasks, strict compliance, or low-latency needs. The talk includes a concrete example of preparing for NVIDIA's earnings call, showing how blueprints enable structured responses and interpretability.

Building Agents (the hard parts!) - Rita Kozlov, Cloudflare

Jul 23, 2025 · 21:12

Rita Kozlov, VP of Product for Cloudflare's developer platform, presents the building blocks of AI agents—client, AI reasoning, workflows, and tools—arguing that effective agents require all four components. She highlights the Model Context Protocol (MCP) as a standard for exposing APIs to LLMs, and demonstrates Cloudflare's Agents SDK, which simplifies hosting remote MCP servers with built-in OAuth, state management via durable objects, and real-time WebSocket communication. Kozlov cites real-world impact: companies using agents for sales automation see 20% revenue increases, 90% faster support response times, and 50–75% time savings. She walks through a human-in-the-loop credit card approval workflow built with Nok, showing how durable objects maintain long-running state, prevent duplicate actions, and route approvals across Slack, email, or in-app notifications. The talk emphasizes that once an MCP server is deployed, it can be used directly from Cursor, Claude, ChatGPT, or a custom client, including voice interfaces via WebRTC-to-WebSocket translation.

POC to PROD: Hard Lessons from 200+ Enterprise GenAI Deployments - Randall Hunt, Caylent

Jul 23, 2025 · 19:16

Randall Hunt from Caylent shares hard lessons from over 200 enterprise GenAI deployments, arguing that evals, embeddings, and prompt engineering matter far more than fine-tuning. He emphasizes that speed and UX are critical; a slow inference kills adoption, while techniques like generative UI and caching can mitigate latency. Hunt details real-world examples: using audio amplitude spectrographs for sports highlight reels, pooling multimodal embeddings for nature footage search, and noting that nurses prefer chat over voice bots in noisy hospitals. He reports zero regressions moving from Claude 3.7 to 4, and advises optimizing context and economics, such as leveraging Amazon Bedrock batch for 50% cost reduction. The talk underscores that knowing your end customer and minimizing irrelevant context are key to production success.