Chrome

Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori

Aug 14, 2026 · 21:00

Dhruv Batra of Yutori argues computer-use models, not APIs, will agentify the web's long tail. The head may expose endpoints, but 200 million active sites won't add MCP servers: restaurants post JPEG PDF menus and school districts answer procurement via FOIA requests scanned onto Google Drive. Reading HTML fails because scores and stock status load asynchronously and render as pixels, so the browser is a rendering engine: pixels are the source of truth, the bitter lesson for web agents. His Navigator model takes screenshots in, clicks out, writes JavaScript when faster, and verifies on screen; it hits 97% human eval on Mind2Web (8 of 300 wrong) at 80 cents per task versus $2.30. That endpoint will be another layer: simulated browsers clicking and returning structured results.

Teaching AI to Find Real Vulnerabilities — Prof. David Brumley, Bugcrowd

Aug 1, 2026 · 27:17

Carnegie Mellon professor and Bugcrowd chief AI officer David Brumley argues that teaching AI to hack mirrors human learning: a ladder from crashes to arbitrary code execution, graded by deterministic oracles rather than LLM judges. He shows why benchmarks fail when targets hold multiple vulnerabilities — models reward-hack the easiest bug — and presents his 'audit task' scoring precision and recall across all discovered bugs. Testing on Chrome's V8 with 41 real vulnerabilities, MITHOS hit 73% full code execution and GPT 68%, while Gemini and Kimi scored 0%. Several exploits were novel: MITHOS reverse-engineered Math.random to forge a pointer, found a new WASM path, and produced a real zero day. The episode grounds this in stories from picoCTF winner Fluorescence to DARPA's Cyber Grand Challenge, urging RL environments built on real bugs over benchmark-maxxing security.

Why Large? Tiny LMs & Agents on Edge/Robotics — Cormac Brick, Google

Jul 25, 2026 · 21:45

Cormac Brick of Google AI Edge argues that the real constraint on edge AI is DRAM cost, not compute, making tiny models essential for widespread deployment. Small models (1–4 billion parameters) require 4–8 GB of RAM, but their quantized Gemma 2B model—at just 2.9 bits per weight—runs on a Raspberry Pi at 7.6 tokens per second and on a Qualcomm NPU at 31 tokens per second. For even lower-end devices, tiny models (50–500 million parameters) need under 2 GB of RAM and can be fine-tuned for specific tasks like voice-to-function calling, reaching over 86% reliability on ten actions. A shipped example is an offline voice dictation app that uses two fine-tuned sub-billion Gemma models to clean up ums and ahs without a subscription. The talk covers the Gemini-optimized toolchain (Lighter TLM, MediaPipe), synthetic data generation for fine-tuning, and the trade-off between zero-shot prompting in small models versus fine-tuning tiny models for reach and speed.

Full Workshop: Setting Yourself Up for Success —Jason Liu, OpenAI Codex

Jul 24, 2026 · 1:15:02

Jason Liu walks through how to set up OpenAI's Codex for maximum productivity, using voice dictation, app shots, computer use, and a personal memory vault to delegate and automate almost every task. He demonstrates creating skills and plugins from past work, using compaction to keep threads running for weeks with hundreds of sub-agents, and setting up automations with heartbeats and goals. Specific examples include auto-checking flights, editing iMovies via computer use, and having a chief-of-staff thread that monitors Slack and updates task lists. He also covers security permissions (auto-review vs full auto), tips for new users to start with low thinking mode to save tokens, and how to build skills that self-improve by editing their own files. The workshop emphasizes that threads can now communicate with each other, enabling manager-like orchestration and long-running work streams.

Perception Agents — Antje Barth, Amazon AGI Lab

Jul 23, 2026 · 21:45

Antje Barth of Amazon AGI Lab introduces perception agents — AI that sees and interacts with screens visually rather than through APIs — arguing that while agents can click and type, they fail at end-to-end knowledge work because reliability and trust are missing without verifiable outputs. Unlike coding agents, which succeeded due to unit tests, most work lives in the messy seams of applications where no easy verification exists. Barth's solution is a perception-action loop: the agent perceives the rendered screen, plans, acts, and checks its own work in real time, much like humans collaborating over a shared screen. She demonstrates two open-source tools: an annotation Chrome extension that lets users mark elements and issue precise commands, and a verification tool that checks visual design and user flows against specs. A live demo shows a Bee device capturing a meeting transcript that instantly triggers agent actions and verification. The talk closes with a call to build in the open, emphasizing that shared context between human and agent is the missing piece for reliable, trusted AI assistants.

Frontier results, on device - RL Nabors, Arize

Jun 29, 2026 · 30:52

RL Nabors (Arize) argues that most frontier-model calls can be replaced by smaller, local models, saving cost, latency, and energy. She presents a four-step framework: prototype big with a foundation model, collect a golden dataset, run capability evals using Arize's open-source Phoenix, then select the 'sage' (small and good enough) model. Demonstrating with her social app Mima, she tested Qwen 2.5, Qwen 3, LLaMA 3.2, and Gemma 4 against Claude Sonnet on summary accuracy, latency, and cost. LLaMA 3.2 (3B params) won at 90% accuracy and 1-second P50 latency, versus Gemma 4's 8 seconds. Prompt engineering—specifically few-shot prompting—closed the gap further, achieving 92.9% factual consistency and 100% JSON validity. Nabors emphasizes running regression evals to prevent regressions, and notes that on-device inference eliminates data exposure and round-trip latency.

The agent-ready web: Simplify user actions with WebMCP — Tara Agyemang, Google

Jun 11, 2026 · 21:34

Tara Agyemang from the Google Chrome team introduces WebMCP, a proposed web standard that replaces brittle DOM scraping with structured tools for AI agents. She explains two implementation paths: the declarative API (adding HTML attributes to forms) and the imperative API (registering custom JavaScript tools). A live demo shows a concert ticket purchase completed in three tool calls: search, open page, purchase. WebMCP is in early preview on Chrome 146, with an eval CLI and inspector extension available for testing.

Defying Gravity - Kevin Hou, Google DeepMind

Dec 2, 2025 · 25:10

Kevin Hou from Google DeepMind introduces Antigravity, a new agent-first IDE that combines three surfaces: an AI editor, an agent-controlled Chrome browser, and an Agent Manager. The central claim is that agents should live outside the IDE, enabling longer-running tasks and multimodal interactions. Key features include Artifacts—dynamic representations like plans, screen recordings, and diagrams—that replace raw chain-of-thought with visual summaries. The browser enables context retrieval and verification via screen recordings, while image generation allows iterative design through comments. Hou explains the research-product flywheel: DeepMind researchers use Antigravity internally to identify model gaps, improving capabilities like computer use and instruction following, which then ship in the product. He also details four categories of model improvements: intelligence, tools, long-running tasks, and multimodal, all powered by Gemini 3 Pro and related models.

How LLMs work for Web Devs: GPT in 600 lines of Vanilla JS - Ishan Anand

Jul 13, 2025 · 1:41:34

Ishan Anand shows that GPT-2 small implemented in 600 lines of vanilla JavaScript makes LLMs understandable for web developers without ML backgrounds. He explains tokenization via byte-pair encoding, 768-dimensional embeddings representing semantic meaning via co-occurrence, and the Transformer's attention mechanism that lets tokens share context. The multi-layer perceptron learns next-token prediction through backpropagation, while the language head converts embeddings to token probabilities using softmax. Anand demonstrates each step—tokenization, embedding lookup, positional encoding, attention, MLP, and output—in a browser debugger, and notes that GPT-2's architecture underpins ChatGPT, with innovations like scale, supervised fine-tuning, and RLHF. The workshop provides an intuitive mental model of Transformers, turning perceived AI magic into understandable machinery.

Windsurf everywhere, doing everything, all at once - Kevin Hou, Windsurf

Jun 23, 2025 · 16:03

Windsurf head of product Kevin Hou explains the company's philosophy of a shared human-AI timeline and its vision to be everywhere, do everything, and work all the time. The AI ingests context from tools like Google Docs, Figma, GitHub, and Notion, and takes actions beyond coding—opening PRs, deploying, and reviewing code. Windsurf generates 90 million lines of code daily and processes over 1,000 messages per minute. Its new model SWE-1, trained on software engineering workflows, achieves near-frontier performance with a fraction of the resources. Hou emphasizes a data flywheel where user feedback on real workflows drives continuous improvement, arguing that successful AI products in 2025 require harmony among model, data, and application.

Arrakis: How To Build An AI Sandbox From Scratch - Abhishek Bhardwaj, OpenAI

Jun 3, 2025 · 40:18

Abhishek Bhardwaj, founder of Arrakis, explains why MicroVM-based AI sandboxes are the next unlock for AI agents, detailing how Arrakis provides secure, fast code execution and computer-use environments. He argues that sandboxes are essential for tool-calling models, reinforcement learning, and multi-tenant security, with Arrakis booting in under 7 seconds and supporting backtracking via snapshot/restore. The talk covers the choice of Cloud Hypervisor over other VMMs, overlayFS storage with per-sandbox read-write layers, iptables-based networking, and a built-in code execution server. A demo shows Claude using Arrakis via MCP to build a collaborative Google Docs clone and then revert to a snapshot, illustrating how a full Linux sandbox enables agents to autonomously debug and iterate without extensive prompting.