Multimodal Collaborative Agents for Next-Gen Commerce — Nidhi Kaushik Vyas, Google DeepMind
Sep 1, 2026 · 21:08
Nidhi Kaushik Vyas of Google DeepMind argues shopping agents fail as wrappers around the search bar, assuming well-formed intent, when users arrive with only a vibe — closing the articulation gap is the agent's job. Her discovery-research-response loop starts by building a working state from conversation, context and reference images, separating hard constraints from soft ones an image implies, and flagging inventory as a real-time variable. Research picks the highest information-gain question — room width, since all is moot if furniture won't fit — using visual boards for subjective tastes. Response adapts format — summary for policy questions, comparison tables, imagery — and autoraters grade every stage, including counterfactual tests flipping query parts to check constraints move when they should. A Q&A covers merchant ontology and UCP.
