Wearing the Agent: From Group Chats to Glasses — Sai Krishna Rallabandi
Jul 29, 2026 · 19:09
Sai Krishna Rallabandi argues that group and wearable settings force agents to be redesigned around shared memory and security, as single-user assumptions break in multi-user contexts. He shares eight months of deploying Jodith among friends and family, highlighting two core challenges: guarding and memory. On security, two individually safe skills—OCR and reporting—can collide at runtime, leaking PII 90% of the time; his defense is a deterministic guard at the action surface and a LoRA fine-tuned SLM that catches prompt injection even when characters are obfuscated with dots. For memory, he proposes a continuously adapting relevance scorer to compact context and graph-based retrieval for evolving group conversations. Finally, he advocates per-user LoRA adapters on a shared memory layer to bake in privacy permissions instead of code-based access control.
