CrabRAG: Why Automated Assistants Need Graph Memory, Not More Tokens — Stephen Chin, Neo4j

My name's Stephen Chin. I run the Developer Relations team here at Neo4j, and I'm excited to talk to you about something we've all come to love: our crustacean friends. So we have, um, OpenClaw, MASCOT, we have a bunch of other crustaceans, and we're gonna— we're gonna focus on one member of the crustacean family.

I, I, I love crab, so our little boy, CrabD. And I think in the in the journey to, to figure out how to apply agents, how to do things which are more autonomous, we're all looking for ways where we can get better results, more accurate answers, and to actually capture all of this.

But the tools kind of work against us. So, um, here's our, our friend CrabD. He's a— he's a personal assistant. Very happy, very eager. He wants to, to help us out with our lives, maybe to help us to code, to help us to, you know, manage our email, to do different things.

But he's got a problem. And our poor boy CrabD has a very bad memory. He wakes up every day and his memory file flips, and now it's a new day and he forgets everything from yesterday. Does this happen to you where you, you wake up and you're using OpenClaw and suddenly it's on a new set of memory files and remembers nothing that you actually did the previous day?

He's got a lot of tools at his disposal. I mean, we love giving our agents tools, but sometimes he doesn't pick theright tool for the job. I don't think either of these are gonna help him drink his, his bowl of soup.

So that's not the tool which he was looking to, to reach for. And a little bit forgetful at times. So, you know, I think I don't remember everybody I meet, but I'm pretty good at faces. Like, if I've— if I've met you before, I recognize faces.

It's like, "Pleased to meet you." Um, CrabD's not as good at that. So very forgetful. It's like you're reteaching it every day to do the same sort of tasks. And we want agents which are more helpful, which are able to do more for us.

So let's dig into how CrabD actually works. So it's basically a, a memory loop,right? So we're, we're prompting, we're thinking about the response, maybe calling tools, observing what happens. But the hard part is the memory. The hard part is what you put in context, what you're recalling from.

And the way which you have memory structured in most tools is an example of, um, how OpenClaw structures things. Is you have a Solid.md for your agent's memory. You have maybe, um, memory files. You have different tool files.

You have daily memory files. Now, if you look at this, there's one thing which is in common with all of these. They're just markdown files. So markdown files are great. That's easy for us to read. Like, we can— we can look through it.

We can quickly figure out what's not needed and compact them. Um, they're intentionally small for agents because you have a limited context window and also you need to keep theright things at the top of the context. But if your whole memory is a bunch of markdown files, you're wasting a lot of tokens.

So, um, my, my average agents are, are loading up at least 100k in tokens for each round. Um, they're doing a lot of skills. They're adding a lot of things into the context constantly. It's very repetitive because they, they basically load up everything in the hopes that something will be useful in the context.

At small scale, that works where you get the results you want with a high-quality model. It doesn't work at large scale. And I'm gonna show a demo of large scale where we take OpenClaw and we let it run loose on my home lab.

So, um, high demo risk, but a lot of fun. And, um, a classic digital twin scenario. So I think we'll have— we'll have a lot of fun here. Um, anybody use Hermes Agent at all?

Yeah.

Okay. I'm, I'm a big fan of Hermes Agent. Um, I think it's got a much better memory system. It kind of, at the end of each task, it goes and it reflects and it adds back in new skills or new things which it needs.

So, um, it's a really powerful system. And, um, you know, again, we're relying a lot on markdown files. Skills are just basically markdown files. But we can teach the agent to do a lot of things with skills. And it can— it can get theright skill if it gets loaded up and then good things happen.

But sometimes we don't get theright skill loaded up. So our, our poor boy CrabD here is not gonna get that clam. He just doesn't have the open clamshell skill. Lots of shrimp, no clams.

Maybe you pick the wrong skill for the job and suddenly you're, you're jet skiing on the on the beach,right? This, this is not— this is not gonna get him very far.

And

sometimes you, you might get that clam open, but then you don't have the skill to eat them.

So skills, you need to have theright skills, theright chain of skills. Um, actually, we have an awesome project by one of the Neo4j folks which is just been added as an RVicks paper, which is a graph for skills.

So that's an exciting way of like, like figuring out what theright skills are. But maybe we can do better. So, um, Goose is a project that's part of the, um, Agentic AI Foundation. It's a new foundation which MCP is part of.

Um, Anthropic is backing this. We're, we're also a member of this. So it's a— it's a great automation tool for a lot of enterprise workflows. You can also use it kind of like a personal assistant. It relies heavily on MCP as the layer, over 70 MCP extensions.

And what it does is it treats memory just like another MCP server. So this is great,right? It's, it's pluggable. You can call different commands on it to retrieve memories, remember memories, um, forget memories. Memories are just plain files on disks, so now you can manipulate them.

So same great idea, same fundamental problem. We're, we're storing the memory. We're storing the memory of agents as markdown files on disks. And again, you end up with what if what if you pick the wrong tool for the job, the wrong paddle?

Now, in this case, if you pick the wrong paddle, you're a genius 'cause you've invented the most the fastest rising sport in the US, which is pickleball.

Um, actually, the origin of pickleball was, was, um, a family wanted to create a new game, and they just took what they had around the house, a badmining court, and, um, made up the rules along the way. So creation can be good when you have the wrong tools.

Maybe you remember everything, but it's too much. It's too much weight because you can't actually solve the problem. So our poor friend Goose here is encumbered by too many notes, too many memories.

Or most dangerously, now you have MCP tools, you, you're one step away from calling the forget command and just wiping out your own memory.

Okay. So we wanna be able to do better than this. So vector databases,right? So we can store everything. We can create embeddings for it. Now we actually have a da uh, a database. We can store it in a vector database.

So this is great. I mean, you have to pick theright vector database. Um, and then now you can do similarity searches. So you can pull back information which is which is relevant. So we're doing much better. We have a larger repository of knowledge.

We can pull back related information. Um, OpenClaw comes with PG vector out of the box. Give an embedding, you can just start using this. Um, LansDB is a great option. I'm gonna use both of those in my demo.

But the challenge here is similar. What, what vectors give you, which is similarity in vector space, is not the same as actual relationships. And so you get hallucinations. You get a lot of problems when you're relying solely on vector lookup as the answer.

And it compounds with more complex scenarios when you're doing things like, like I'm gonna show you an example of a digital twin. When you're doing things which are very complex, they, they just don't scale. And you make silly mistakes like this is obviously is not what poor CrabD wanted to munch into.

And it's a very expensive lunch for him.

Also, it's sometimes impossible to get to the answer even though you have all the facts because those large multi-hop reasoning chains don't work on similarity searches. They're also very expensive on traditional relational databases.

And often things look similar, but they're not exactly the same. And this is one of the problems with the responses you get from a vector database is you suffer from getting facts which are related in some way and they're not your shell.

And you don't you don't wanna take the wrong shell out of the locker room. That's, that's very unfortunate. So enter graphs. Graphs are a great way of finding the relationships, finding those identities, mapping out the paths, getting that full chain.

And they're built for this sort of connected data. So now that you have first-class nodes, which are the, the circles, edges, those are the relationships between different objects. And then you can put properties on top of graphs to store information.

You can also store embeddings in your graph, and that gives you a way to both use vectors and graphs together. Um, architecturally, the demo I'm gonna show you is, um, both a vector search and a graph search. So it uses the vector search to get the seed nodes where it starts the traversal.

And then it uses a graph search, pulling the, the nearest neighbors and then ranking those by how related they are. And this gives you this complex multi-hop queries to solve more difficult, more domain-specific problems and to figure out where that where that reef is that we wanna get to with all the, the tasty, um, the tasty junk food across the ocean.

And graphs are they're accurate. So they give you very precise information, explainable because you can look at the graph which got returned, and auditable because now you can actually say these are the this is the context. This is the part of the graph which resulted in that answer.

So it's very powerful. And it gives you more tools as a developer where if you're not getting theright answer, you, you know where it's coming from. You can actually see and introspect the graph. And you can change how you're doing extraction.

You can reduce duplicate nodes in the graph. And then you can get to and converge very quickly on a great answer. If you're not a graph expert, guess what? Claude is. Claude can write Cypher better than I can.

Claude can extract build entity extractors, and it can do pretty much everything you need to do to get started with graphs today as long as you know the, the basic kind of model for what you wanna accomplish. And that's what I'm gonna cover in the demo.

So we're gonna do have Claude write each action into the graph as he works. We're gonna follow up by traversing, not rereading it. And then in a fresh session, we will get the results we wanna get out. Now, what I did for this, um, high-stakes demo is I over the past week or two, I took my home lab as the demo environment, did a full digital twin as a graph, and I have two separate environments built off the same original markdown files.

One is a vector database store. That's our, our A test. And the second is a graph store. That's our B test. And the graph store is built on top of, um, Cogni. So I'm using Cogni, which is a startup.

Um, they do amazing stuff in the memory space. They have a Neo4j backend. This is my the structure. So we have a bunch of Proxmox servers in my, my home lab. It's really a couple computers around my desk.

And I built a separate VLAN for the demo, so it's segmented off my real network. So it was trained on real network for my network, but now it's, it's cut off. It can only answer from memory. It can't actually look up the hosts and kind get dynamic information.

So let's see how it does

in a live demo. Okay. So

allright. Here we have our, our CrabRAG cockpit. Um, and I have five different questions queued up with schematics. You can see this is the same home lab schematic that you saw earlier in the slides. And, um, let's, let's start with this one.

So WR and exposed end-of-life soft WAN exposed end-of-life software. So we're gonna basically, we're gonna try to find out if there's anything on my network which is exposed to the network, the, the internet, the WAN that's running out-of-date software which put my home lab at risk,right?

So if, if somebody can attack the home lab. And, um, you can see here that there, there is some servers, um, Tinkster Lands, which is my daughter's Minecraft server. It's running, oh my God, DBNA Jesse. And let's see how the, the two agents did in looking this up.

Okay. So we got the vector response back. Couldn't find specific details. Excluded by policy. For more precise information, yada, yada, yada. Source it separately. Okay. That's, that's not very helpful. Now, on the graph side, it's done a bunch of Cypher queries.

Here are the Cypher queries it's fired off. Um, this is the graph traversal. And the, the color coding on the graph traversal is these blue guys. These are the seed nodes. So this came from a, a vector lookup and a ranking.

But it didn't stop there. It does the one-hop traversals. Those are all the gray nodes. Some of the nodes get highlighted in green, and those are the ones which, which won and got into context. And you can see the answer here.

So guest name, Tinkster Land, exactly as expected. Um, OS version out of date, and it's flagging. So, so it gives us very precise, actionable information. And so that's the difference between same, same exact data. One is a vector store.

One is a graph store. And you can see the difference where the, the vector store is having a lot of trouble pulling the information out, the relevant information out. Okay. Let's try another one just for fun. Um,

let's see. Expose 0.0.0.0 management ports. That's, that's bad. So, um, basically, you, you don't want your management ports on the network exposed to the, you know, the world. And there's a bunch of these. So I have a, a new matrix server I set up and also HProxy, which are exposed to the internet.

That's bad. The rest of these, like my Cogni demo, my OpenClaw instance, those are inside the LAN. You need to get into the LAN to access them. That's, that's what you want. Okay. And let's see how the two agents did in i-identifying this.

So the memory search returns some information, and it's telling me check services configuration. Expect BFSense rule. So it told me to go do the job for it. Um, okay. And on the graph memory side, found an open port exposed to WAN, HAProxy, and OpenVPN, which are the are the two we expected.

Now, this you can see the shape of this graph is entirely different from the previous one. And what it did is it, it actually found the node for, for my router, the PFSense router. And it was able to follow that directly to all of the results which related to it and then give us, like, a very precise answer.

Allright. So, um,

so now, so now we've seen our little boy CrabD with his certified Neo4j developer T-shirt is able to do a lot more,right? Now he's able to follow that full chain, crack, eat, do the next thing. So he's getting his he's getting his clams.

He's helping me fix all the security holes in my network. Um, oh, by the way, I, I patched all those security holes after the demo. Um, so this was good for me too. It found a bunch of security holes in my home lab, and then I, I went and patched them later.

And, um, now we have an agent which actually can do interesting things. Now, if you can imagine, like, I have a three or four-node home lab at home. If, if you have a big enterprise which has a huge data center, if you're doing things in financial services where you have, like, a huge set of companies and customer records you're trying to do, if you're doing anything at, at large scale where it doesn't fit into the one million context window of the modern models, you really need a better memory system than just throwing things in markdown files.

Our little boy CrabD knows his whole crew, all the crustacean friends.

And he's read the book. Um, so we just finished my, my co-authors and I, Michael Hunger and Asus Barasa, finished GraphRAG: The Definitive Guide. The full book is out on, on early release. It'll be published, um, in a couple months once I finish the editorial process.

But super excited about this. It's got information not only on GraphRAG, but also on building memory, on, on different industry vertical use cases, on agents. So it's kind of the, the whole umbrella if you're building on top of graph solutions, how you need to build applications with the technologies you need end to end.

And then finally, a great free resource which everybody in this room can take advantage of is Neo4j, um,'s Graph Academy. So it's free online training, um, dev.neofj.com/ga-rag or the QR code below there. And, um, we have courses on doing agent memory, doing context graphs, and everything you need to get know to get started and to do some of the amazing stuff which I showed you on stage today.

So thank you so much for coming to the, the kickoff talk for the GraphTrack.

You're in theright place for all of the content from Graph experts. Andreas Kolleger, my colleague and I, crafted a great set of speakers from industry experts, people who really know about graph technology. So hang out here, find out more, and then you can see me in the Neo4j booth.

Thank you.