Code knowledge graph vs RAG: why structure beats similarity

There are two default answers to 'give my AI assistant context about this codebase.' The first is RAG: chunk the repo, embed the chunks, retrieve the most similar ones at query time. The second is a code knowledge graph: extract the actual structure — calls, imports, definitions, references — and let the assistant traverse it. They sound like variations on a theme. They are close to opposites, and the difference shows up exactly where codebase questions get hard.

What RAG actually does

A RAG pipeline — the approach introduced by Lewis et al. (2020) for knowledge-intensive NLP tasks — splits documents into chunks, turns each chunk into a vector embedding, and at query time returns the top-k chunks nearest to the question in embedding space. The retrieval criterion is resemblance: text that looks like the question. For prose, that's often exactly right — ask about a refund policy and the paragraphs discussing refunds are the answer.

Code breaks the assumption underneath this. In a codebase, the signal isn't in what any chunk says; it's in how the pieces relate. And chunking destroys relationships by construction — a function and its callers live in different files, a schema and the code that queries it share almost no vocabulary, and the answer to 'what breaks if this changes?' isn't written down in any chunk at all.

Three places similarity fails on code

First, the multi-hop wall. 'What breaks if I change this function's signature?' is a three-or-four-hop question: function → callers → their callers → the schema they serialize. A vector store returns chunks that mention the function, or signatures, or breakage, and hopes the model stitches the chain together. Each hop multiplies the failure modes.

Second, the vocabulary gap. Similarity retrieval finds text that shares language with the query. But a caller doesn't describe its callee, and a Terraform resource doesn't describe the service that consumes it. The most structurally relevant code for a question is frequently the least textually similar to it.

Third, no provenance. A retrieved chunk arrives with a cosine score — 0.83 — that says nothing about why it's relevant or whether the relationship it implies is real. You can't audit a similarity score. Retrieval that sounds right and retrieval you can check are different products.

What a graph does instead

A code knowledge graph stores the relationships directly. Graphify extracts them from the AST with tree-sitter, a deterministic parser — a call edge exists because the parser found the call, not because two chunks landed near each other in embedding space — and uses your model only for non-code sources, tagging those edges INFERRED so they're distinguishable from what was extracted. Questions become traversals: follow the edges, and the answer arrives as a path of real nodes through real files, each edge labeled with how it was established. Hop three is exactly as reliable as hop one, and the answer is its own explanation.

Where RAG still wins

Honesty requires the other column. Similarity search is genuinely better for fuzzy recall over prose — 'where did we discuss retry behavior?' across docs, issues, and ADRs — and for any query where resemblance is the actual relevance criterion. It's also simpler to stand up over arbitrary unstructured text. If your corpus is documentation and your questions are lookups, RAG is a fine tool. The claim here is narrower and stronger: for structural questions about code, retrieval by resemblance is the wrong shape, and no amount of chunking strategy fixes a representation that threw the structure away.

Check the claim on your own repo

This argument is testable in an afternoon: uv tool install graphifyy, then /graphify . in your repo, then ask graphify path how two distant pieces of your system connect — and ask your RAG setup the same question. The side-by-side comparison lives at Graphify vs RAG, with the vector-database-specific version at Graphify vs vector databases. For the mechanics of how the graph gets built and tagged, start with concepts.