RAG taught the model to answer from your data: find similar chunks of text — mix them into the prompt. But "similar chunks" have a ceiling that real systems hit. The next step is giving the model not chunks of text but a structure of knowledge: concepts and relations. That's a knowledge graph — an ontology filled with facts.

Where vector search is blind

Vector RAG finds fragments similar to the question in meaning. That works great for "what is X" questions — the answer sits in one or two chunks. And poorly for three classes of questions:

  • Multi-hop. "Which services will a change to the payment API contract affect?" The answer isn't written anywhere as one chunk — it is derived along a chain of links: contract → its consumers → what depends on the consumers. Vector search will find the document about the contract but won't walk the chain.
  • Aggregating. "How many integrations with external providers do we have?" The answer is a count across the whole knowledge base, not a text fragment.
  • Exact links. "Who owns service X?" — there are many similar chunks (mentions in different docs, old and new), but you need one authoritative fact, not "the most similar paragraph."

The common cause: text stores knowledge implicitly, smeared across phrasings. To answer questions about links, links must be stored as links.

A knowledge graph: facts instead of paragraphs

A knowledge graph stores knowledge as fact triples: subject → relation → object. "payment-service — consumes — orders-api"; "orders-api — owned by — team checkout." Nodes are concepts, edges are relations.

And to keep the graph from becoming a heap of arbitrary arrows, you need an ontology — the graph's schema: which node types exist (service, team, contract), which relations are allowed (a service consumes a contract, but not "a service is friends with a team"), which rules apply. An ontology is to a graph what a concept model is to a product: without it everyone writes facts in their own vocabulary and the graph never knits together.

A graph can be walked: from a contract to its consumers, from them to dependents — any number of hops, with an exact, verifiable result at each step. Exactly what vector search can't do.

GraphRAG: graph plus model

GraphRAG is RAG where the prompt's source is not only a vector index but a knowledge graph. The typical scheme:

  1. Entities are extracted from the question ("payment API" → the orders-api node).
  2. The graph is walked around those entities — an exact subgraph is collected: consumers, owners, dependencies.
  3. The subgraph (plus, if needed, vector-retrieved text chunks) is mixed into the prompt.
  4. The model phrases the answer — from facts with links, not from "similar paragraphs."

The model and the graph cover each other's weaknesses: the graph can't converse or understand natural-language questions — the model can; the model invents things and doesn't derive chains reliably — the graph holds verifiable facts and links. The LLM also helps build the graph: extracting entities and relations from documents — with mandatory validation against the ontology, or hallucinations leak into the graph.

When a graph pays off — and when it's overkill

A knowledge graph isn't a default upgrade over RAG but a tool with a price: designing the ontology, populating it, and keeping it current (a stale graph is worse than none — it lies confidently).

  • Not needed when questions to the knowledge base are "find and retell": plain RAG is cheaper and sufficient.
  • Needed when questions are about links and consequences: change impact analysis, service and team dependencies, compliance ("which systems touch personal data?"), catalogs with relations.

And notice a familiar pattern: a registry of services with owners, contracts, and links between contexts — we've already described such an artifact as specs scaled to the platform level. That is your product's knowledge graph, just written in files: the question "what does this contract change affect" over it is a classic graph walk. Ontology, spec, and knowledge graph are one discipline at different scales: write concepts and relations down explicitly — and both people and agents can work over them.

In short

  • Vector RAG is blind on multi-hop, aggregating, and exact-link questions: text stores links implicitly.
  • A knowledge graph = fact triples "subject → relation → object"; an ontology is its schema (node types, allowed relations, rules), without which the graph never knits.
  • GraphRAG: entities from the question → graph walk → an exact subgraph into the prompt → an answer from facts. The model and the graph cover each other's weaknesses.
  • The graph's price is ontology design and freshness upkeep; for "find and retell" plain RAG suffices, for questions about links and consequences the graph is irreplaceable.
  • Ontology, spec, and knowledge graph are one discipline of writing concepts and relations down explicitly — both people and agents work over it.

This is the last article in the domain-model cluster. Together with the system of concepts, the conceptual model, the ubiquitous language, and the spec, you have the full path: from "what does the problem consist of" to a document an agent executes — and a graph you can ask questions.