RAG vs GraphRAG: choosing for your documents

Choose RAG when users mainly need precise answers from identifiable passages. Test graph-based retrieval when the answer depends on relationships across documents or a structured overview of the collection. GraphRAG is not a replacement for RAG: it is one way to enrich retrieval. Choose it only if measured gains justify the additional indexing, maintenance and governance work.

RAG and GraphRAG, in plain words

RAG—retrieval-augmented generation—finds relevant information and provides it to a language model so it can answer using that context.

Think of a librarian who finds the right pages before helping you answer a question. The quality depends on finding the right material, not just on the librarian’s writing style.

Graph-based RAG adds explicit connections between entities, such as projects, components, suppliers and requirements, to help retrieve related information.

This is like giving the librarian an index of relationships as well as a catalogue. A graph can help connect the right pages, but an incorrect connection can also send the search in the wrong direction.

Microsoft’s GraphRAG project is a specific implementation: it extracts entities and relationships, groups them into communities and produces summaries. In this article, “graph-based RAG” is the broader architecture family; “Microsoft GraphRAG” refers to that implementation. The distinction matters when comparing products and cost estimates.

Start with the question, not the database

A larger document collection does not automatically require a graph. Ask what users need to do with it.

User question	Likely starting point	Why
What does this procedure say about inspection intervals?	Passage-based RAG	A specific authoritative passage may contain the answer
Which version of this requirement applies to project A?	RAG with version and project filters	Metadata and document quality may matter more than graph traversal
Which projects depend on a component supplied by this company?	Structured query or graph-backed retrieval	The relationships are central to the question
What recurring problems appear across these reports?	Corpus-level synthesis; test Microsoft GraphRAG global search	A few similar passages may miss the overall pattern
How many open orders do we have today?	Query the authorised system of record	Generated summaries are not a substitute for a live transactional query

These are architectural starting points, not guaranteed results. A robust solution may route different question types to different retrieval methods.

What changes in the architecture?

Two illustrative retrieval paths. Access controls, freshness and evaluation apply to both.

A passage-based system usually prepares searchable document sections, retrieves candidates, and supplies selected material to the model. It need not rely only on vector similarity: keyword search, metadata filtering and reranking can all contribute.

A graph-based system also models relationships. When those relationships already exist in a reliable database, use them instead of inventing them from prose. When extracted by a model, they need provenance, quality checks and a way to handle ambiguity.

Microsoft GraphRAG offers local search around entities and global search over community summaries. Its global search documentation explains how summaries are combined to answer questions about the collection as a whole. That is a different problem from finding one paragraph that answers a narrow question.

When is a graph worth testing?

Consider it when several important questions repeatedly fail because information must be connected across sources, and the missing connections can be represented meaningfully.

Illustrative industrial example: one report names a material, a second records a component using that material, and a third links the component to a project. A question about affected projects may require following those relationships. A graph can make that path explicit.

This is an illustrative scenario, not a customer result or a claim that Elia Studio has delivered a particular GraphRAG deployment.

Before building a new graph, check whether a relational join, a curated reference table or an existing engineering knowledge base already answers the question more directly. Use the simplest dependable mechanism that satisfies the business requirement.

Fix these problems before changing architecture

A graph will not repair documents that were never ingested, unreadable scans, outdated versions or access rights that were lost during indexing.

Investigate the failure first:

• Was the source available and correctly parsed?
• Did the query use the right language, product identifier and version?
• Were useful passages split or filtered out?
• Did retrieval find the evidence but the model misuse it?
• Is the question actually about a structured business record?

Microsoft’s RAG evaluation guidance separates retrieval quality from final-answer quality. That separation helps avoid rebuilding the retrieval layer when the real problem lies elsewhere.

The hidden costs: updates and evidence

Compare total effort, not only the price of answering one query.

Dimension	Passage-based RAG	Graph-enriched retrieval
Preparation	Parsing, splitting, metadata and search indexes	The same foundations, plus graph construction and possibly summaries
Updates	Reprocess changed passages and affected indexes	Also update relationships and dependent derived material
Quality control	Relevant retrieval, correct citations and grounded answers	Also correct entity identity, links and relationship provenance
Operations	Storage, search, model usage and monitoring	Additional graph and extraction operations, depending on design
Best comparison	Cost per useful, authorised answer	The same metric, including graph lifecycle costs

Microsoft documents more than one indexing method. Its indexing-methods guide describes standard extraction and a cheaper FastGraphRAG approach with different graph-quality trade-offs. “GraphRAG cost” is therefore not one fixed number. Measure your language, documents, update frequency and chosen method.

Sovereignty and confidentiality are separate decisions

Neither a vector index nor a graph makes a system sovereign by itself. Check where documents, extracted entities, embeddings, summaries and model requests are processed, stored and backed up, and which organisations can access them.

A graph or summary derived from restricted documents can disclose their contents indirectly. Permissions must apply to derived information as well as original files. Avoid generating a shared global summary from incompatible access scopes and assuming query-time filtering will remove every disclosure.

Keep links to source evidence and a process for updating or deleting derived material when the source changes. For a production system, document ownership of those operations. See our POC-to-production guide.

Run a fair comparison before committing

Use the same authorised corpus, representative questions and scoring rules. Compare a credible RAG baseline—not an intentionally weak vector-only demonstration—with the graph-enriched alternative.

Separate narrow factual questions, relationship questions and collection-wide synthesis. For each category, assess correctness, source support, completeness, access-control failures, latency and cost. Include unanswered questions: the system should acknowledge missing evidence instead of manufacturing a relationship.

For graph-specific checks, inspect ambiguous entity names, false links, missing links, stale relationships and source deletions. Test French and English separately when both occur in your working documents.

Set acceptance criteria before reviewing results. Our agent evaluation guide explains how to avoid average scores hiding important failures. Keep graph retrieval for the question families where its measured benefit is worth the operating burden.

Five questions to ask your AI expert

1. Which actual questions cannot be answered by our current retrieval system, and why?
2. Are the relationships authoritative records or model-generated hypotheses?
3. How do users inspect the evidence behind a connection?
4. What happens when a source is corrected, deleted or restricted?
5. What is the additional lifecycle cost per useful answer?

Start with a decision, not a technology commitment

Define the workflow, collect representative questions and test the simplest credible baseline. Add graph-based retrieval where it solves a demonstrated problem—not because the architecture looks more sophisticated.

Explore our AI consulting and delivery services, or contact Elia Studio to frame a document-assistant use case.