How to Build a Graph RAG System (Playbook)

Standard retrieval returns the chunks most similar to a question. It struggles when the answer requires connecting facts scattered across many documents: which suppliers are linked to which components affected by which advisory, or how a policy change propagates through related procedures. Graph RAG builds a knowledge graph from the content and retrieves through relationships as well as similarity. This playbook covers when it fits and how to build it, following FISTA's AI enablement practice. Concepts are in what is graph rag and what is a knowledge graph.

What does the system do?

Stage	Function
Schema	Entity types, relation types, attributes for the domain
Extraction	Entities and relations from documents with confidence and provenance
Resolution	Deduplication and linking of entities across documents
Graph construction	Graph store with facts linked to source passages
Summarization	Entity and community summaries for aggregate questions
Retrieval	Graph traversal combined with vector search
Generation	Grounded answers citing facts and passages
Evaluation	Multi-hop question sets; extraction quality

Step 1: Confirm the need

Collect the questions the system must answer and classify them: direct lookup, multi-hop relationship, aggregate over entities. Run a standard hybrid RAG baseline and measure. Build graph RAG only if the multi-hop and aggregate categories matter and the baseline fails on them. Decision guidance is in when to use a knowledge graph and rag vs long context.

Step 2: Define the schema

Specify entity types (people, organizations, products, components, policies), relation types with direction and meaning, and attributes, in collaboration with domain experts. The schema constrains extraction and defines what questions the graph can answer. It is versioned. See how to write an ai spec.

Step 3: Extract with constraints and provenance

Use language models constrained to the schema with structured output to extract entities and relations from each chunk, with confidence and the source passage recorded. Validate extraction on a labeled sample per entity and relation type before scaling; extraction errors compound into wrong graph paths. Concepts are in what is structured output and how to build an ai data extraction pipeline.

Step 4: Resolve entities

Merge duplicate entities across documents using identifiers, attributes, and embedding similarity with thresholds, and route ambiguous merges to review. Poor resolution fragments the graph or conflates distinct entities; both produce wrong answers.

Step 5: Construct the graph and link passages

Store entities, relations, and attributes in a graph database, with every fact linked to the passages that support it and to the permission metadata of those passages. Keep the vector index of passages alongside. Generate entity and community summaries for aggregate questions, regenerated as the graph changes.

Step 6: Retrieve by traversal and similarity

For a question: identify candidate entities by lexical and vector matching; traverse their neighborhoods within bounded depth and relation types relevant to the question; retrieve linked passages; run vector search over passages and summaries; and assemble context from facts and passages under permission filters and token budgets. Hybrid retrieval foundations are in how to build a hybrid search system.

Step 7: Generate grounded answers

Generate with citations to both graph facts and passages, state the path of reasoning for multi-hop answers, and refuse when the graph and passages do not support an answer. Validate citations. Grounding controls are in how to ground an llm.

Step 8: Evaluate

Build a labeled set of multi-hop and aggregate questions with expert answers and expected source paths. Measure answer correctness, path correctness, groundedness, citation validity, and refusal correctness, and compare against the standard RAG baseline on the same questions. Track extraction precision and recall per type separately. Method is in the AI evaluation and testing whitepaper.

Step 9: Operate

Keep the graph fresh through incremental extraction on document changes, propagate deletions, regenerate summaries, monitor extraction quality on samples, and watch retrieval latency as the graph grows. Observability design is in the AI observability whitepaper.

Worked example: supplier and component risk

A manufacturer's risk team asks questions like which products are affected by an advisory about a component sourced from a supplier in a given region. Chunk retrieval finds the advisory but cannot connect suppliers, components, products, and regions across the bill-of-materials documents, supplier contracts, and advisories. The schema defines supplier, component, product, facility, region, and advisory entities with sourcing, contains, located-in, and affects relations. Extraction is validated per relation type on a labeled sample; entity resolution merges supplier names across contract and catalog documents. Retrieval identifies the advisory entity, traverses to affected components, their suppliers, and the products containing them, and retrieves the supporting passages. Answers cite the path and the documents. Evaluation on the risk team's question set shows the graph system answering multi-hop questions the baseline could not, while direct questions are still routed to the cheaper standard path.

What does it cost to run?

Extraction is the dominant cost, proportional to document volume and schema complexity, plus graph storage and combined retrieval. Justify it against the value of the multi-hop questions specifically. Drivers are in enterprise rag cost.

What are the common mistakes?

• Building a graph for questions chunk retrieval already answers.
• No schema, so extraction produces an incoherent graph.
• Unvalidated extraction feeding confident wrong paths.
• Facts without passage links, so answers cannot be cited.
• Unbounded traversal producing huge contexts.
• Evaluating on direct questions where the graph adds nothing.

How FISTA Solutions builds graph RAG systems

FISTA Solutions builds graph RAG systems to this playbook only after a baseline proves the need: expert-defined schemas, constrained extraction with validation and provenance, entity resolution with review, graphs linked to passages and permissions, combined traversal and vector retrieval, grounded generation with path citations, and evaluation on multi-hop questions against the baseline. The AI enablement practice delivers the platform, AI agents build on it, and forward deployed engineers embed with your domain experts to define the schema and question set. The record behind the work is 150+ projects with 99.9% uptime.

To assess whether graph RAG fits your questions, message FISTA on WhatsApp, or read the enterprise RAG reference architecture whitepaper for the base system.