How to Build a Confluence Knowledge Agent

Confluence accumulates the organization's knowledge and then hides it behind search results that rank old pages above current ones and restricted pages that appear for some users and not others. A Confluence knowledge agent retrieves under each user's permissions, answers with citations, says how current the content is, and turns what it cannot answer into a documentation backlog for the people who own the pages. This playbook builds it, applying the enterprise RAG reference architecture and the knowledge-management context in AI knowledge management; the engineering-specific variant is how to build an internal developer agent.

Step 1: What is the agent's role?

Element	Definition
Purpose	Answer questions from Confluence with citations under each user's permissions, and surface documentation gaps
Scope	Selected spaces; question answering; freshness and conflict flags; documentation requests; draft page updates for owner review
Non-scope	Publishing pages; changing permissions; answering from general knowledge; restricted content for unauthorized users
Owner	Knowledge or documentation lead, with space owners accountable for content

Step 2: What should be indexed, and how?

Content	Handling
Pages and sections	Chunk by heading with page hierarchy and breadcrumbs preserved
Page metadata	Space, owner, last updated, version, labels, restrictions
Macros and tables	Rendered to text with structure preserved; tables as tables
Attachments	Index supported document types with the page's permissions
Comments	Index with lower weight; mark as discussion, not policy
Archived and draft pages	Exclude by default

Permissions and dates are first-class metadata; both are used at retrieval time. Re-index incrementally on page changes rather than on a nightly schedule, because a runbook corrected at noon should not be answered from its broken version until tomorrow.

Step 3: How are permissions enforced?

Every retrieval carries the asking user's identity and filters by space and page restrictions; restricted pages never surface for users who cannot open them; answers are never cached across permission boundaries; and permission changes trigger re-indexing of the affected pages. The delegated-identity model is in the agent identity and access control whitepaper.

Step 4: How is freshness handled?

Each answer states the date of the content it used. Recency is weighted where the space's content is time-sensitive (runbooks, policies) and not where it is not (design rationale). Conflicts between pages are flagged in the answer with both sources, rather than silently resolved by rank. Stale-page reports, pages older than a threshold for their space, go to space owners monthly.

Step 5: How are answers grounded and gaps captured?

Answers cite pages and sections with links; ungrounded answers are prohibited; when retrieval finds nothing relevant, the agent says so and offers the space owner as a contact. Every unanswered question is logged as a documentation request to the relevant space, which is how the agent improves the wiki rather than papering over it. Hybrid retrieval and reranking follow how to build a hybrid search system.

Step 6: What is the evaluation set?

Real questions from the channels where people ask (support, onboarding, team chats), with verified answers and supporting pages, including questions with no answer in Confluence. Score retrieval, grounding, correctness, freshness statements, and refusal correctness. Gate changes on the set; sample live answers weekly; thumbs-down responses feed the set and the backlog.

Step 7: Rollout

1. One space with an engaged owner, answers only, in a channel where those questions are asked.
2. Add freshness and conflict flags.
3. Add documentation-request routing.
4. Add draft page updates for owner review.
5. Expand space by space with owners' agreement.

What does the agent look like in daily operation?

A new engineer asks in the team channel how deployments are approved. The agent answers from the current process page, cites the section, notes the page was updated three weeks ago, and links the approval form. A second question about the on-call escalation path returns two conflicting pages; the agent presents both with dates and flags the conflict to the space owner, who retires the old one that afternoon.

A product manager asks about a customer's contract terms; the page is restricted to the legal space and the manager has no access, so the agent finds nothing it may show and points to the legal team contact. At month end, the space owners receive a report: eleven questions nobody's pages could answer, six stale runbooks, and three page conflicts, which becomes their documentation sprint. Every answer carried its citations, and nobody saw a page they could not open.

How does this fit the wider platform?

The Confluence agent shares retrieval, identity, gateway, and evaluation with every other agent; the same indexed, permission-aware content serves the IT helpdesk role and the HR inquiry role through their own surfaces. The retrieval layer is built once, as described in the enterprise RAG reference architecture.

What is measured?

Metric	Why
Answer correctness and grounding	Core quality
Citation click-through and ratings	Trust
Unanswered questions and gaps closed	Documentation improvement
Stale and conflicting pages resolved	Content health
Escalations to people	Coverage

What are the common mistakes?

1. Indexing without permissions, turning the agent into a way around restrictions.
2. Ignoring dates, so old pages answer confidently.
3. Answers without citations.
4. Gaps not routed, so the wiki never improves.
5. Publishing without owner review.
6. Indexing every space at once, which floods retrieval with low-quality pages and makes the first answers worse than a targeted single-space launch would have been.
7. Treating comments as policy, when they are discussion that may contradict the page they sit under.

How does FISTA Solutions help?

FISTA Solutions builds knowledge agents over Confluence and other wikis as governed AI agents on the retrieval and identity platform its AI enablement practice establishes, with forward deployed engineers working with space owners to curate content and build the evaluation set. FISTA has delivered 150+ projects for 50+ companies across 12+ countries.

To build a knowledge agent for one space, message FISTA on WhatsApp, or read how to build AI search for SharePoint for the Microsoft equivalent.