RAG is an interpreter. The LLM Wiki is a compiler

RAG trained us to accept a strange habit: every question pays the full price of understanding the corpus again. The LLM Wiki pattern refuses to pay that price at query time. It moves synthesis to ingestion, where an agent reads each source once, writes what it learned into a persistent set of cross-linked markdown pages, and keeps those pages consistent as new sources arrive.

The pattern took shape in April 2026, when Andrej Karpathy posted about using LLMs to build personal knowledge bases and followed up with an idea file, llm-wiki.md, published as a gist. The gist is deliberately tool-agnostic: you hand it to an agent such as Claude Code or Codex and let the agent scaffold the rest. His own summary of the roles is hard to improve on: Obsidian is the IDE, the LLM is the programmer, the wiki is the codebase. Skill packages and open-source variants appeared within days.

Six months on, the implementations are still young, but the architecture has settled enough to reason about as engineers: file layouts, operations, memory semantics, integration surfaces, and the places where it breaks. My position is that the wiki is not a smarter retriever. It is a build artifact, and it deserves the discipline we give build artifacts.

The architecture shift: compile, then read

A retrieval-first system is an interpreter. It takes raw text and a question, and works out the meaning from scratch on every run. A compilation-first system is a compiler. It does the expensive semantic work once per source, emits an intermediate representation (the wiki), and every query executes against that representation.

The unit of work changes accordingly. In RAG the unit is the query: embed it, pull top-k chunks, synthesize, discard. In an LLM Wiki the unit is the source: read it, decide which pages it touches, rewrite those pages, record what changed.

What retrieval actually hands the model

Top-k retrieval returns fragments ranked by similarity to the question, not by their role in the corpus. Three failure modes follow, and they are structural rather than tuning problems.

• Decontextualized fragments. A chunk reading “the limit was raised to 500 requests per second” arrives without the service it applies to, the date, or the document that later lowered it again. Chunk boundaries routinely sever a definition from the term it defines.
• Silent contradictions. A 2024 runbook and a 2026 runbook both rank highly for the same question. The model blends them or picks one, and nothing records that a conflict exists. The same contradiction is re-resolved, differently, on every query that hits it.
• Redundancy eating the budget. If forty documents repeat the same onboarding paragraph, eight of your ten retrieval slots can go to near-duplicates, and the one chunk with the exception never makes the cut.

Underneath all three sits a ceiling: synthesis is bounded by k. A question whose answer depends on thirty sources cannot be answered well from ten chunks, however good the reranker is.

What compilation resolves before the question exists

At ingestion the agent sees the new source next to the pages it affects, so it can do work no query-time system can amortize:

• Deduplicate forty restatements into one canonical page with forty citations.
• Resolve identity: “the IAM team”, “platform security” and “secops-iam” become one entity page with aliases.
• Surface conflicts explicitly: the 2024 and 2026 claims both stay on the page, each with its citation, inside a conflicts block that a human or a lint pass must close.
• Materialize relationships as bidirectional links, so the graph that RAG re-infers per query exists on disk.

The economics follow from where the cost lands. RAG ingestion is cheap and every query pays retrieval plus synthesis. Wiki ingestion is expensive (one source can rewrite 10 to 15 pages) and queries become mostly reading. If a source will inform more than a handful of answers over its lifetime, compiling it wins.

The deeper gain is auditability. Every decision the compiler made is a markdown diff in git. You can review it, revert it, or blame it. A vector index offers no equivalent artifact to review.

The three-layer architecture

The pattern has three layers with strict ownership boundaries. Most failures come from blurring them: an agent that “fixes” a raw source, or a human who hand-edits a wiki page the agent will silently overwrite on the next ingest.

Data flows one way: raw sources feed ingest, the wiki is the only layer that changes, and external readers write back only through a reviewed edit queue.

kb/
├── CLAUDE.md              # layer 3: schema (AGENTS.md for Codex and others)
├── raw/                   # layer 1: immutable sources, agent read-only
│   ├── manifest.jsonl     # sha256, path, ingested_at, ingest_commit
│   ├── papers/
│   ├── transcripts/
│   └── prs/
└── wiki/                  # layer 2: LLM-owned markdown
    ├── index.md           # catalog: one line per page, by category
    ├── log.md             # append-only operation log
    ├── sources/           # one summary page per raw document
    ├── entities/          # services, teams, vendors, people-as-owners
    ├── concepts/          # definitions, decisions, patterns
    └── syntheses/         # answers worth keeping, filed back from queries

Layer 1: raw sources

Raw storage is the audit trail and the ground truth every wiki claim must trace back to. The agent reads it and never writes to it.

• Enforce immutability mechanically, not by prompt: read-only mounts, chmod -R a-w raw/, or a pre-commit hook that rejects any diff under raw/ that is not a pure addition.
• Key every source by content hash in a manifest. Re-ingestion detection, retraction (“remove everything derived from source a91f3c“) and drift audits all depend on it.
• Normalize to text at the boundary (PDF to markdown, transcript to speaker-labelled text) and store the normalized form next to the original, so the agent never re-parses binaries.

Layer 2: the wiki

The wiki is the only layer the agent owns, and it owns it completely. Two files carry the navigation load.

• index.md lists every page with a link, a one-line summary and an updated date, grouped by category. It is the first thing the agent reads on any query, so its line format is a performance budget: 15 to 25 tokens per entry.
• log.md is an append-only chronological record of ingests, queries and lint passes. Karpathy suggests a consistent prefix per entry so the log stays greppable with plain Unix tools. It doubles as episodic memory.

Every other page follows a template the schema defines. A workable entity page:

---
type: entity
aliases: [payments-gw, "the gateway"]
sources: [a91f3c, 07be21, c4d90e]
updated: 2026-10-12
revisions: 7
---
# payment gateway

Internal service fronting card acquirers. Owned by [[team-payments-core]]. [src:a91f3c]
Rate limit: 800 rps per merchant since the Q3 capacity review. [src:c4d90e]

## conflicts
- Timeout: 30 s per [src:07be21] (runbook, 2024) vs 12 s per [src:c4d90e] (ADR-112, 2026). Unresolved.

## related
[[acquirer-routing]] · [[adr-112-timeouts]] · [[incident-2026-08-14]]

Two conventions carry most of the weight: every factual sentence ends in a source tag, and contradictions get their own section instead of being edited away.

Layer 3: the schema

The schema file (CLAUDE.md for Claude Code, AGENTS.md for Codex) is the compiler’s source code. It turns a general-purpose agent into a disciplined wiki maintainer, and it is the most underrated of the three layers.

# wiki schema v3

## invariants
- Never write under raw/. Never delete a wiki page; mark it `status: retired`.
- Every factual sentence ends with [src:<hash>]. No tag, no claim.
- Never resolve a contradiction silently. Add it under `## conflicts` with both sources.
- Slugs: kebab-case, singular, no dates except for incidents and ADRs.

## ingest
1. Read index.md. List entities and concepts in the source; map each to an existing page or mark NEW.
2. Write sources/<hash>.md (summary, key claims, entities mentioned).
3. Update each mapped page; create NEW pages from templates/.
4. Add reciprocal [[links]]. Update index.md. Append to log.md: `## [date] ingest | <title> | <n> pages`.

## query
Read index.md first. Open at most 8 pages, follow links one hop if needed. Cite as [[slug]].
If the answer is a new synthesis, propose filing it under syntheses/.

Treat the schema like code. Version it, review changes to it, and expect to co-evolve it with the agent over the first few weeks: most of the quality gains come from tightening it after reading what the agent actually wrote.

The agentic operational loop

Three operations maintain the wiki: ingest writes, query reads (and sometimes files back), lint repairs. The blueprints below are Python architecture sketches, not a library. agent.structured() stands for any call that returns schema-validated output (tool use or JSON mode); the interesting part is where state lives and what gets checked before it is written.

from dataclasses import dataclass, field
from pathlib import Path
import hashlib

ROOT = Path("kb")
RAW, WIKI = ROOT / "raw", ROOT / "wiki"
INDEX, LOG = WIKI / "index.md", WIKI / "log.md"
SCHEMA = (ROOT / "CLAUDE.md").read_text()

@dataclass
class Conflict:
    claim: str
    existing_src: str
    new_src: str

@dataclass
class PageEdit:
    path: str                 # "entities/payment-gateway.md"
    action: str               # "create" | "update"
    content: str              # full new markdown, never a partial patch
    conflicts: list[Conflict] = field(default_factory=list)

Ingest

Ingest runs in two passes. The plan pass sees only the index and the source, and decides which pages are affected. The write pass sees the source plus only those pages. This keeps context bounded even when the wiki holds thousands of pages.

def ingest(src: Path, agent) -> list[str]:
    digest = hashlib.sha256(src.read_bytes()).hexdigest()[:12]
    if manifest_has(digest):
        return []                                   # idempotent re-runs
    text = normalize(src)                           # pdf/html/vtt -> markdown

    # pass 1: plan against the catalog, not the whole wiki
    plan = agent.structured(
        system=SCHEMA,
        prompt=f"INDEX:\n{INDEX.read_text()}\n\nSOURCE [{digest}]:\n{text}",
        schema=IngestPlan,      # entities/concepts -> existing slug | NEW
    )

    # pass 2: rewrite only the touched pages (typically 10 to 15)
    touched = {p: (WIKI / p).read_text() for p in plan.existing_pages()}
    edits: list[PageEdit] = agent.structured(
        system=SCHEMA,
        prompt=render_write_prompt(text, digest, plan, touched),
        schema=list[PageEdit],
    )

    validate(edits, digest)     # see below; reject the whole batch on failure
    for e in edits:
        write_atomic(WIKI / e.path, e.content)
    rebuild_index_entries(edits)
    append_log("ingest", src.name, digest, edits)
    manifest_add(digest, src, commit=git_commit(f"ingest {src.name}: {len(edits)} pages"))
    return [e.path for e in edits]

The validator is where the architecture earns its trust. It is deterministic and cheap, and it runs before anything touches disk:

def validate(edits: list[PageEdit], digest: str) -> None:
    for e in edits:
        assert not e.path.startswith(("..", "/", "raw/")), "write outside wiki/"
        for sentence in factual_sentences(e.content):
            assert SRC_TAG.search(sentence), f"uncited claim in {e.path}"
        for link in wikilinks(e.content):
            assert link_target_exists(link) or created_in(edits, link), f"dangling [[{link}]]"
        if e.action == "update":
            removed = claims(old(e.path)) - claims(e.content)
            assert removed <= {c.claim for c in e.conflicts}, "claim dropped without a conflict entry"

That last check is the contradiction contract. The agent may add claims freely, but it may not remove or change an existing cited claim unless it records a Conflict. Conflicts land in the page’s ## conflicts section and in log.md with a conflict tag, so a human or a later lint pass can close them. The agent flags; it does not adjudicate.

Commit once per ingest. A source that touches fifteen pages has a fifteen-page blast radius, and git revert <ingest-commit> is the cleanest retraction mechanism you will get.

Query

Query is navigation, not search. The agent reads the catalog, picks pages, optionally follows links one hop, and answers with page citations.

def query(question: str, agent, budget: int = 30_000) -> Answer:
    index = INDEX.read_text()                              # ~8k tokens at 400 pages
    picks = agent.structured(
        prompt=f"{index}\n\nQUESTION: {question}\nChoose up to 8 pages.",
        schema=list[str],
    )
    pages = {p: read_page(p) for p in picks}

    for p in list(pages):                                  # one-hop expansion, budgeted
        for link in ranked_links(pages[p], question):
            if count_tokens(pages) >= budget:
                break
            pages.setdefault(link, read_page(link))

    ans = agent.answer(question, pages, cite="[[slug]]")
    if ans.is_new_synthesis:
        propose_filing(ans, under="syntheses/")            # answers compound too
    append_log("query", question, sorted(pages))
    return ans

Be precise about the token claim. A query reads the index plus five to ten pages, often 15k to 30k tokens. Classic top-10 RAG over 512-token chunks sends about 5k. The wiki does not win on raw tokens per query; it wins on what those tokens contain. Each page is already deduplicated, reconciled and linked, so the model spends its context on synthesized knowledge rather than on overlapping fragments, and it never has to load the corpus to answer a cross-source question.

Filing answers back is what makes queries compound. A comparison the agent worked out today becomes a syntheses/ page tomorrow’s query reads in one hop. Route those through the same validator; a filed answer is an ingest whose source is the conversation.

Lint

Lint is the background maintenance pass. Split it in two tiers, because most teams overspend on LLM calls for checks a graph traversal already answers.

def lint_structural() -> Report:                     # every commit, no LLM, seconds
    pages = load_pages()
    g = link_graph(pages)
    return Report(
        orphans=[p for p in pages if g.in_degree(p) == 0 and p not in ROOTS],
        broken=[(a, b) for a, b in g.edges if b not in pages],
        unindexed=set(pages) - indexed_slugs(INDEX),
        orphaned_sources=[p for p in pages if cited_hashes(p) - manifest_hashes()],
        missing_pages=terms_mentioned_without_page(pages, min_pages=3),
        open_conflicts=pages_with_section(pages, "conflicts"),
    )

def lint_semantic(agent, sample: int = 50) -> Report:  # nightly, sampled, bounded cost
    r = Report()
    for cluster in dense_clusters(link_graph(load_pages()), max_size=6)[:sample]:
        r.contradictions += agent.structured(
            prompt=render_cluster(cluster), schema=list[Conflict])
    for p in pages_where(lambda m: m.updated_before(days=90) and newer_sources_cite(m.slug)):
        r.stale_claims += agent.structured(
            prompt=render_staleness(p, newer_sources_for(p)), schema=list[StaleClaim])
    return r

def repair(report: Report, agent) -> None:
    fix_mechanically(report.broken, report.unindexed)    # relink, reindex
    for c in report.contradictions + report.stale_claims:
        add_conflict_block(c)                             # never auto-resolve
    for term in report.missing_pages:
        draft_stub(term, agent)                           # goes through validate()

Run the structural tier as a pre-commit hook or CI check. Schedule the semantic tier nightly with a fixed sample size so its cost stays flat as the wiki grows. Orphans, dangling links and unindexed pages are graph facts; detecting a stale claim is the only part that truly needs a model.

Agent memory mechanics

Agents suffer the no-memory problem far more than chat does. A chat session can tolerate re-retrieval; an agent running for days rediscovers the same facts across dozens of tasks, retries approaches that already failed, and loses every decision when its context is compacted. The LLM Wiki gives that agent memory with an address, a format and a history.

One directory, four kinds of memory

The usual taxonomy of agent memory maps cleanly onto the wiki layout:

Memory type	What it holds	Where it lives	Written by
Semantic	facts, definitions, design decisions	concepts/	ingest of docs, ADRs, papers
Entity	services, modules, teams, vendors, with aliases	entities/	ingest, plus identity resolution
Procedural	how to build, test, deploy; failed approaches and why	playbooks/	agent write-back at session end
Episodic	what happened, when, in which session	log.md	every operation, append-only

The procedural layer is the one most implementations skip, and the one coding agents need most. “The integration tests on billing need the stub acquirer running on port 9091; the docker-compose profile is wrong, use make stub-up” is exactly the knowledge that costs an agent forty minutes to rediscover.

Why Markdown beats an opaque vector space for agent memory

• Deterministic addressing. entities/payment-gateway.md is a path, not a similarity score. Two sessions asking about the gateway read the same page.
• Consolidation instead of accumulation. Vector memory appends: after a month you have twelve near-duplicate memories, some contradicting each other. The wiki rewrites one page, and conflicts are visible.
• Human-editable. An engineer who sees a wrong claim fixes it in the editor. Nobody hand-edits an embedding.
• Reviewable change. Memory updates are diffs, so a bad write is caught in review or reverted, not discovered weeks later as a strange answer.

Coding agents over weeks

For a codebase, keep the wiki in the repository (for example .wiki/) so it versions with the code, and wire a session protocol into the schema:

## session protocol (CLAUDE.md)
start:
  - read .wiki/index.md, then .wiki/entities/<module>.md for every module in the task
  - read .wiki/playbooks/<task-type>.md if it exists
end:
  - update entity pages for modules you changed: new invariants, removed behaviour
  - append dead ends to playbooks/<task-type>.md under "## tried and failed"
  - cite commits as [src:git:<sha>] and PRs as [src:pr:<id>]
  - append one log.md entry: `## [date] session | <ticket> | pages: ...`

The effect is cumulative. Session one learns that the retry wrapper swallows idempotency keys and writes it down. Session nine, run by a different agent on a different ticket, reads it in the module page before touching the code. Citing commit SHAs as sources also gives lint a staleness signal: a page citing code that has since been rewritten is a candidate for review.

Agent-to-agent workflows

When several agents share one wiki, it becomes a blackboard, and blackboards degrade fastest under concurrent writes. Two rules keep it sane:

1. Workers propose, a curator merges. Worker agents write page edits on a branch or as patches into a queue. A single curator agent (or a human) validates and merges, which serializes writes per page and keeps the conflict contract intact.
2. Reads are free, writes are scoped. A research agent may read the whole wiki but write only to syntheses/ and sources/; a coding agent writes only to the entity pages of modules it touched. Encode the scopes in the schema and enforce them in the validator.

Enterprise integration via MCP

A wiki that only one agent on one laptop can read is a personal tool. Wrapping the wiki directory in an MCP server turns it into shared infrastructure: any MCP-capable client (an IDE agent, a Slack bot, a CI job, an incident assistant) can navigate the compiled knowledge with the same primitives the maintaining agent uses.

Server surface

Expose the navigation model, not a generic file system. Resources for stable reads, tools for navigation, and exactly one write path that never touches disk directly.

from mcp.server.fastmcp import FastMCP

mcp = FastMCP("llm-wiki")

@mcp.resource("wiki://index")
def index() -> str:
    """The catalog. Clients should read this before anything else."""
    return INDEX.read_text()

@mcp.resource("wiki://page/{slug}")
def page(slug: str) -> str:
    return read_page_scoped(slug, caller_scopes())     # ACL per directory

@mcp.tool()
def search(query: str, limit: int = 10) -> list[dict]:
    """BM25 over titles, aliases, summaries and headings. Returns slug + one-line summary."""
    return bm25.search(query, limit)

@mcp.tool()
def neighbors(slug: str, depth: int = 1) -> dict:
    """Inbound and outbound [[links]] around a page, with one-line summaries."""
    return link_graph().ego(slug, depth)

@mcp.tool()
def history(slug: str, limit: int = 5) -> list[dict]:
    """Recent commits touching the page: date, source hash, summary."""
    return git_log_for(WIKI / f"{slug}.md", limit)

@mcp.tool()
def propose_edit(slug: str, patch: str, source_ref: str, rationale: str) -> str:
    """Queue a unified diff for the curator. Never writes to the wiki directly."""
    return review_queue.put(slug, patch, source_ref, rationale, caller())

Note what is absent: no write_page, no delete. External clients get read access broadly and write access only through a review queue that feeds the same validator and the same curator as internal ingest.

Three integration patterns

CI pipelines. On every merged PR, a job writes the PR description, linked ticket and diff summary into raw/prs/<id>.md and triggers ingest. Entity pages for touched modules update within minutes of merge. The reverse direction is just as useful: a PR check calls neighbors on the modules a diff touches and comments with the relevant ADRs and known pitfalls before review starts.

Slack bots. The bot is an MCP client with read scopes. It answers from wiki pages and links them, so every answer is one click from its citations. When a human corrects it in thread, the bot calls propose_edit with the message permalink as source_ref, and the correction enters the review queue instead of evaporating in chat history.

Observability and incident tooling. When an alert fires, the incident assistant reads the entity page for the alerting service: owners, upstream dependencies, known failure modes, the last three incidents. After resolution, the postmortem lands in raw/postmortems/ and is ingested, which updates the same entity page. The loop closes: each incident makes the next triage faster.

Governance concerns that are not optional

• Prompt injection through sources. A raw document can contain injected instructions. The ingest agent must treat raw content as data, the validator must reject edits that touch pages outside the plan, and schema changes must never be derivable from source content.
• Access control by directory. Map scopes to paths (wiki/finance/, wiki/hr/) and enforce them in the server, not in the prompt. The index served to a caller should list only the pages that caller may read.
• Audit. Git history answers “who changed this claim, when, from which source”. log.md answers “which questions read this page”. Keep both; regulators and security teams will ask for each.

Implementing it with GitHub Copilot Business

A team on Copilot Business already has every primitive the pattern needs, under different names. Instruction files carry the schema, custom agents carry the roles, prompt files carry the three operations, MCP exposes the wiki, and Copilot CLI runs the batch and scheduled work that an IDE session should not. This section is a full repository blueprint for VS Code agent mode and Copilot CLI, with MCP enabled at the org level.

LLM Wiki concept	Copilot Business artifact	Location
Schema	Repository and path-specific instructions	.github/copilot-instructions.md, .github/instructions/*.instructions.md
Maintainer role	Custom agent with write tools	.github/agents/wiki-curator.agent.md
Reader role	Custom agent, read-only tools	.github/agents/wiki-reader.agent.md
Ingest, query, lint	Prompt files invoked as slash commands	.github/prompts/*.prompt.md
Shared read surface	MCP server registered per workspace	.vscode/mcp.json, ~/.copilot/mcp-config.json
Batch ingest, nightly lint	Copilot CLI in programmatic mode	scripts and GitHub Actions
Validator	Deterministic script, pre-commit and CI	tools/wiki_check.py

Reference topology: in-repo and central wikis

Use both, for different kinds of knowledge. An in-repo .wiki/ holds module memory that must version with the code: invariants, gotchas, procedural notes from agent sessions. A central knowledge repository holds organization knowledge that spans repos: architecture, platform services, ADRs, postmortems, vendor docs.

payments-service/                     # any code repository
├── AGENTS.md                         # short pointer to the wiki schema
├── .wiki/
│   ├── SCHEMA.md                     # layer 3, repo-local rules
│   ├── index.md  log.md
│   ├── entities/  playbooks/  syntheses/
│   └── raw/                          # PR digests, session notes (append-only)
├── .github/
│   ├── copilot-instructions.md
│   ├── instructions/wiki.instructions.md
│   ├── agents/wiki-curator.agent.md  wiki-reader.agent.md
│   ├── prompts/wiki-ingest.prompt.md  wiki-query.prompt.md
│   │          wiki-lint.prompt.md    wiki-session-end.prompt.md
│   └── workflows/wiki-check.yml
├── .vscode/mcp.json
└── tools/wiki_check.py

kb-central/                           # one org-wide knowledge repository
├── SCHEMA.md
├── raw/      architecture/ adrs/ postmortems/ vendors/
├── wiki/     index.md log.md entities/ concepts/ sources/ syntheses/
├── mcp/      server.py               # the server from the MCP section
├── scripts/  ingest_batch.sh
└── .github/  agents/ prompts/ workflows/wiki-nightly.yml

The rule that keeps them from diverging: code repos read the central wiki through MCP and never write to it. Anything a repo learns that belongs centrally goes through propose_edit, and the central curator merges it.

Step 1: org admin prerequisites

These are policy settings in the organization’s Copilot settings, done once by an org owner:

• Enable the MCP servers policy. MCP access is off by default for Business organizations. Turn it on; if your org uses an MCP registry allowlist, add the internal llm-wiki server to it so developers cannot point Copilot at arbitrary servers.
• Enable Copilot CLI for the members or teams who will run batch ingest, if your org gates it.
• Review content exclusions. Exclusions apply to agent reads too. Exclude secrets and regulated data paths, but do not exclude raw/ or the wiki, or the agent will compile from nothing.
• Budget GitHub AI Credits. Ingest with a frontier model burns through AI Credits fast: one source can mean several long calls. Run bulk ingest from a dedicated service seat or a small curator team, and keep queries on the included model where quality allows.

Step 2: the schema as instruction files

Copilot applies .github/copilot-instructions.md to every chat in the repo. Keep it short and point to the schema, so the full wiki rules load only when wiki files are in play.

<!-- .github/copilot-instructions.md -->
This repository keeps an agent-maintained knowledge base in `.wiki/`.
- Before changing code in a module, read `.wiki/index.md` and the module's page in `.wiki/entities/`.
- Org-wide knowledge (architecture, platform services, ADRs) is in the `llm-wiki` MCP server. Read it; never try to write it.
- Never edit files under `.wiki/raw/`. Never edit `.wiki/` pages unless you are the wiki-curator agent.

The full schema rides in a path-specific instruction file, applied only when the agent touches the wiki:

---
applyTo: ".wiki/**"
---
# wiki schema (repo-local)

## invariants
- `.wiki/raw/` is append-only. Never modify or delete a file there.
- Every factual sentence ends with a source tag: [src:<hash>], [src:git:<sha>] or [src:pr:<number>].
- Never resolve a contradiction silently. Record it under `## conflicts` with both sources.
- Never delete a page. Set `status: retired` in its frontmatter.

## page templates
- entities/<module>.md: purpose, owners, invariants, dependencies, known pitfalls, conflicts, related.
- playbooks/<task>.md: steps that work, `## tried and failed` with the reason, last verified commit.

## after every write
Run `python tools/wiki_check.py --staged` and fix every error before finishing.

The wiki_check.py call is the important line. Copilot cannot be trusted to self-police the invariants, so the schema delegates enforcement to the deterministic validator from the lint section and tells the agent to treat its errors as blocking.

Step 3: two custom agents, scoped by tools

The curator writes; the reader cannot. Tool lists in the frontmatter enforce the split, and llm-wiki/* grants every tool of the MCP server.

---
name: wiki-curator
description: Maintains the knowledge base in .wiki/. Ingests sources, updates pages, runs checks. Never edits application code.
tools: ['read', 'search', 'edit', 'execute', 'llm-wiki/*']
handoffs:
  - label: Run lint on what I just changed
    agent: wiki-curator
    prompt: /wiki-lint scope=staged
---
You are the wiki curator. Follow the schema in `.github/instructions/wiki.instructions.md` exactly.

Workflow for every task:
1. Read `.wiki/index.md` first. Plan which pages are affected before opening them.
2. Edit only files under `.wiki/` (never `.wiki/raw/`).
3. Run `python tools/wiki_check.py --staged`. Fix all errors.
4. Append one entry to `.wiki/log.md`.
5. Summarize: pages created, pages updated, conflicts opened.
If a needed fact belongs to org-wide knowledge, call `llm-wiki/propose_edit` instead of writing it here.

---
name: wiki-reader
description: Answers questions from the repo wiki and the org wiki. Read-only. Cites pages.
tools: ['read', 'search', 'llm-wiki/*']
---
Answer from compiled knowledge, not from raw sources or guesses.
1. Read `.wiki/index.md`; for org topics read the `wiki://index` resource from llm-wiki.
2. Open at most 8 pages; follow [[links]] one hop if the answer needs it.
3. Cite every claim as [[page]]. If the wiki does not cover it, say so and name the missing page.
Never call `propose_edit`; suggest that the user run /wiki-ingest instead.

The reader still lists llm-wiki/*, so its read-only guarantee for org knowledge comes from the server, which exposes no write tool except the review queue. That is the right place to enforce it.

Step 4: prompt files for the operations

Each operation becomes a slash command bound to the right agent. Developers type /wiki-ingest, /wiki-query, /wiki-lint or /wiki-session-end in VS Code chat.

---
description: Ingest a source into the repo wiki
agent: wiki-curator
---
Ingest ${input:source:path under .wiki/raw/ or a PR number}.

1. If it is a PR number, write `.wiki/raw/prs/<number>.md` with the title, description, linked issue and a
   file-level summary of the diff. This is the only time you create a file under raw/.
2. Compute its short hash and add it to `.wiki/raw/manifest.jsonl`.
3. Plan: list the entities and concepts it touches, mapped to existing pages or NEW. Show the plan.
4. Write `sources/<hash>.md`, then update or create each page in the plan. Expect 5 to 15 pages.
5. Any existing claim the source contradicts goes under `## conflicts`. Do not edit the old claim.
6. Update index.md, run the checker, append to log.md.

---
description: Record what this session learned into the module wiki
agent: wiki-curator
---
Review the changes in this session (`git diff` against the session's starting commit) and the chat so far.
For each module touched:
- Update its entity page with new or changed invariants, citing commits as [src:git:<sha>].
- Add anything that failed and why under `## tried and failed` in the relevant playbook.
Write nothing that is not supported by the diff or by a command output in this session.

The session-end prompt is what turns VS Code agent mode into long-running memory. Make it a team habit, the same way people write a commit message: close the task, run /wiki-session-end, review the diff, commit code and wiki together.

Step 5: register the wiki MCP servers in VS Code

Commit .vscode/mcp.json so every developer gets the same servers. The local server serves the repo wiki; the central one serves kb-central from an internal host.

{
  "servers": {
    "llm-wiki": {
      "type": "http",
      "url": "https://kb.internal.example.com/mcp"
    },
    "repo-wiki": {
      "type": "stdio",
      "command": "python",
      "args": ["-m", "wiki_mcp", "--root", "${workspaceFolder}/.wiki", "--read-only"]
    }
  }
}

Run the central server behind your SSO proxy and derive the caller’s directory scopes from the authenticated identity, as described in the MCP section. The repo server is read-only on purpose: inside the repo, the curator agent edits files directly and the validator checks them; MCP is for navigation (search, neighbors, history), which is faster and cheaper than having the agent grep markdown.

Step 6: Copilot CLI for batch ingest

Interactive ingest is fine for one PR. Backfilling forty ADRs or a quarter of postmortems belongs in a script. Copilot CLI’s programmatic mode (copilot -p) runs one headless task per source with an explicit tool allowlist, and --agent reuses the same curator definition.

#!/usr/bin/env bash
# kb-central/scripts/ingest_batch.sh: one source per headless run, one commit per source
set -euo pipefail

for src in "$@"; do
  copilot -p "Ingest ${src} following SCHEMA.md. Then run the checker and append to wiki/log.md." \
    --agent wiki-curator \
    --allow-tool 'write' \
    --allow-tool 'shell(python tools/wiki_check.py:*)' \
    --allow-tool 'shell(git add:*)' \
    --deny-tool 'shell(git push)' \
    --deny-tool 'shell(rm:*)'

  python tools/wiki_check.py --staged          # trust, but verify outside the agent
  git commit -m "ingest $(basename "$src")" -- wiki/ raw/manifest.jsonl
done

Three details matter. One source per invocation keeps each run’s context small and each commit revertible. The deny rules win over allow rules, so the agent cannot push or delete even if a prompt in a source tells it to. And the script re-runs the validator itself rather than trusting the agent’s report that it passed.

For the CLI’s own MCP access, register the central server in ~/.copilot/mcp-config.json (or interactively with /mcp add), so batch runs can call search and neighbors to find pages to update instead of scanning the whole index.

Step 7: nightly lint and CI guardrails

Two workflows close the loop. The check runs on every PR in every repo and needs no model at all:

# .github/workflows/wiki-check.yml
name: wiki-check
on: pull_request
jobs:
  check:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with: { fetch-depth: 0 }
      - name: raw sources are append-only
        run: |
          if git diff --name-only --diff-filter=MDR origin/${{ github.base_ref }}... -- .wiki/raw/ | grep .; then
            echo "::error::files under .wiki/raw/ were modified or deleted"; exit 1
          fi
      - name: structural lint
        run: python tools/wiki_check.py --all --fail-on orphans,broken,unindexed,uncited

The semantic lint runs nightly in kb-central through Copilot CLI and opens a pull request instead of committing to main, so a human reviews every conflict the model surfaced:

# kb-central/.github/workflows/wiki-nightly.yml
name: wiki-nightly-lint
on:
  schedule: [{ cron: "0 2 * * 1-5" }]
jobs:
  lint:
    runs-on: ubuntu-latest
    permissions: { contents: write, pull-requests: write }
    steps:
      - uses: actions/checkout@v4
      - run: npm install -g @github/copilot
      - name: semantic lint (sampled)
        env:
          # token of the curator service seat; see GitHub's Copilot CLI docs for the required permission
          COPILOT_GITHUB_TOKEN: ${{ secrets.WIKI_CURATOR_TOKEN }}
        run: |
          git switch -c lint/$(date +%F)
          copilot -p "/wiki-lint scope=sample size=50" --agent wiki-curator \
            --allow-tool 'write' --allow-tool 'shell(python tools/wiki_check.py:*)' \
            --deny-tool 'shell(git push)'
      - name: open PR
        run: |
          git add wiki/ && git commit -m "nightly lint $(date +%F)" || exit 0
          git push -u origin HEAD && gh pr create --fill --label wiki-lint
        env: { GH_TOKEN: ${{ github.token }} }

The push and PR happen in a separate step with the workflow token, outside the agent’s permissions. The agent proposes; the pipeline and a reviewer decide.

Rollout and limits worth knowing

Start with one repository and one curator, not the whole org:

1. Week 1: admin policies, kb-central scaffold, schema v1, ingest 10 to 20 high-value sources by hand with /wiki-ingest and read every diff.
2. Week 2: tighten the schema from what you saw, add wiki_check.py and the PR workflow, enable /wiki-session-end in the pilot repo.
3. Weeks 3 to 4: deploy the central MCP server, batch-ingest the backlog with the CLI script, switch on nightly lint.
4. After that: add repos one at a time, each with its own .wiki/ and the shared .github/ files from a template repository.

A few constraints are specific to Copilot and shape the design:

• No resident background process. Copilot acts when invoked. Anything periodic, such as lint, staleness audits or regenerate-and-diff checks, runs through the CLI in GitHub Actions.
• Edit tools are not path-scoped. A custom agent with edit can technically write anywhere in the workspace. The instructions say “never raw/“, but the guarantee comes from the PR check and the CLI deny rules, not from the prompt.
• Model choice varies by request. Pin the model in the curator agent’s frontmatter for ingest, so a model change is a deliberate schema change and triggers your knowledge regression suite.
• AI Credits are the real cost meter. Track them per curator seat; if the bill grows faster than the wiki’s read traffic, you are ingesting low-value sources.

Trade-offs, scale and drift management

Neither architecture dominates everywhere. They put cost and risk in different places, and the right choice depends on corpus size, change rate and how much synthesis your questions need.

Dimension	Traditional RAG	LLM Wiki
Token efficiency	Cheap per query (top-k chunks, a few thousand tokens) but redundant: overlapping fragments, and cross-source questions need large k or fail. Ingestion costs only embedding.	Expensive ingestion (one source rewrites 10 to 15 pages). Queries read the index plus a handful of pages: more tokens than top-k, far denser in content, and amortized across every later question.
Synthesis quality	Bounded by k and chunking. Contradictions resolved silently and inconsistently per query. No memory of earlier answers.	Synthesis done once, with all affected pages in view. Conflicts explicit and cited. Good answers filed back and reused.
Scale and infrastructure	Scales predictably to millions of documents; retrieval is a search problem. Needs a vector store, an embedding pipeline, often a reranker.	A folder of markdown and git. Navigation by index degrades as the catalog grows; large wikis need hierarchical indexes, then search, then embeddings over the wiki itself.
Maintenance and governance	Near-zero upkeep once indexing runs, but no artifact to review: errors surface only as bad answers. Freshness is immediate.	Needs lint, conflict triage and periodic audits. In exchange every change is a reviewable, revertible diff with source attribution. Freshness lags until re-ingest.

Where the crossover sits

The pattern is at its best while the agent can navigate by reading. In practice that holds while the compiled wiki stays somewhere around 50k to 100k tokens of summaries and index, the range of Karpathy’s own setup of roughly a hundred sources and a few hundred pages. Inside that envelope the agent reads index.md in one pass, selects pages accurately, and often loads most of what matters into context. No retrieval infrastructure is needed at all.

Past it, three things happen together: the index no longer fits comfortably alongside the pages it points to, page selection from a long catalog gets less reliable, and ingest plans miss pages they should have updated, which is how duplicates creep back in. The standard escalation:

1. Hierarchical indexes: index.md points to per-category indexes, and the agent descends one level at a time.
2. Lexical search over titles, aliases and summaries (BM25 is enough for a well-named wiki).
3. Embeddings over wiki pages: at this point you are running RAG, but over compiled, deduplicated pages instead of raw chunks, which is a much better retrieval target.

For enterprise corpora in the millions of documents, do not try to compile everything. Compile the high-value, high-reuse slice (architecture, products, policies, recurring incidents) into the wiki and keep RAG over raw storage for the long tail and for anything that changes faster than you re-ingest. The two layers complement each other; the wiki answers “what do we know”, RAG answers “what does document X say right now”.

Managing knowledge drift

Drift is the failure mode specific to this architecture. Every ingest is a chance for small errors: a qualifier dropped, a number rounded, two entities merged that should not be. Each edit builds on the previous text, so errors compound like a game of telephone, and after a few hundred ingests a page that started accurate can be confidently wrong. Controls that work:

• No claim without a source tag. Enforced by the validator, not requested in the prompt. An uncited sentence is the first symptom of drift.
• Regenerate-and-diff audits. Periodically pick pages (weighted by read traffic from log.md), rebuild each one from its cited raw sources in a fresh context, and compare claims with the live page. Divergence opens a conflict. Think of it as a property test for knowledge.
• Rebuild instead of patch past a revision threshold. When a page’s revisions counter passes a limit (10 to 15 is a reasonable start), regenerate it from sources rather than editing the last edit. This breaks the telephone chain.
• Never auto-resolve conflicts. The agent flags; humans or a dedicated review pass decide. An agent that resolves its own contradictions will eventually resolve them in the wrong direction, quietly.
• Pin the ingest model and keep a knowledge regression suite. A fixed set of questions with known answers and expected citations, rerun after any model or schema change. If the answers move, you changed the compiler.
• Bisect with git. Per-ingest commits let you bisect and find exactly which source introduced a wrong claim and revert that ingest alone.

The LLM Wiki is a bet that knowledge work should look more like software engineering: a source tree, a compiler, a build artifact, tests, review, version control. That framing is also its honest limit. It inherits the costs of a build system along with the benefits, and teams that skip the validator, the lint tiers and the drift audits will end up with a fluent, well-linked, slowly wrong wiki. Teams that keep the discipline get something RAG never offered: a knowledge base that improves with every source, one reviewable diff at a time.

Sources

• Andrej Karpathy, llm-wiki.md idea file (gist, April 2026)
• Overview of the gist and its three layers
• astro-han/karpathy-llm-wiki agent skill

Copilot implementation references:

• Custom agents in VS Code
• About GitHub Copilot CLI
• Using GitHub Copilot CLI