Dmitry
58 lines
3.3 KiB
Markdown
58 lines
3.3 KiB
Markdown
# LLM Wiki (idea)
|
|
|
|
Captured: 2026-04-12
|
|
|
|
---
|
|
|
|
LLM Wiki
|
|
|
|
A pattern for building personal knowledge bases using LLMs.
|
|
|
|
The core idea
|
|
|
|
Most people's experience with LLMs and documents looks like RAG: you upload a collection of files, the LLM retrieves relevant chunks at query time, and generates an answer. This works, but the LLM is rediscovering knowledge from scratch on every question. There's no accumulation. Ask a subtle question that requires synthesizing five documents, and the LLM has to find and piece together the relevant fragments every time. Nothing is built up. NotebookLM, ChatGPT file uploads, and most RAG systems work this way.
|
|
|
|
The idea here is different. Instead of just retrieving from raw documents at query time, the LLM incrementally builds and maintains a persistent wiki — a structured, interlinked collection of markdown files that sits between you and the raw sources. When you add a new source, the LLM doesn't just index it for later retrieval. It reads it, extracts the key information, and integrates it into the existing wiki — updating entity pages, revising topic summaries, noting where new data contradicts old claims, strengthening or challenging the evolving synthesis. The knowledge is compiled once and then kept current, not re-derived on every query.
|
|
|
|
This is the key difference: the wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you've read. The wiki keeps getting richer with every source you add and every question you ask.
|
|
|
|
You never (or rarely) write the wiki yourself — the LLM writes and maintains all of it. You're in charge of sourcing, exploration, and asking the right questions. The LLM does all the grunt work — the summarizing, cross-referencing, filing, and bookkeeping that makes a knowledge base actually useful over time.
|
|
|
|
Architecture
|
|
|
|
There are three layers:
|
|
|
|
Raw sources — your curated collection of source documents. Articles, papers, images, data files. These are immutable — the LLM reads from them but never modifies them. This is your source of truth.
|
|
|
|
The wiki — a directory of LLM-generated markdown files. Summaries, entity pages, concept pages, comparisons, an overview, a synthesis. The LLM owns this layer entirely.
|
|
|
|
The schema — a document that tells the LLM how the wiki is structured, what the conventions are, and what workflows to follow when ingesting sources, answering questions, or maintaining the wiki.
|
|
|
|
Operations
|
|
|
|
Ingest: read source → write a summary page → update index → update relevant entity/concept pages → append to log.
|
|
|
|
Query: answer questions with citations; optionally file answers back into the wiki.
|
|
|
|
Lint: find contradictions, stale claims, orphans, missing cross-references, and suggest new questions/sources.
|
|
|
|
Indexing and logging
|
|
|
|
Two special files help navigation:
|
|
|
|
index.md is content-oriented: catalog of everything in the wiki.
|
|
|
|
log.md is chronological: append-only record of ingests, queries, lint passes.
|
|
|
|
Optional tools
|
|
|
|
As scale grows, add local search (e.g., qmd) and/or scripts.
|
|
|
|
Why this works
|
|
|
|
Humans abandon wikis because maintenance burden grows. LLMs can touch many files cheaply, keeping cross-references current.
|
|
|
|
Note
|
|
|
|
Abstract pattern; implementation depends on domain and preferences.
|