What is the difference between agent memory and documentation?

Agent memory plugins mine session transcripts into snippets, embed them in a vector database and inject the top-k most similar snippets into every prompt. Document-based memory instead gives the agent a structured Markdown workspace — specs, decisions, indexes, research — that it reads before working and updates afterwards. Memory retrieves by embedding similarity; documentation is organised by meaning and kept current by the agent itself, so it is auditable and shareable in git.

Why do RAG-based agent memory systems fail?

Five recurring problems, as laid out in Kevin Liao's October 3, 2026 essay: similarity search ranks closeness in embedding space, not correctness or currency; snippets are stored without the context that made them true; the past is treated as truth even though the codebase changes daily; the agent cannot search for what it does not know it needs; and a store of thousands of embeddings is unauditable, so stale or wrong memories silently shape behaviour.

Is a Markdown workspace better than Mem0, Zep or Letta?

For a single codebase worked on by coding agents, usually yes: it needs no embeddings, no background daemons, and every fact is a file you can read and commit. For memory about many users across a product, or for facts that arrive from conversations rather than from work the agent did, vector-backed systems like Mem0, Zep and Letta still fit better because nobody is maintaining a document per user. Many teams run both: documents for project knowledge, a memory store for user preferences.

How do I set up document-based memory for a coding agent?

Create a brain directory (for example internal/ or docs/agent/) with an index file, then instruct the agent in AGENTS.md or CLAUDE.md to read the index and the relevant documents before starting any task and to update or add documents after finishing while the full context is still loaded. Keep documents short and typed — instructions, specs, decisions, research — and commit them with the code. Operator Memory (open source, github.com/aerovato/operator-memory) packages this loop as a plugin.

Quick Answer

Agent Memory vs Documentation: RAG or Markdown? (2026)

Published: October 4, 2026

The short answer

For project knowledge — where a feature lives, why it was built, what was agreed — a documented Markdown workspace the agent reads before working and updates after beats a vector store of transcript snippets. Similarity search surfaces what is close, not what is correct or current, and it cannot be audited. Vector memory still earns its place for per-user facts that arrive from conversation rather than from work. Kevin Liao’s October 3, 2026 essay “Agents Don’t Need Memory. They Need Documentation.” put this argument on the Hacker News front page (57 points, 42 comments); this page lays out both sides. Facts verified October 4, 2026.

Two architectures

	Vector-store memory (Mem0, Zep, Letta, most “memory” plugins)	Document-based memory (Operator Memory, hand-rolled `internal/`)
Source of truth	Snippets extracted from past transcripts	Documents the agent writes and maintains
Retrieval	Top-k by embedding similarity on every prompt, plus a search tool	Agent reads an index, then the relevant files
Organisation	Flat (or tiered short/long-term) embeddings	Typed docs: instructions, specs, decisions, research, indexes
Freshness	Background daemons: dedupe, merge, “dreaming” rewrites	Updated by the agent at the end of each task
Auditability	Thousands of rows in SQLite/pgvector	`git diff`
Sharing	Per-installation store	Commit it; teammates and other agents read the same files
Token cost	Injection on every prompt + daemon runs	Reads only what the index points to
Best at	Facts about people learned in conversation	Facts about the project learned by working on it

The case against recall

Liao’s argument is that every memory plugin shares one pipeline — mine transcripts → generate snippets → embed → inject top-5 → expose a search tool — and that the fancier features (multi-tier memory, overnight “dreamers”, rerankers, continuous compression) are patches on the same flawed assumption: the agent forgets, so capture more and retrieve smarter. The failure modes follow directly:

Similarity is not relevance. Five snippets about authentication from different months rank by closeness to the prompt, not by which one is still true.
Snippets lose context. A 200-token memory drops the motivation, the constraints and the environment that made the decision sensible.
The past is treated as truth in a codebase that changes every day.
Agents cannot search for what they do not know. A search tool only helps if the agent already suspects something exists.
The store is unauditable. Which of 10,000 embeddings are stale, never retrieved, or quietly wrong?

The alternative is how humans already work: nobody rewatches a meeting from three years ago; they read the doc. AGENTS.md files were the first step — they exist because people knew agents needed context — but one file is not a brain. Liao’s loop replaces prompt → build → forget with prompt → consult → build → update: before working, read the index and the relevant documents; after working, fix what is now stale and add what is missing, while the whole task is still in context.

The case for vector memory (the part the essay skips)

The essay is right about coding agents on a single project and wrong if generalised to everything:

Per-user memory at scale. A support or companion product with 100,000 users cannot maintain a curated document per user. Facts like “prefers terse answers” and “is on the Pro plan” arrive in conversation, not from work the agent did, and a memory store with extraction and dedupe is the only practical place for them. This is the workload Mem0, Zep and Letta are benchmarked on (compared here).
Unstructured history. When the question is “what did the customer say in March,” transcript search is the right tool, not a document someone had to anticipate.
The update step is the weak link. Document memory depends on the agent reliably revising docs after every task. Skip it a few times and the brain is as stale as any vector store — just more confidently so, because it reads as authoritative.
Documents rot differently, not never. A spec that no longer matches the code is a stale memory with better formatting. Liao’s loop mitigates this by making updates part of every task; it does not eliminate it.

The honest synthesis: documents for the project, a memory store for the people, and never inject either on every prompt without a reason.

Setting up document-based memory today

Create the brain. internal/ (Liao’s original) or docs/agent/, with INDEX.md listing every document and one line on when to read it.
Type the documents. Instructions (how review works here), specs (what was agreed with the user), decisions (what was chosen and why), research (notes on a foreign API), indexes (where things live). Short files beat long ones; the agent reads by file.
Wire the loop into AGENTS.md / CLAUDE.md. “Before any task, read INDEX.md and the documents it points to for this area. After the task, update stale documents and add missing ones.” The after-step is the one teams forget.
Commit it. The brain lives in the repo. Reviewers see memory changes in the same PR as the code they describe.
Or install the plugin. Operator Memory (free, open source) packages exactly this: no vector database, no embeddings, no summarisers or daemons, plain Markdown you can read and share.

If you also need per-user memory, add Mem0 or Zep beside the documents rather than instead of them, and scope what each one is allowed to answer.

Last verified: October 4, 2026.

The short answer

Two architectures

The case against recall

The case for vector memory (the part the essay skips)

Setting up document-based memory today

Sources