Knowledge Layer Architecture: Storage, Retrieval, Updates
The short answer
A knowledge layer has four parts: ingestion, storage, retrieval and freshness. Design each around one rule: every answer must trace back to a specific version of a specific source that the asking user is allowed to see. This page is the architecture spoke of our guide to choosing an AI knowledge layer.
| Layer | Design decision | Default that works |
|---|---|---|
| Ingestion | Parse, chunk, attach metadata | Structure-aware chunks of 300–800 tokens; keep headings and page numbers |
| Storage | Where text, vectors and metadata live | PostgreSQL + pgvector until scale forces a dedicated engine |
| Retrieval | How candidates are found and ranked | Hybrid search → permission filter → rerank top 30–50 to 5–10 |
| Freshness | When each source is updated | Event-driven for operational data; scheduled incremental for documents |
| Audit | What was retrieved per answer | Log query, chunk IDs, versions and scores for every answer |
Storage design
Store each chunk as one record with:
- Text and its embedding (record the embedding model name and version — you will change it).
- Provenance: source URL or path, document ID, version or hash, owner, ingestion time.
- Access control: the groups or tenant that may read it, copied from the source system.
- Status: active or superseded, so old versions stop competing in retrieval without being lost.
Choosing the store. pgvector keeps vectors in PostgreSQL, so metadata filters, permissions and joins use one database and one backup. Move to a dedicated engine when you need native hybrid search at large scale, multi-vector representations or very high query volume: Qdrant (v1.20.2, November 16, 2026), Weaviate (v1.40.6, November 19, 2026) and Milvus (v3.0.6, November 11, 2026) all support hybrid queries. Compare them in best vector databases.
Retrieval design
- Hybrid search. Combine dense vectors (meaning) with keyword or sparse search (exact terms: product codes, error messages, names). Pure vector search misses exact identifiers.
- Filter by permissions at query time, using the requesting user’s identity. Filtering at indexing time breaks as soon as someone’s access changes.
- Retrieve wide, then rerank. Pull 30–50 candidates and let a reranker choose the final 5–10; see best reranker models.
- Prefer the newest active version when two chunks say different things, and pass the version date to the model so it can say “as of”.
- Return citations: chunk IDs and source links travel with the answer.
Update frequency by source
Set freshness per source, from how fast it changes and what a stale answer costs:
| Source | Change rate | Update method | Target lag |
|---|---|---|---|
| Support tickets, CRM records, incidents | Constant | Change events or webhooks (CDC) | Minutes |
| Product documentation, runbooks | Daily–weekly | Incremental sync on modified-since | Hourly to daily |
| Internal wiki, shared drives | Daily | Incremental sync | Daily |
| Policies, contracts, manuals | Monthly or on release | Triggered on publish; supersede old version | Same day as release |
| Code repositories | Per commit | Re-index changed files on merge | Per merge |
Re-embed only what changed (compare content hashes). Schedule a full rebuild only when you change the embedding model or chunking strategy, and run it side by side before switching traffic.
Prove it works before you scale
Collect 50–100 real questions with known answers and re-run them after every chunking, embedding, reranker or freshness change. Track retrieval hit rate separately from answer quality, so you can tell a retrieval failure from a model failure. Chunking choices are compared in best chunking libraries for RAG; the step-by-step build order is in how to build an agent knowledge layer.
Last verified: November 20, 2026.