RAG Returns Outdated Documents: Effective Dates and Ranking

Three mechanisms stack, and only the first is obvious.

Near-duplicate ranking is arbitrary. A revision changes a minority of tokens in a minority of chunks. For every unchanged chunk the two versions are separated by embedding noise, and the winner is effectively random per query.

Revisions dilute. Amendments add qualifiers, exceptions and cross-references. "Employees may expense meals up to $75 per day" is a tighter match for "what is the meal expense limit" than the replacement clause that wraps the same number in three sentences of conditions. The retired chunk is not merely tied with its successor. It can be genuinely more similar to the question.

Hybrid search does not rescue you. Reciprocal rank fusion, the hybrid-search default with its familiar k of 60 from the 2009 paper that introduced it, fuses ranks and is blind to dates. A superseded chunk that ranks well in both the dense and the sparse list gets promoted by fusion, not demoted. Same for reranking, which improves ordering but not currency: a cross-encoder asked to judge relevance to "what is the meal expense limit" will happily put the 2023 clause first, because it is relevant. It is just not in force.

Worth noting before the objection arrives: version awareness is not the expensive option. The published indexing token cost for the version-aware system was 97 percent below the graph-based comparison. That system does build a structure over versions, but modelling lineage is far cheaper than extracting an entity graph, and the schema-and-filter approach in this post sits at the cheaper end of the same family.

Our post on how to update a RAG knowledge base without rebuilding everything, under the heading "Version Control for RAG Knowledge Bases", says:

Store both versions in your vector database initially, tagged with version metadata. Keep the current version active in production while maintaining one or two previous versions for rollback.

That is wrong for regulated content, and the reasoning error is worth naming because it is everywhere. It conflates two different problems. Rollback safety asks whether you can get the previous version back if the new one turns out to be bad. Query safety asks whether a retired clause can reach a user. Rollback safety wants predecessors addressable. Query safety wants them unreachable from the default query path. Keeping predecessors live in the production namespace for a 7 to 14 day validation window solves the first by breaking the second, and it breaks it during precisely the window when the new version is most likely to be queried.

The amended rule: prior versions move to an archive collection at the moment the successor becomes effective, not after a soak period. They stay addressable by doc_family_id and version_ordinal, so rollback is a pointer flip rather than a reindex, and they stay out of the collection production queries hit.

The same post's staging pattern, where old vectors stay active while new ones are inserted and then get deleted in one batch, is defensible because the window is seconds rather than days. One qualification: the staging flag has to be enforced in the query filter, not merely present in metadata. A flag that nothing reads is a comment.

Teams reach for deletion first and discover it is not on the menu.

In financial services, SEC Rule 17a-4 requires preservation of one class of records for not less than six years with the first two in an easily accessible place, and another class for not less than three years on the same terms. Paragraph (f) lets a firm either preserve records exclusively in a non-rewriteable, non-erasable format or elect the audit-trail alternative, which requires a complete time-stamped audit trail of all modifications and deletions. That alternative has only been electable since January 3, 2023, with a May 3, 2023 compliance date for broker-dealers, which is recent enough that plenty of production indexes were designed under a WORM-only assumption.

In healthcare, the HIPAA Security Rule documentation standard at 45 CFR 164.316(b)(2)(i) requires six years from the date of creation or the date when the document last was in effect, whichever is later. The superseded version therefore has a longer mandated life than the one that replaced it.

For device manufacturers the ground moved this year. The Quality Management System Regulation took effect on February 2, 2026 and rebuilt 21 CFR Part 820 around an incorporated international quality standard. Clause 4.2.4 of that standard states the contradiction outright: retain at least one copy of every obsolete document for a defined period covering the device lifetime, and ensure obsolete documents are not unintentionally used. That is this entire post compressed into one clause.

Pulling the other way, GDPR Article 5(1)(e) storage limitation says personal data may be kept in identifiable form no longer than necessary. So "just delete the old version" fails in both directions, and the resolution is tiering rather than deletion. If you do need genuine erasure inside that archive, the mechanics are their own problem: see what DELETE actually does in a vector database.

This is also where deployment shape decides cost. An archive collection is cheap and boring when it sits on infrastructure you already control, next to the storage that carries the WORM or audit-trail obligation anyway: the same engine, a second collection, a stricter access policy and its own retention clock. On a managed multi-tenant index the same tier is a second billable index, a second data-residency conversation, and a support ticket every time legal asks what is in it.

Effective dates belong on the chunk, not only on a document row in a registry, because retrieval filters chunks. Eight fields carry the whole design.

Two rules that get violated constantly. First, effective_from is not ingested_at. A policy approved on March 3 and effective April 1 has an effective_from of April 1, and indexing it on March 3 with the ingestion timestamp makes it current four weeks early. Second, write effective_to explicitly as a far-future sentinel rather than leaving it null or absent, because an absent key does not match a range condition in most stores and a null needs special-case SQL.

Store capability checked 2026-08-13. Two notes on that last row, because it is the one that ships broken. pgvector 0.8.0, released in October 2024, added iterative index scans specifically for this, and you have to turn them on:

SET hnsw.ef_search = 100;
SET hnsw.iterative_scan = strict_order;

SELECT chunk_id, content
FROM   chunks
WHERE  effective_from <= $1
  AND  effective_to   >  $1
  AND  status = 'effective'
ORDER  BY embedding <=> $2
LIMIT  10;

On Pinecone the same interval is numeric and top-level-conjoined:

{"$and": [
  {"effective_from": {"$lte": 1786665600}},
  {"effective_to":   {"$gt":  1786665600}},
  {"status":         {"$eq":  "effective"}}
]}

The generic hazard here is selectivity, not syntax. A narrow effective-date predicate over an HNSW graph is exactly the pattern that quietly returns three results instead of ten, which we covered in why metadata filters collapse vector search recall. Budget for it: raise ef_search, enable iterative scans, or partition by effective period so the filter is a collection choice instead of a predicate.

The schema only helps if something maintains it. Four checks, in the order they earn their keep.

Family reconciliation. For every doc_family_id in the index, ask the system of record which version is currently effective and compare. This catches the common failure where ingestion succeeded and the supersession update did not. Run it nightly; it is a cheap set difference.

Hash-chain drift. Recompute source_hash against the source and alert on mismatch. A changed hash with an unchanged version_ordinal means someone edited a document in place without cutting a version, which is both an index bug and a document-control finding.

Orphan detection. Flag any chunk whose status is effective while a sibling in the same family carries a later effective_from. This is the single highest-value query in the set, because it finds the exact condition that produces the failure: two live versions in one family.

The source that never emits a delete. Plenty of document systems overwrite in place or archive silently, so nothing ever tells you a version retired. That is a trigger-topology problem, and the answer depends on whether you are on webhooks, change data capture, polling or TTL, covered in RAG update triggers. On a polling source, treat any family whose reconciliation has not run inside its TTL as suspect and drop it from the default path rather than serving it.

The parallel failure mode is worth knowing about, since it has the same shape and the same root cause: an index that lags its system of record. Stale permissions produce revoked access that still returns documents.

Order matters, and the order is: hard filter, then prior, then reranker.

Hard effective-date filter. This is the only stage that guarantees anything. Every query carries an as-of timestamp, defaulting to now, and retrieval is constrained to chunks whose interval contains it. Do not make this optional and do not make it a boost. A boost is a suggestion.

Recency prior. Even after filtering you can have several versions legitimately in force across jurisdictions, product lines or entities. A half-life decay over effective_from sorts these sensibly. A 2025 preprint isolates the freshness case on a hard NVD CVE test constructed so the newest relevant item is not the most similar one: a cosine-only retriever scored 0.00 on Latest@10, sorting semantic hits by date reached 0.20, and a half-life recency prior reached 0.60 (arXiv:2509.19376). The authors describe the result as partial and parameter-sensitive, and it is a single-author preprint, so treat the direction as the finding rather than the decimals. The direction is unambiguous: a prior helps a lot and still leaves 40 percent on the table, which is why it sits behind the filter rather than in place of it.

Reranker last. A cross-encoder resolves ordering within a set that is already current. It does not resolve currency, because "in force" is not a property visible in the passage text.

Split the store. docs_current holds exactly one version per family, the effective one, and serves every default query. docs_archive holds every predecessor, indexed the same way, reachable only when a caller passes an explicit as-of timestamp or an audit flag.

The reason to make this a collection boundary rather than a metadata flag is failure behaviour. A forgotten filter on a shared collection returns a superseded clause and looks like a correct answer. A service with no credentials for the archive collection returns nothing and looks like a bug, which someone fixes. Build the version that fails loudly.

Make the as-of timestamp an explicit retrieval parameter throughout, defaulting to now but always present in the request and always logged with the result. "Which version was in force when this answer was generated" is a question auditors ask, and it is unanswerable if the timestamp was implicit.

Recall is the wrong metric here. Recall counts a retired chunk from the right document family as a hit, which is exactly the failure you are trying to catch. Measure Latest@k: the fraction of eval queries where the currently effective chunk appears in the top k.

Build the eval set from superseded pairs. Take 50 to 100 families that have been revised at least once, write the question a user would actually ask about the changed clause, and record the correct answer under both versions. Include implicit changes, where a threshold moved without the document announcing it, since that is where the published baselines collapsed to single digits. Then wire the check into the same pipeline that triggers ingestion, so a regression fails the run rather than a user, and gate releases on it.

One superseded pair that slips through a release will reproduce in production within days, and in a regulated context that is not a wrong answer. It is a retired control being cited as current, with a timestamp and a log line proving your system said it.

For the wider architecture this sits inside, see our RAG systems pillar. If you are building this against a corpus that cannot leave your network, effective-dating, the archive tier and the audit trail are all cheaper inside your own boundary than across a managed index, and that is the deployment shape we build on.