Imagine a hypothetical knowledge assistant that answers from company documents. A policy is updated in the source system, but the assistant continues citing the old version for hours. Worse, a deleted document sometimes remains searchable because the indexing pipeline processed its original update but missed the later deletion event.
Our RAG index keeps returning stale or deleted documents. How would you fix freshness?
This is not mainly a vector-search problem. It is a distributed data-consistency problem between the authoritative source, the ingestion pipeline, the search index, and caches.
Why the common answer is incomplete
Most candidates answer: “Re-index documents more frequently.”
Shorter polling intervals can reduce average delay, but they do not guarantee ordering, deletion propagation, or recovery after a failed batch. A fast job can still write an older version after a newer one, and a document that disappears from a source listing may never produce a deletion record. Re-indexing everything also wastes embedding capacity and can make large collections permanently behind.
A production design needs explicit versions, durable change events, tombstones, idempotent writes, and reconciliation against the source of truth.
A production approach
1. Define freshness as a measurable contract
Choose the source of truth and define how quickly each content class must become searchable after a change. A public handbook may tolerate several minutes, while access revocation or legal-policy deletion may require a much shorter boundary. Measure source commit time, ingestion time, index visibility time, and cache visibility time separately. “The job ran” does not prove the new version is queryable.
2. Give every source object a stable identity and version
Store a source-system identifier, tenant, content version, updated timestamp, and deletion state with every indexed chunk. Derive chunk identities from the document identity and a deterministic chunk key rather than random IDs. Version metadata lets the index reject stale writes and lets investigators trace a returned passage back to the exact source revision that produced it.
3. Capture changes durably
Prefer a source event stream, change-data capture, or an outbox over periodic full scans when the source supports it. Persist the change before acknowledging ingestion so worker crashes do not lose it. Include create, update, permission change, and delete events. When only polling is available, use a durable cursor and overlapping windows so clock skew or a failed page does not create a permanent gap.
4. Make the indexing pipeline idempotent and ordered
Each stage should accept the same event more than once without creating duplicate chunks. Before writing, compare the incoming version with the latest applied version for that document. Ignore an older version that arrives late. Write new chunks under the new version, then atomically switch the searchable document pointer or filter so readers never observe a partially replaced document.
5. Propagate deletions as tombstones
Treat deletion as data, not as the absence of data. Write a versioned tombstone that removes or filters all chunks for the document and invalidates related caches. Retain enough tombstone history to defeat delayed update events. For security-sensitive removals, add a query-time denylist or source authorization check so stale index entries cannot become an access-control bypass while asynchronous deletion finishes.
6. Separate content freshness from embedding work
Metadata, permissions, and deletion changes may not require recomputing embeddings. Update filters and visibility state independently when possible. For content updates, compute only changed chunks and reuse unchanged embeddings with content hashes. This reduces backlog and allows urgent revocations to move ahead of expensive enrichment work.
7. Make query responses version-aware
Return source version and last-indexed time with retrieved candidates. Enforce tenant and current permissions at query time. For workflows with strict freshness, compare the index version with a lightweight source metadata lookup before using the passage. If the index is known to be behind, decline to answer from stale evidence or retrieve directly from the authoritative source rather than presenting an old policy as current.
8. Reconcile continuously
Run a periodic comparison between source inventory and index inventory. Detect missing documents, orphaned chunks, version mismatches, failed tombstones, and permission drift. Reconciliation is the repair path for lost events and software defects; it should produce targeted repair jobs rather than rebuilding every document. Test the process by intentionally dropping and reordering events in a non-production environment.
Engineering trade-offs
Event-driven ingestion improves typical freshness but adds brokers, cursors, retry state, and ordering concerns. Full scans are simpler and useful for reconciliation, while becoming expensive at scale. Atomic version swaps require additional storage during an update. Query-time source checks improve freshness guarantees but add latency and dependency on the source system.
The correct design depends on the consequence of stale data. A recommendation article can tolerate a different window from an employee termination, document deletion, or permission revocation. Security boundaries should not rely solely on eventual cleanup of a search index.
Common mistakes
- Using source timestamps as ordering guarantees across systems with different clocks.
- Treating a missing source row as if it automatically creates an index deletion.
- Deleting only the document record while leaving child chunks searchable.
- Allowing a delayed worker to overwrite a newer document version.
- Re-embedding unchanged content when only permissions changed.
- Reporting pipeline completion without measuring search visibility.
- Rebuilding the entire index as the only repair mechanism.
Failure handling and observability
Track event age, queue age, processing attempts, source version, applied version, chunk counts, embedding failures, tombstone delay, and index visibility probes. Monitor freshness percentiles by source and tenant, not only an average. Alert on reconciliation mismatches, late-event rejection, growing dead-letter queues, orphaned chunks, and permission changes that exceed their security deadline.
If the embedding service fails, preserve the durable change event and retry within a bounded policy while older content remains clearly versioned. If deletion propagation fails, add the resource to a fast denylist and block it at retrieval until cleanup succeeds. If the source API is unavailable, do not advance the polling cursor. Recovery must resume from the last durable position and remain safe when events replay.
Interview insight
The interviewer is not testing whether you can schedule an embedding job every five minutes. They are testing whether you recognize index freshness as replicated-state management. A staff-level answer defines the consistency contract, versions every change, represents deletion explicitly, and builds reconciliation for the failures that asynchronous pipelines will eventually experience.