The uncomfortable truth about retrieval systems is that launch-day accuracy tells you nothing about month-six accuracy. Documents change, policies update, product specs shift — and unless someone designed for that, the index keeps serving the version it indexed at launch, with exactly the same confident tone it used on day one. A stale answer delivered with total confidence is worse than no answer at all, because the user has no signal to doubt it.
Where staleness creeps in
- Source documents get updated in place but the index isn't resynced on any schedule, so the retrieval layer serves the pre-update version indefinitely.
- Deprecated documents never get removed — old policy PDFs sit in the same folder as their replacements, and the system has no way to know which one is authoritative.
- Nobody owns the sources — when no person is accountable for a given document category, updates happen inconsistently and the knowledge base drifts silently out of sync with reality.
What keeps a knowledge base honest
- Scheduled resync — an automated pipeline that re-indexes on a cadence matched to how often the source actually changes, not a one-time launch job.
- Visible provenance — every answer surfaces which document and version it came from, with a 'last verified' date, so a user can judge freshness themselves.
- Owned sources — each document category has a named owner responsible for flagging updates and removals, not an assumption that content maintains itself.
Design the resync pipeline in the same sprint as the initial index, not as a follow-up ticket. 'We'll add freshness later' is how six-month-old policy documents end up answering compliance questions with complete confidence.
Why this matters more than it sounds
For anything touching policy, pricing, or compliance, a confidently wrong answer isn't a minor UX issue — it's the kind of thing that shows up in a customer complaint or an audit finding months later, traced back to a document nobody remembered to update.