What Is Data Freshness for LLMs? – Definition and How to Manage It
Learn why model knowledge goes stale, the three layers where freshness is decided, how to measure staleness, and how to surface it to users.
Freshness is the gap between when information was true and when it reaches the model. It is decided in three places, and confusing them is why systems that "have retrieval" still answer with stale facts.
1. Three Layers of Staleness
| Layer | Staleness source | Typical age |
|---|---|---|
| Model weights | Training cutoff | Months to years |
| Retrieval index | Last crawl or sync | Hours to weeks |
| Live tools | Request time | Seconds |
An answer is only as fresh as the layer that supplied the fact — and a model will happily answer from weights when retrieval returns nothing.
2. Why Weights Cannot Be Kept Fresh
Training is expensive and periodic. Nothing written after the cutoff exists in the weights, and no instruction changes that.
Worse, the model has no representation of its own boundary, so it does not know which of its answers are stale. That is why "check whether this is current" is not a capability a prompt can invoke.
3. Index Freshness Is a Pipeline Property
An index is as current as its last successful crawl of each document. Common failures:
- Crawl schedule slower than the content changes.
- Failed fetches leaving old versions in place silently.
- Deleted pages remaining retrievable.
- Unchanged-content checks that skip pages which did change.
Recrawl frequency should follow each source's change rate rather than one global schedule.
4. Measuring Staleness
Store fetched_at with every chunk and report the distribution, not the average. What matters is the tail: the oldest document that can still be retrieved for a common query.
Useful checks are the age of the oldest retrievable chunk, the share of the corpus older than a threshold, and the number of documents whose source no longer exists.
5. Routing by Freshness Need
Classify questions before answering them:
- Time-sensitive — prices, availability, current events. Route to live tools.
- Slow-moving — documentation, policies, specifications. Serve from the index.
- Timeless — definitions and concepts. The model's weights are usually sufficient.
Misrouting the first category is the single most common cause of confidently wrong answers.
6. Surface It to the User
Show when the information was collected. A dated answer lets a reader judge it; an undated one implies currency the system cannot guarantee.
Where a question is time-sensitive and only stale sources are available, saying so is a better answer than a confident one built on old text.
7. Practical Measures
- Timestamp everything at collection.
- Set recrawl intervals per source, not globally.
- Remove chunks whose source page is gone.
- Alert on crawl failures — silence is indistinguishable from success otherwise.
- Instruct the model to refuse when retrieved context does not cover the question.
- Display collection dates in answers.
8. Summary
Freshness is set by three layers — weights, index, and live tools — and an answer inherits the staleness of whichever supplied the fact.
Route time-sensitive questions to live retrieval, keep per-source recrawl schedules, monitor the oldest retrievable content rather than the average, and show collection dates so readers can judge for themselves.