Backfilling: Why Your History Can Mislead a Live Model
By Adam Montgomery
-
September 2026
The hidden risk in your history
Backfilling
The most common way a model looks great in testing and quietly underperforms in production, and how Churney keeps it out of your predictions.
What backfilling is
Data arrives with event timestamps that belong to an earlier period: your database has changed its own history. The past you see today is not the past your systems saw back then.
01 · Additions
New rows inserted with old timestamps: late-arriving payments, delayed logs.
02 · Deletions
Rows removed after the fact: refunds, fraud reversals, GDPR erasure.
03 · Mutations
Existing rows updated: a plan upgrade, a corrected value.
04 · Combinations
Any mix of the three, happening together.
Why it's a risk
The core problem is temporal leakage. A model trained on today's record learns from a version of the past more complete than any real-time system could have seen at those moments.
The trap
It looks better on history than it is live.
How Churney prevents it
It rarely looks like a mistake: payments settle late, refunds post over the weekend, updates land in a nightly batch. We plan for it in three ways.
Detect
We compare successive snapshots of your data to measure how much the record changes after the fact, and exactly when.
Time-shift
Where history settles uniformly, we let the model stand in the past, using only the data available at prediction time.
Align
We match our sync and prediction schedule to your update cadence, and widen safety margins to capture late data.
