Freeze the Prediction Moment

In this chapter

• Define a target with units and an observation horizon.
• Exclude future information from input features.
• Build a chronological split that records unavailable outcomes.

Ask what was known at the time

Repair Forecast helps a coordinator plan equipment turnaround when a request opens. The target might be elapsed working hours until closure, or a binary flag for exceeding eight working hours. These are different learning problems and need a precise calendar definition. A weekend-inclusive duration is not interchangeable with staffed working time. Write the target definition before inspecting which version gives the best-looking score.

A feature is eligible only if it could have been obtained at that prediction moment. Queue size at opening and recorded equipment age are plausible inputs. Final repair cost, completion notes and parts actually consumed are future facts. A SQL join can leak those facts even when its column name looks harmless: joining the latest equipment record may import an inspection performed after the request. Use point-in-time records and document the availability timestamp for each feature.

Make exclusions visible

The miniature dataset uses UTC timestamps and already-computed synthetic target hours. Requests opened after the cutoff form a later evaluation period. A pre-cutoff request whose outcome was not yet known is reported separately rather than used for training. The example is a split illustration, not a statistically adequate dataset.

from datetime import datetime

def at(value):
    return datetime.fromisoformat(value.replace("Z", "+00:00"))

rows = [
    {"id": "r1", "opened": "2026-01-02T09:00:00Z", "closed": "2026-01-04T09:00:00Z", "hours": 16},
    {"id": "r2", "opened": "2026-01-10T09:00:00Z", "closed": "2026-01-20T09:00:00Z", "hours": 80},
    {"id": "r3", "opened": "2026-01-16T09:00:00Z", "closed": "2026-01-17T09:00:00Z", "hours": 9},
    {"id": "r4", "opened": "2026-01-18T09:00:00Z", "closed": "2026-01-19T09:00:00Z", "hours": 6},
]
cutoff = at("2026-01-15T00:00:00Z")
train = [r for r in rows if at(r["opened"]) < cutoff and at(r["closed"]) < cutoff]
later = [r for r in rows if at(r["opened"]) >= cutoff]
pending = [r for r in rows if at(r["opened"]) < cutoff <= at(r["closed"])]
assert [r["id"] for r in train] == ["r1"]
assert [r["id"] for r in pending] == ["r2"]
assert len(later) == 2
print({"train": len(train), "later": len(later), "pending": len(pending)})

Recognize two subtle evaluation traps

The failure mode is a random split that shares future conditions or repeated equipment histories across training and evaluation. A second trap is fitting a scaler, vocabulary or feature selector on the complete dataset before splitting. Those transformations learn from data too. Fit them on the training period and apply the frozen transformation to later records. If repeated equipment units require group isolation, combine that constraint with the time boundary rather than assuming one split solves both.

Pending outcomes deserve attention. Dropping every slow unresolved repair can bias the dataset toward easy cases. Use an earlier label-mature training window, report exclusions and decide whether a time-to-event method is needed in a later extension. Reserve a final test period until model and threshold choices are finished. Temporal evaluation estimates one particular deployment scenario; it does not eliminate changes in suppliers, workload or recording practices.

Create a feature-availability ledger

• List six candidate fields with their units, source and first availability time.
• Mark final cost and completion notes as forbidden at request opening.
• Add a future validation cutoff and a final test period; record all excluded or pending requests.

Expected checks

• Every allowed feature exists at prediction time.
• Training transformations see only training records.
• Pending counts and group overlap are visible in the split report.

Check your understanding

Which field is an appropriate candidate feature for a prediction made when a request opens?

• The final repair cost.
• The queue size recorded at opening.
• The technician's completion summary.

Official tools & further reading

• scikit-learn cross-validation and temporal splitting