Forecast plausibility checks (FPC)¶
This section documents forecast plausibility check utilities provided by eb-evaluation.
FPC utilities evaluate whether forecast outputs fall within reasonable and operationally plausible bounds.
eb_evaluation.diagnostics.fpc
¶
Forecast Primitive Compatibility (FPC) diagnostics.
This module defines a derived, auditable classification that diagnoses whether a given forecast primitive (typically point forecasts with scale-based adjustment) is structurally compatible with the observed demand process.
FPC is not a performance metric and is not intended as an optimization objective. It consumes FRF diagnostics (e.g., NSL, UD, HR@τ, CWSL response) and produces a small set of interpretable compatibility states used for governance and policy gating.
Design goals: - Deterministic and auditable (explicit signals in / signals out) - Metric-agnostic (does not require a specific modeling approach) - Lightweight dependencies (NumPy optional; Pandas optional)
FPCClass
¶
Bases: str, Enum
Compatibility taxonomy for forecast primitives.
FPCSignals
dataclass
¶
Observable signals used to classify forecast primitive compatibility.
All fields are intended to be computed from evaluation data and are interpretable on their own. None of these values should be "fitted".
Notes: - nsl_ are fractions in [0, 1] - hr_ are fractions in [0, 1] - ud is a magnitude in units of y (same units as actual demand) - cwsl_* are dimensionless ratios (normalized by demand)
FPCThresholds
dataclass
¶
Thresholds for FPC classification.
These are intentionally simple and interpretable. You can tune these to your operational context, but treat them as governance parameters (not a model).
FPCResult
dataclass
¶
Result container for classification + rationale.
classify_fpc(signals, thresholds=None)
¶
Classify forecast primitive compatibility from observable signals.
Strategy: - INCOMPATIBLE when we see a strong mismatch signature (very low coverage AND tiny response), reinforced by low tolerance hit-rate and/or high UD. - COMPATIBLE when coverage is meaningfully above zero AND responds to RAL in a material way. - Otherwise MARGINAL.