Skip to content

Panel evaluation

This section documents evaluation utilities for panel-style data.

Panel evaluation supports analysis of entity-by-time datasets commonly produced by forecasting and readiness pipelines.

eb_evaluation.dataframe.panel

Panel-style evaluation output (DataFrame utilities).

This module provides a convenience wrapper that evaluates a DataFrame at multiple hierarchy levels and returns a long-form (tidy) panel suitable for reporting, plotting, and downstream aggregation.

The implementation delegates the core computation to eb_evaluation.dataframe.hierarchy.evaluate_hierarchy_df and then reshapes the wide per-level outputs into a single stacked table with:

  • a level column (which hierarchy level produced the row)
  • optional grouping key columns (depending on the level)
  • metric / value columns for tidy analysis

evaluate_panel_df(df, levels, actual_col, forecast_col, cu, co, tau=None)

Evaluate metrics at multiple levels and return a long-form panel DataFrame.

This is a convenience wrapper around eb_evaluation.dataframe.hierarchy.evaluate_hierarchy_df that:

  1. Computes a wide metrics DataFrame per hierarchy level.
  2. Stacks them into a single table with a level column.
  3. Melts metrics into metric / value pairs.

Parameters:

Name Type Description Default
df DataFrame

Input DataFrame containing at least actual_col and forecast_col plus any grouping columns referenced in levels.

 required
levels dict[str, Sequence[str]]

Mapping of level name to the column names used to group at that level.

Example:

levels = { ... "overall": [], ... "by_store": ["store_id"], ... "by_item": ["item_id"], ... "by_store_item": ["store_id", "item_id"], ... }

 required
actual_col str

Column name for actual demand / realized values.

 required
forecast_col str

Column name for forecast values.

 required
cu

Underbuild (shortfall) cost coefficient passed through to CWSL/FRS evaluations.

 required
co

Overbuild (excess) cost coefficient passed through to CWSL/FRS evaluations.

 required
tau float | None

Tolerance parameter for HR@tau. If None, HR@tau is omitted.

 None

Returns:

Type Description
DataFrame

Long-form (tidy) panel with columns:

  • level : hierarchy level name
  • <group cols> : the grouping keys for that level (may be empty for overall)
  • metric : metric name
  • value : metric value

Each row corresponds to a single metric evaluated at a specific level/group.
Notes
  • The set of metric columns is derived from the outputs of eb_evaluation.dataframe.hierarchy.evaluate_hierarchy_df. Only metrics present in the combined wide table are melted.
  • Grouping key columns vary by level. The returned panel includes the union of all grouping key columns across levels; levels that do not use a given key will have NaN in that column.