Framework Integrations

This section documents framework-specific integration utilities provided by eb-metrics, enabling Electric Barometer metrics to be used directly in model training and selection workflows.

All content below is generated automatically from NumPy-style docstrings in the source code.

Framework Utilities

eb_metrics.frameworks

Framework integrations for Electric Barometer metrics.

The eb_metrics.frameworks package provides optional adapters that allow Electric Barometer metrics to plug into common machine-learning and forecasting workflows (training, model selection, and evaluation) without redefining metric semantics.

Currently supported integrations include:

• scikit-learn scorers (for model selection utilities such as GridSearchCV)
• Keras / TensorFlow loss functions (for training deep learning models)

Notes

• These integrations are thin wrappers around core Electric Barometer metrics. Metric definitions live in :mod:eb_metrics.metrics.
• Some integrations rely on optional third-party dependencies and may import those dependencies lazily (e.g., TensorFlow).

Conceptual definitions and interpretation are documented in the companion research repository (eb-papers).

make_cwsl_keras_loss(cu, co)

Create a Keras-compatible loss implementing Cost-Weighted Service Loss (CWSL).

This factory returns a TensorFlow/Keras loss function that mirrors the NumPy-based CWSL metric but is shaped to work in deep learning workflows.

The returned loss function has signature:

loss(y_true, y_pred) -> tf.Tensor

and returns a tensor of shape (batch_size,) (one loss value per example), which Keras will then reduce according to the configured reduction mode.

For each sample, CWSL is computed as:

\[ \begin{aligned} s &= \max(0, y - \hat{y}) \\ o &= \max(0, \hat{y} - y) \\ \mathrm{CWSL} &= \frac{c_u \sum s + c_o \sum o}{\sum y} \end{aligned} \]

where the summations reduce over the final axis (e.g., forecast horizon).

Parameters:

Name	Type	Description	Default
cu	float	Per-unit cost of underbuild (shortfall). Must be strictly positive.	required
co	float	Per-unit cost of overbuild (excess). Must be strictly positive.	required

Returns:

Type	Description
Callable	A Keras-compatible loss function loss(y_true, y_pred) returning a per-sample CWSL tensor of shape (batch_size,).

Raises:

Type	Description
ImportError	If TensorFlow is not installed.
ValueError	If cu or co are not strictly positive.

Notes

• Inputs are cast to tf.float32.
• The reduction is performed over the last axis. This supports both:
• (batch_size, horizon) tensors (typical forecasting setup), and
• (batch_size,) tensors (single-step forecasting), where reduction over the last axis still behaves correctly.
• Division by zero is avoided by clamping the per-sample demand denominator with Keras epsilon.

References

Electric Barometer Technical Note: Cost-Weighted Service Loss (CWSL).

cwsl_loss(y_true, y_pred, *, cu, co, sample_weight=None)

Compute the (positive) Cost-Weighted Service Loss (CWSL) for scikit-learn usage.

This helper is a thin wrapper around :func:eb_metrics.metrics.cwsl that:

• enforces strict positivity of cu and co (to align with typical scikit-learn scorer usage), and
• converts inputs to numpy.ndarray prior to evaluation.

Parameters:

Name	Type	Description	Default
y_true	array-like of shape (n_samples,)	Realized demand values.	required
y_pred	array-like of shape (n_samples,)	Forecast values.	required
cu	float	Per-unit cost of underbuild (shortfall). Must be strictly positive.	required
co	float	Per-unit cost of overbuild (excess). Must be strictly positive.	required
sample_weight	array-like of shape (n_samples,)	Optional per-sample weights passed through to CWSL.	None

Returns:

Type	Description
float	Positive CWSL value (lower is better).

Raises:

Type	Description
ValueError	If cu or co are not strictly positive, or if the underlying CWSL computation is undefined for the given inputs.

Notes

CWSL is defined as a demand-normalized asymmetric cost:

\[ \mathrm{CWSL} = \frac{\sum_i w_i \left(c_u s_i + c_o o_i\right)}{\sum_i w_i y_i}, \quad s_i = \max(0, y_i-\hat{y}_i), \quad o_i = \max(0, \hat{y}_i-y_i) \]

References

Electric Barometer Technical Note: Cost-Weighted Service Loss (CWSL).

cwsl_scorer(cu, co)

Build a scikit-learn scorer based on Cost-Weighted Service Loss (CWSL).

The returned object can be used wherever scikit-learn expects a scorer, for example:

• GridSearchCV(..., scoring=cwsl_scorer(cu=2.0, co=1.0))
• cross_val_score(..., scoring=cwsl_scorer(cu=2.0, co=1.0))

Parameters:

Name	Type	Description	Default
cu	float	Per-unit cost of underbuild (shortfall). Must be strictly positive.	required
co	float	Per-unit cost of overbuild (excess). Must be strictly positive.	required

Returns:

Type	Description
Callable	A scikit-learn scorer that returns negative CWSL (higher is better).

Notes

scikit-learn assumes scores are maximized. Because CWSL is a loss (lower is better), this scorer is configured with greater_is_better=False so scikit-learn negates the value internally.

In other words, the score returned by scikit-learn is:

\[ \text{score} = -\,\mathrm{CWSL} \]

References

Electric Barometer Technical Note: Cost-Weighted Service Loss (CWSL).