multi_objective

Module with functions to compute, compare, and optimize multi-objective forecasts.

score_backtest(y_true, y_preds, agg_method=None)

Return DataFrame of forecast metrics across entities.

Metrics returned:

• MAE
• MASE
• MSE
• Overforecast
• RMSE
• RMSSE
• SMAPE
• Underforecast

Note: MAPE is excluded to avoid potential divide by zero errors. We recommend looking at SMAPE instead.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_preds	DataFrame	Stacked predicted values across CV splits. DataFrame contains four columns: entity, time, target, "split".	required
agg_method	Optional[str] = None	Method ("mean", "median") to aggregate scores across entities by. If None, forecasts in overlapping splits are weighted equally, i.e. no aggregation is applied.	None

Returns:

Name	Type	Description
scores	DataFrame	DataFrame with computed metrics column by column across entities row by row.

score_forecast(y_true, y_pred, y_train)

Return DataFrame of forecast metrics across entities.

Metrics returned:

• MAE
• MASE
• MSE
• Overforecast
• RMSE
• RMSSE
• SMAPE
• Underforecast

Note: SMAPE is used instead of MAPE to avoid potential divide by zero errors.

Parameters:

Name	Type	Description	Default
y_true	DataFrame	Ground truth (correct) target values.	required
y_pred	DataFrame	Predicted values.	required
y_train	DataFrame	Observed training values.	required

Returns:

Name	Type	Description
scores	DataFrame	DataFrame with computed metrics column by column across entities row by row.

summarize_scores(scores, agg_method='mean')

Given a DataFrame of forecast metrics, return a dataclass of metrics aggregated by agg_method.

Parameters:

Name	Type	Description	Default
scores	DataFrame	DataFrame of scores. N rows of entities by M columns of metrics.	required
agg_method	str	Method ("mean", "median") to aggregate scores across entities by.	'mean'

Returns:

Name	Type	Description
metrics	Metrics	Dataclass of scores aggregated across entities.