evaluate
function that can be used in combination with loss functions from
utilsforecast.losses
.
Evaluate hierarchical forecast using different metrics.
Type | Default | Details | |
---|---|---|---|
df | FrameT | Forecasts to evaluate. Must have id_col , time_col , target_col and modelsβ predictions. | |
metrics | list | Functions with arguments df , models , id_col , target_col and optionally train_df . | |
tags | dict | Each key is a level in the hierarchy and its value contains tags associated to that level. | |
models | Optional | None | Names of the models to evaluate. If None will use every column in the dataframe after removing id, time and target. |
train_df | Optional | None | Training set. Used to evaluate metrics such as mase . |
level | Optional | None | Prediction interval levels. Used to compute losses that rely on quantiles. |
id_col | str | unique_id | Column that identifies each serie. |
time_col | str | ds | Column that identifies each timestep, its values can be timestamps or integers. |
target_col | str | y | Column that contains the target. |
agg_fn | Optional | mean | Statistic to compute on the scores by id to reduce them to a single number. |
benchmark | Optional | None | If passed, evaluators are scaled by the error of this benchmark model. |
Returns | FrameT | Metrics with one row per (id, metric) combination and one column per model. If agg_fn is not None , there is only one row per metric. |