Aggregation/Visualization Utils

The HierarchicalForecast package contains utility functions to wrangle and visualize hierarchical series datasets. The aggregate function of the module allows you to create a hierarchy from categorical variables representing the structure levels, returning also the aggregation contraints matrix

\mathbf{S}

. In addition, HierarchicalForecast ensures compatibility of its reconciliation methods with other popular machine-learning libraries via its external forecast adapters that transform output base forecasts from external libraries into a compatible data frame format.

Aggregate Function

`aggregate`

aggregate(df, spec, exog_vars=None, sparse_s=False, id_col='unique_id', time_col='ds', id_time_col=None, target_cols=('y',))

Utils Aggregation Function. Aggregates bottom level series contained in the DataFrame df according to levels defined in the spec list. Parameters:

Name	Type	Description	Default
`df`	`Frame`	Dataframe with columns `[time_col, *target_cols]`, columns to aggregate and optionally exog_vars.	required
`spec`	`list[list[str]]`	list of levels. Each element of the list should contain a list of columns of `df` to aggregate.	required
`exog_vars`	`Optional[dict[str, Union[str, list[str]]]]`	dictionary of string keys & values that can either be a list of strings or a single string keys correspond to column names and the values represent the aggregation(s) that will be applied to each column. Accepted values are those from Pandas or Polars aggregation Functions, check the respective docs for guidance. Default is None.	`None`
`sparse_s`	`bool`	Return `S_df` as a sparse Pandas dataframe. Default is False.	`False`
`id_col`	`str`	Column that will identify each serie after aggregation. Default is “unique_id”.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Default is “ds”.	`‘ds’`
`id_time_col`	`Optional[str]`	Column that will identify each timestep after temporal aggregation. If provided, aggregate will operate temporally. Default is None.	`None`
`target_cols`	`Sequence[str]`	list of columns that contains the targets to aggregate. Default is (“y”,).	`(‘y’,)`

Returns:

Type	Description
`tuple[FrameT, FrameT, dict]`	tuple[FrameT, FrameT, dict]: Y_df, S_df, tags Y_df: Hierarchically structured series. S_df: Summing dataframe. tags: Aggregation indices.

`aggregate_temporal`

aggregate_temporal(df, spec, exog_vars=None, sparse_s=False, id_col='unique_id', time_col='ds', id_time_col='temporal_id', target_cols=('y',), aggregation_type='local')

Utils Aggregation Function for Temporal aggregations. Aggregates bottom level timesteps contained in the DataFrame df according to temporal levels defined in the spec list. Parameters:

Name	Type	Description	Default
`df`	`Frame`	Dataframe with columns `[time_col, target_cols]` and columns to aggregate.	required
`spec`	`dict[str, int]`	Dictionary of temporal levels. Each key should be a string with the value representing the number of bottom-level timesteps contained in the aggregation.	required
`exog_vars`	`Optional[dict[str, Union[str, list[str]]]]`	dictionary of string keys & values that can either be a list of strings or a single string keys correspond to column names and the values represent the aggregation(s) that will be applied to each column. Accepted values are those from Pandas or Polars aggregation Functions, check the respective docs for guidance. Default is None.	`None`
`sparse_s`	`bool`	Return `S_df` as a sparse Pandas dataframe. Default is False.	`False`
`id_col`	`str`	Column that will identify each serie after aggregation. Default is ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Default is ‘ds’.	`‘ds’`
`id_time_col`	`str`	Column that will identify each timestep after aggregation. Default is ‘temporal_id’.	`‘temporal_id’`
`target_cols`	`Sequence[str]`	List of columns that contain the targets to aggregate. Default is (‘y’,).	`(‘y’,)`
`aggregation_type`	`str`	If ‘local’ the aggregation will be performed on the timestamps of each timeseries independently. If ‘global’ the aggregation will be performed on the unique timestamps of all timeseries. Default is ‘local’.	`‘local’`

Returns:

Type	Description
`tuple[FrameT, FrameT, dict]`	tuple[FrameT, FrameT, dict]: Y_df, S_df, tags Y_df: Temporally hierarchically structured series. S_df: Temporal summing dataframe. tags: Temporal aggregation indices.

`make_future_dataframe`

make_future_dataframe(df, freq, h, id_col='unique_id', time_col='ds')

Create future dataframe for forecasting. Parameters:

Name	Type	Description	Default
`df`	`Frame`	Dataframe with ids, times and values for the exogenous regressors.	required
`freq`	`Union[str, int]`	Frequency of the data. Must be a valid pandas or polars offset alias, or an integer.	required
`h`	`int`	Forecast horizon.	required
`id_col`	`str`	Column that identifies each serie. Default is ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	Column that identifies each timestep, its values can be timestamps or integers. Default is ‘ds’.	`‘ds’`

Returns:

Name	Type	Description
`FrameT`	`FrameT`	DataFrame with future values.

`get_cross_temporal_tags`

get_cross_temporal_tags(df, tags_cs, tags_te, sep='//', id_col='unique_id', id_time_col='temporal_id', cross_temporal_id_col='cross_temporal_id')

Get cross-temporal tags. Parameters:

Name	Type	Description	Default
`df`	`Frame`	DataFrame with temporal ids.	required
`tags_cs`	`dict[str, ndarray]`	Tags for the cross-sectional hierarchies.	required
`tags_te`	`dict[str, ndarray]`	Tags for the temporal hierarchies.	required
`sep`	`str`	Separator for the cross-temporal tags. Default is ”//”.	`’//‘`
`id_col`	`str`	Column that identifies each series. Default is ‘unique_id’.	`‘unique_id’`
`id_time_col`	`str`	Column that identifies each (aggregated) timestep. Default is ‘temporal_id’.	`‘temporal_id’`
`cross_temporal_id_col`	`str`	Column that will identify each cross-temporal aggregation. Default is ‘cross_temporal_id’.	`‘cross_temporal_id’`

Returns:

Type	Description
`tuple[FrameT, dict[str, ndarray]]`	tuple[FrameT, dict[str, np.ndarray]]: df, tags_ct df: DataFrame with cross-temporal ids. tags_ct: Tags for the cross-temporal hierarchies.

Hierarchical Visualization

`HierarchicalPlot`

HierarchicalPlot(S, tags, S_id_col='unique_id')

Hierarchical Plot This class contains a collection of matplotlib visualization methods, suited for small to medium sized hierarchical series. Parameters:

Name	Type	Description	Default
`S`	`Frame`	DataFrame with summing matrix of size `(base, bottom)`, see aggregate function.	required
`tags`	`dict[str, ndarray]`	hierarchical aggregation indexes, where each key is a level and its value contains tags associated to that level.	required
`S_id_col`	`str`	column that identifies each aggregation. Default is ‘unique_id’.	`‘unique_id’`

`HierarchicalPlot.plot_summing_matrix`

plot_summing_matrix()

Summation Constraints plot This method simply plots the hierarchical aggregation constraints matrix

\\mathbf{S}

. Returns:

Name	Type	Description
`fig`	`Figure`	figure object containing the plot of the summing matrix.

`HierarchicalPlot.plot_series`

plot_series(series, Y_df, models=None, level=None, id_col='unique_id', time_col='ds', target_col='y')

Single Series plot Parameters:

Name	Type	Description	Default
`series`	`str`	string identifying the `'unique_id'` any-level series to plot.	required
`Y_df`	`Frame`	hierarchically structured series ( $\\mathbf{y}\_{[a,b]}$ ). It contains columns `['unique_id', 'ds', 'y']`, it may have `'models'`.	required
`models`	`Optional[list[str]]`	string identifying filtering model columns. Default is None.	`None`
`level`	`Optional[list[int]]`	confidence levels for prediction intervals available in `Y_df`. Default is None.	`None`
`id_col`	`str`	column that identifies each series. Default is ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	column that identifies each timestep, its values can be timestamps or integers. Default is ‘ds’.	`‘ds’`
`target_col`	`str`	column that contains the target. Default is ‘y’.	`‘y’`

Returns:

Name	Type	Description
`fig`	`Figure`	figure object containing the plot of the single series.

`HierarchicalPlot.plot_hierarchically_linked_series`

plot_hierarchically_linked_series(bottom_series, Y_df, models=None, level=None, id_col='unique_id', time_col='ds', target_col='y')

Hierarchically Linked Series plot Parameters:

Name	Type	Description	Default
`bottom_series`	`str`	string identifying the `'unique_id'` bottom-level series to plot.	required
`Y_df`	`Frame`	hierarchically structured series ( $\\mathbf{y}\_{[a,b]}$ ). It contains columns [‘unique_id’, ‘ds’, ‘y’] and models.	required
`models`	`Optional[list[str]]`	string identifying filtering model columns. Default is None.	`None`
`level`	`Optional[list[int]]`	confidence levels for prediction intervals available in `Y_df`. Default is None.	`None`
`id_col`	`str`	column that identifies each series. Default is ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	column that identifies each timestep, its values can be timestamps or integers. Default is ‘ds’.	`‘ds’`
`target_col`	`str`	column that contains the target. Default is ‘y’.	`‘y’`

Returns:

Name	Type	Description
`fig`	`Figure`	figure object containing the plots of the hierarchically linked series.

`HierarchicalPlot.plot_hierarchical_predictions_gap`

plot_hierarchical_predictions_gap(Y_df, models=None, xlabel=None, ylabel=None, id_col='unique_id', time_col='ds', target_col='y')

Hierarchically Predictions Gap plot Parameters:

Name	Type	Description	Default
`Y_df`	`Frame`	hierarchically structured series ( $\\mathbf{y}\_{[a,b]}$ ). It contains columns [‘unique_id’, ‘ds’, ‘y’] and models.	required
`models`	`Optional[list[str]]`	string identifying filtering model columns. Default is None.	`None`
`xlabel`	`Optional[str]`	string for the plot’s x axis label. Default is None.	`None`
`ylabel`	`Optional[str]`	string for the plot’s y axis label. Default is None.	`None`
`id_col`	`str`	column that identifies each series. Default is ‘unique_id’.	`‘unique_id’`
`time_col`	`str`	column that identifies each timestep, its values can be timestamps or integers. Default is ‘ds’.	`‘ds’`
`target_col`	`str`	column that contains the target. Default is ‘y’.	`‘y’`

Returns:

Name	Type	Description
`fig`	`Figure`	figure object containing the plot of the aggregated predictions at different levels of the hierarchical structure.

Example

from statsforecast.core import StatsForecast
from statsforecast.models import AutoETS
from datasetsforecast.hierarchical import HierarchicalData

Y_df, S, tags = HierarchicalData.load('./data', 'Labour')
Y_df['ds'] = pd.to_datetime(Y_df['ds'])
S = S.reset_index(names="unique_id")

Y_test_df  = Y_df.groupby('unique_id').tail(24)
Y_train_df = Y_df.drop(Y_test_df.index)

fcst = StatsForecast(
    models=[AutoETS(season_length=12, model='AAZ')],
    freq='MS',
    n_jobs=-1
)
Y_hat_df = fcst.forecast(df=Y_train_df, h=24).reset_index()

# Plot prediction difference of different aggregation
# Levels Country, Country/Region, Country/Gender/Region ...
hplots = HierarchicalPlot(S=S, tags=tags)

hplots.plot_hierarchical_predictions_gap(
    Y_df=Y_hat_df, models='AutoETS',
    xlabel='Month', ylabel='Predictions',
)

Getting Started

Tutorials

API Reference

Aggregation/Visualization Utils

Aggregate Function

`aggregate`

`aggregate_temporal`

`make_future_dataframe`

`get_cross_temporal_tags`

Hierarchical Visualization

`HierarchicalPlot`

`HierarchicalPlot.plot_summing_matrix`

`HierarchicalPlot.plot_series`

`HierarchicalPlot.plot_hierarchically_linked_series`

`HierarchicalPlot.plot_hierarchical_predictions_gap`

Example

Getting Started

Tutorials

API Reference

​Aggregate Function

​aggregate

​aggregate_temporal

​make_future_dataframe

​get_cross_temporal_tags

​Hierarchical Visualization

​HierarchicalPlot

​HierarchicalPlot.plot_summing_matrix

​HierarchicalPlot.plot_series

​HierarchicalPlot.plot_hierarchically_linked_series

​HierarchicalPlot.plot_hierarchical_predictions_gap

​Example

Aggregate Function

`aggregate`

`aggregate_temporal`

`make_future_dataframe`

`get_cross_temporal_tags`

Hierarchical Visualization

`HierarchicalPlot`

`HierarchicalPlot.plot_summing_matrix`

`HierarchicalPlot.plot_series`

`HierarchicalPlot.plot_hierarchically_linked_series`

`HierarchicalPlot.plot_hierarchical_predictions_gap`

Example